Artificial Intelligence for IT Operations (AIOps) has emerged as a critical approach for managing complex IT systems, yet its advancement has been hindered by the lack of comprehensive public datasets for evaluation. This study addresses this challenge by introducing three large-scale, real-world benchmark datasets specifically designed for AIOps research and development. We present datasets covering key AIOps scenarios: key performance indicator (KPI) anomaly detection with diverse patterns from multiple Internet companies, multi-dimensional root cause localization with 400 labeled cases, and failure discovery and diagnosis containing 169 injected failures across various system components. These datasets are distinguished by their scale, authenticity, and detailed ground-truth labels, enabling rigorous evaluation of AIOps algorithms. The practical value of our datasets has been validated through three annual AIOps algorithm competitions (2018–2020), which attracted over 400 participating teams and led to numerous research publications. By providing these comprehensive benchmarks, our work establishes a foundation for reproducible AIOps research and facilitates fair comparison of different approaches, serving a role for AIOps analogous to that of ImageNet in computer vision.

As the scale of software systems expands, maintaining their stable operation has become an extraordinary challenge. System logs are semi-structured text generated by the recording function in the source code and have important research significance in software service anomaly detection. Existing log anomaly detection methods mainly focus on the statistical characteristics of logs, making it difficult to distinguish the semantic differences between normal and abnormal logs, and performing poorly on real-world industrial log data. In this paper, we propose an unsupervised framework for log anomaly detection based on generative pre-training-2 (GPT-2). We apply our approach to two industrial systems. The experimental results on two datasets show that our approach outperforms state-of-the-art approaches for log anomaly detection.