Artificial Intelligence for IT Operations (AIOps) has emerged as a critical approach for managing complex IT systems, yet its advancement has been hindered by the lack of comprehensive public datasets for evaluation. This study addresses this challenge by introducing three large-scale, real-world benchmark datasets specifically designed for AIOps research and development. We present datasets covering key AIOps scenarios: key performance indicator (KPI) anomaly detection with diverse patterns from multiple Internet companies, multi-dimensional root cause localization with 400 labeled cases, and failure discovery and diagnosis containing 169 injected failures across various system components. These datasets are distinguished by their scale, authenticity, and detailed ground-truth labels, enabling rigorous evaluation of AIOps algorithms. The practical value of our datasets has been validated through three annual AIOps algorithm competitions (2018–2020), which attracted over 400 participating teams and led to numerous research publications. By providing these comprehensive benchmarks, our work establishes a foundation for reproducible AIOps research and facilitates fair comparison of different approaches, serving a role for AIOps analogous to that of ImageNet in computer vision.

In recent years, microservice architecture has gained increasing popularity. However, due to the complex and dynamically changing nature of microservice systems, failure detection has become more challenging. Traditional root cause analysis methods mostly rely on a single modality of data, which is insufficient to cover all failure information. Existing multimodal methods require collecting high-quality labeled samples and often face challenges in classifying unknown failure categories. To address these challenges, this paper proposes a root cause analysis framework based on a masked graph autoencoder (GAE). The main process involves feature extraction, feature dimensionality reduction based on GAE, and online clustering combined with expert input. The method is experimentally evaluated on two public datasets and compared with two baseline methods, demonstrating significant advantages even with 16% labeled samples.

A holistic analysis of problem and incident tickets in a real production cloud service environment is presented in this paper. By extracting different bags of words, we use principal component analysis (PCA) to examine the clustering characteristics of these tickets. Then K-means and latent Dirichlet allocation (LDA) are applied to show the potential clusters within this Cloud environment. The second part of our study uses a pre-trained bidirectional encoder representation from transformers (BERT) model to classify the tickets, with the goal of predicting the optimal dispatching department for a given ticket. Experimental results show that due to the unique characteristics of ticket description, pre-processing with domain knowledge turns out to be critical in both clustering and classification. Our classification model yields 86% accuracy when predicting the target dispatching department.

As the scale of software systems expands, maintaining their stable operation has become an extraordinary challenge. System logs are semi-structured text generated by the recording function in the source code and have important research significance in software service anomaly detection. Existing log anomaly detection methods mainly focus on the statistical characteristics of logs, making it difficult to distinguish the semantic differences between normal and abnormal logs, and performing poorly on real-world industrial log data. In this paper, we propose an unsupervised framework for log anomaly detection based on generative pre-training-2 (GPT-2). We apply our approach to two industrial systems. The experimental results on two datasets show that our approach outperforms state-of-the-art approaches for log anomaly detection.

Microservices have become popular in enterprises because of their excellent scalability and timely update capabilities. However, while fine-grained modularity and service-orientation decrease the complexity of system development, the complexity of system operation and maintenance has been greatly increased, on the contrary. Multiple types of system failures occur frequently, and it is hard to detect and diagnose failures in time. Furthermore, microservices are updated frequently. Existing anomaly detection models depend on offline training and cannot adapt to the frequent updates of microservices. This paper proposes an anomaly detection approach for microservice systems with multi-source data streams. This approach realizes online model construction and online anomaly detection, and is capable of self-updating and self-adapting. Experimental results show that this approach can correctly identify 78.85% of faults of different types.

One particular challenge for large?scale software systems is anomaly detection. System logs are a straightforward and common source of information for anomaly detection. Existing log?based anomaly detectors are unusable in real?world industrial systems due to high false?positive rates. In this paper, we incorporate human feedback to adjust the detection model structure to reduce false positives. We apply our approach to two industrial large?scale systems. Results have shown that our approach performs much better than state?of?the-art works with 50% higher accuracy. Besides, human feedback can reduce more than 70% of false positives and greatly improve detection precision.

The emerging fifth generation (5G) network has the potential to satisfy the rapidly growing traffic demand and promote the transformation of smartphone-centric networks into an Internet of Things (IoT) ecosystem. Due to the introduction of new communication technologies and the increased density of 5G cells, the complexity of operation and operational expenditure (OPEX) will become very challenging in 5G. Self-organizing network (SON) has been researched extensively since 2G, to cope with the similar challenge, however by predefined policies, rather than intelligent analysis. The requirement for better quality of experience and the complexity of 5G network demands call for an approach that is different from SON. In several recent studies, the combination of machine learning (ML) technology with SON has been investigated. In this paper, we focus on the intelligent operation of wireless network through ML algorithms. A comprehensive and flexible framework is proposed to achieve an intelligent operation system. Two use cases are also studied to use ML algorithms to automate the anomaly detection and fault diagnosis of key performance indicators (KPIs) in wireless networks. The effectiveness of the proposed ML algorithms is demonstrated by the real data experiments, thus encouraging the further research for intelligent wireless network operation.