LogLG: Weakly Supervised Log Anomaly Detection via Log-Event Graph Construction
Authors:
Preprint
Abstract
Fully supervised log anomaly detection methods suffer the heavy burden of annotating massive unlabeled log data. Recently, many semi-supervised methods have been proposed to reduce annotation costs with the help of parsed templates. However, these methods consider each keyword independently, which disregards the correlation between keywords and the contextual relationships among log sequences. In this paper, we propose a novel weakly supervised log anomaly detection framework, named LogLG, to explore the semantic connections among keywords from sequences. Specifically, we design an end-to-end iterative process, where the keywords of unlabeled logs are first extracted to construct a log-event graph. Then, we build a subgraph annotator to generate pseudo labels for unlabeled log sequences. To ameliorate the annotation quality, we adopt a self-supervised task to pre-train a subgraph annotator. After that, a detection model is trained with the generated pseudo labels. Conditioned on the classification results, we re-extract the keywords from the log sequences and update the log-event graph for the next iteration. Experiments on five benchmarks validate the effectiveness of LogLG for detecting anomalies on unlabeled log data and demonstrate that LogLG, as the state-of-the-art weakly supervised method, achieves significant performance improvements compared to existing methods.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- LAnoBERT: System log anomaly detection based on BERT masked language modelApplied Soft Computing, 2023
- GeometryMotion-Transformer: An End-to-End Framework for 3D Action RecognitionIEEE Transactions on Multimedia, 2022
- LVP-M3: Language-aware Visual Prompt for Multilingual Multimodal Machine Translation2022
- Cross-Lingual Cross-Modal Consolidation for Effective Multilingual Video Corpus Moment Retrieval2022
- A2Log: Attentive Augmented Log Anomaly Detection2022
- Log-based Anomaly Detection Without Log Parsing2021
- Weakly-supervised Text Classification Based on Keyword Graph2021
- An Empirical Investigation of Incident Triage for Online Service Systems2019
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- Drain: An Online Log Parsing Approach with Fixed Depth Tree2017