LogFormer: A Pre-train and Tuning Pipeline for Log Anomaly Detection
Authors:
Preprint
Abstract
Log anomaly detection is a key component in the field of artificial intelligence for IT operations (AIOps). Considering log data of variant domains, retraining the whole network for unknown domains is inefficient in real industrial scenarios. However, previous deep models merely focused on extracting the semantics of log sequences in the same domain, leading to poor generalization on multi-domain logs. To alleviate this issue, we propose a unified Transformer-based framework for Log anomaly detection (LogFormer) to improve the generalization ability across different domains, where we establish a two-stage process including the pre-training and adapter-based tuning stage. Specifically, our model is first pre-trained on the source domain to obtain shared semantic knowledge of log data. Then, we transfer such knowledge to the target domain via shared parameters. Besides, the Log-Attention module is proposed to supplement the information ignored by the log-paring. The proposed method is evaluated on three public and one real-world datasets. Experimental results on multiple benchmarks demonstrate the effectiveness of our LogFormer with fewer trainable parameters and lower training costs.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- PLELog: Semi-Supervised Log-Based Anomaly Detection via Probabilistic Label Estimation2021
- HitAnomaly: Hierarchical Transformers for Anomaly Detection in System LogIEEE Transactions on Network and Service Management, 2020
- A study of the performance of general compressors on log filesEmpirical Software Engineering, 2020
- Detecting Anomaly in Big Data System Logs Using Convolutional Neural Network2018
- Drain: An Online Log Parsing Approach with Fixed Depth Tree2017
- Spell: Streaming Parsing of System Event Logs2016
- Automated IT system failure prediction: A deep learning approach2016
- Clustering event logs using iterative partitioning2009
- Abstracting Execution Logs to Execution Events for Enterprise Applications (Short Paper)2008
- What Supercomputers Say: A Study of Five System Logs2007