BERT-Log: Anomaly Detection for System Logs Based on Pre-trained Language Model
Authors:
- 17 November 2022
- journal article
- research article
- Published by Taylor & Francis in Applied Artificial Intelligence
- Vol. 36 (1)
Abstract
Logs are primary information resource for fault diagnosis and anomaly detection in large-scale computer systems, but it is hard to classify anomalies from system logs. Recent studies focus on extracting semantic information from unstructured log messages and converting it into word vectors. Therefore, LSTM approach is more suitable for time series data. Word2Vec is the up-to-date encoding method, but the order of words in sequences is not taken into account. In this article, we propose BERT-Log, which regards the log sequence as a natural language sequence, use pre-trained language model to learn the semantic representation of normal and anomalous logs, and a fully connected neural network is utilized to fine-tune the BERT model to detect abnormal. It can capture all the semantic information from log sequence including context and position. It has achieved the highest performance among all the methods on HDFS dataset, with an F1-score of 99.3%. We propose a new log feature extractor on BGL dataset to obtain log sequence by sliding window including node ID, window size and step size. BERT-Log approach detects anomalies on BGL dataset with an F1-score of 99.4%. It gives 19% performance improvement compared to LogRobust and 7% performance improvement compared to HitAnomaly.Keywords
Funding Information
- CDTU PHD FUND (2020RC002)
This publication has 39 references indexed in Scilit:
- Spell: Streaming Parsing of System Event Logs2016
- Trap Analysis: an automated approach for deriving column height predictions in fault-bounded trapsPetroleum Geoscience, 2016
- LSTM: A Search Space OdysseyIEEE Transactions on Neural Networks and Learning Systems, 2016
- Beehive2013
- Localizing root causes of performance anomalies in cloud computing systems by analyzing request trace logsScience China Information Sciences, 2012
- LogSig2011
- Automated anomaly detection and performance modeling of enterprise applicationsACM Transactions on Computer Systems, 2009
- Detecting large-scale system problems by mining console logs2009
- Failure prediction in IBM BlueGene/L event logs2008 IEEE International Symposium on Parallel and Distributed Processing, 2008
- What Supercomputers Say: A Study of Five System Logs2007