LogLS: Research on System Log Anomaly Detection Method Based on Dual LSTM
Open Access
Abstract
System logs record the status and important events of the system at different time periods. They are important resources for administrators to understand and manage the system. Detecting anomalies in logs is critical to identifying system faults in time. However, with the increasing size and complexity of today’s software systems, the number of logs has exploded. In many cases, the traditional manual log-checking method becomes impractical and time-consuming. On the other hand, existing automatic log anomaly detection methods are error-prone and often use indices or log templates. In this work, we propose LogLS, a system log anomaly detection method based on dual long short-term memory (LSTM) with symmetric structure, which regarded the system log as a natural-language sequence and modeled the log according to the preorder relationship and postorder relationship. LogLS is optimized based on the DeepLog method to solve the problem of poor prediction performance of LSTM on long sequences. By providing a feedback mechanism, it implements the prediction of logs that do not appear. To evaluate LogLS, we conducted experiments on two real datasets, and the experimental results demonstrate the effectiveness of our proposed method in log anomaly detection.Keywords
This publication has 28 references indexed in Scilit:
- Proactive Critical Energy Infrastructure Protection via Deep Feature LearningEnergies, 2020
- Robust log-based anomaly detection on unstable log data2019
- Tools and Benchmarks for Automated Log Parsing2019
- Trimmed categorical cross‐entropy for deep learning with label noiseElectronics Letters, 2019
- Spell: Online Streaming Parsing of Large Unstructured System LogsIEEE Transactions on Knowledge and Data Engineering, 2018
- Execution anomaly detection in large-scale systems through console log analysisJournal of Systems and Software, 2018
- Drain: An Online Log Parsing Approach with Fixed Depth Tree2017
- Log clustering based problem identification for online service systems2016
- Detection of Early-Stage Enterprise Infection by Mining Large-Scale Log Data2015
- Long Short-Term MemoryNeural Computation, 1997