Weakly-Supervised Log-Based Anomaly Detection with Inexact Labels via Multi-Instance Learning
Authors:
- 26 April 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2918-2930
Abstract
Log-based anomaly detection is essential for maintaining software availability. However, existing log-based anomaly detection approaches heavily rely on fine-grained exact labels of log entries which are very hard to obtain in real-world systems. This brings a key problem that anomaly detection models require supervision signals while labeled log entries are unavailable. Facing this problem, we propose a new labeling strategy called inexact labeling that instead of labeling an log entry, system experts can label a bag of log entries in a time span. Furthermore, we propose MIDLog, a weakly supervised log-based anomaly detection approach with inexact labels. We leverage the multiinstance learning paradigm to achieve explicit separation of anomalous log entries from the inexact labeled anomalous log set so as to deduce exact anomalous log labels from inexact labeled log sets. Extensive evaluation on three public datasets shows that our approach achieves an F1 score of over 85% with inexact labels.Keywords
This publication has 61 references indexed in Scilit:
- Hilogx: noise-aware log-based anomaly detection with human feedbackThe VLDB Journal, 2024
- FAIR: A Causal Framework for Accurately Inferring Judgments Reversals2023
- Weakly supervised machine learningCAAI Transactions on Intelligence Technology, 2023
- LogLG: Weakly Supervised Log Anomaly Detection via Log-Event Graph Construction2023
- Log-based Anomaly Detection Without Log Parsing2021
- Real-World Anomaly Detection in Surveillance Videos2018
- Multiple instance learning: A survey of problem characteristics and applicationsPattern Recognition, 2018
- LogSed: Anomaly Diagnosis through Mining Time-Weighted Control Flow Graph in Logs2017
- An Approach to Pinpointing Bug-Induced Failure in Logs of Open Cloud Platforms2016
- Multiple instance learning for soft bags via top instances2015