Abstract
Motivation: As enterprise information systems collect increasingly diverse event streams, the automatic detection of anomalous events is essential for applications like network security, system monitoring, and fraud detection.Methods: This study aims to develop robust techniques for automatically and computationally efficiently identifying anomalous behavior within high-dimensional event sequences, where events occur at specific timestamps (e.g., number of events per hour). The proposed frameworkleverages machine learning to detect anomalies within high-dimensional, high-frequency event sequence time series. By dynamically grouping time series based on shared categorical dimensions, the framework enables scalable anomaly detection across varying levels of aggregation, while maintaining detection sensitivity and specificity.Results: The framework has been validated using real-world telecommunications event data sets and evaluated against the case where no aggregation is performed (‘fully disaggregated’). In the case of our ‘minimum aggregation’ method, the training time is reduced by a factor of∼ 300, and inference time is reduced by a factor of 8. For our other method, ‘ideal aggregation’, the training time is reduced by a factor of 6.5, and the inference time is reduced by a factor of 7.6. The quality of anomaly detection is generally preserved across different aggregation levels of grouped high-dimensional data.Conclusions: This research advances anomaly detection techniques in grouped high-dimensional event sequence data, a complex yet increasingly common data format in fields like network security and system monitoring. The framework’s dimensional aggregation techniques enable efficient pattern recognition while maintaining detection sensitivity and specificity, critical for applications requiring faster anomaly detection response.

This publication has 21 references indexed in Scilit: