An Abnormal Event Classification Model for Big Data Platforms Based on Semi-supervised Learning
Authors:
- 16 December 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
With the increasing diversity of network attacks, the security of big data platforms is receiving more and more attention. To solve the problem of detecting and classifying attack events on unlabeled, multi-source heterogeneous big data platform log data, this paper proposes a semi-supervised security event detection and classification identification model based on a time-series detection algorithm and UEBA. First, based on data analysis and processing and security event knowledge base construction, a time series detection algorithm is used to detect anomalies in some log data. Based on the anomaly identification results, a fine-grained analysis guide rule encoding module is conducted to initially label the anomaly results. Then, semi-supervised learning is performed on a small amount of labeled data by the Pu Learning algorithm to train an optimized detection model to achieve anomaly identification of unlabeled data. Finally, based on the classification results, the XGBoost algorithm is further used to train the recognition results for multi-classification to enhance the real-time detection and prediction capability of subsequent related attacks. The experimental results show that the proposed model can effectively identify anomalous intrusion detection sequences and obtain better classification results.Keywords
Funding Information
- Natural Science Foundation of Fujian Province
This publication has 9 references indexed in Scilit:
- TransSentLog: Interpretable Anomaly Detection Using Transformer and Sentiment Analysis on Individual Log EventIEEE Access, 2023
- Interpretability-Aware Industrial Anomaly Detection Using AutoencodersIEEE Access, 2023
- A Fully Streaming Big Data Framework for Cyber Security Based on Optimized Deep Learning AlgorithmIEEE Access, 2023
- A novel anomaly detection approach based on ensemble semi-supervised active learning (ADESSA)Computers & Security, 2023
- An XGBoost-based multivariate deep learning framework for stock index futures price forecastingKybernetes, 2022
- Combining user behavioural information at the feature level to enhance continuous authentication systemsKnowledge-Based Systems, 2022
- Learning Discriminative Features for Semi-Supervised Anomaly Detection2021
- POSTER2017
- Anomaly Intrusion Detection for Evolving Data Stream Based on Semi-supervised Learning2009