LAMBERT: Leveraging Attention Mechanisms to Improve the BERT Fine-Tuning Model for Encrypted Traffic Classification
Open Access
- 22 May 2024
- journal article
- research article
- Published by MDPI AG in Mathematics
- Vol. 12 (11) , 1624
Abstract
Encrypted traffic classification is a crucial part of privacy-preserving research. With the great success of artificial intelligence technology in fields such as image recognition and natural language processing, how to classify encrypted traffic based on AI technology has become an attractive topic in information security. With good generalization ability and high training accuracy, pre-training-based encrypted traffic classification methods have become the first option. The accuracy of this type of method depends highly on the fine-tuning model. However, it is a challenge for existing fine-tuned models to effectively integrate the representation of packet and byte features extracted via pre-training. A novel fine-tuning model, LAMBERT, is proposed in this article. By introducing an attention mechanism to capture the relationship between BiGRU and byte sequences, LAMBERT not only effectively improves the sequence loss phenomenon of BiGRU but also improves the processing performance of encrypted stream classification. LAMBERT can quickly and accurately classify multiple types of encrypted traffic. The experimental results show that our model performs well on datasets with uneven sample distribution, no pre-training, and large sample classification. LAMBERT was tested on four datasets, namely, ISCX-VPN-Service, ISCX-VPN-APP, USTC-TFC and CSTNET-TLS 1.3, and the F1 scores reached 99.15%, 99.52%, 99.30%, and 97.41%, respectively.Keywords
Funding Information
- the national key Research and Development Program of China (2021YFB2012402, PCL2023A07-4)
- The Major Key Project of PCL (2021YFB2012402, PCL2023A07-4)
This publication has 39 references indexed in Scilit:
- Short-term multi-energy load forecasting for integrated energy systems based on CNN-BiGRU optimized by attention mechanismApplied Energy, 2022
- Tree-RNN: Tree structural recurrent neural network for network traffic classificationExpert Systems with Applications, 2021
- MIMETIC: Mobile encrypted traffic classification using multimodal deep learningComputer Networks, 2019
- Deep packet: a novel approach for encrypted traffic classification using deep learningSoft Computing, 2019
- Period-aware content attention RNNs for time series forecasting with missing valuesNeurocomputing, 2018
- An efficient feature generation approach based on deep learning and feature selection techniques for traffic classificationComputer Networks, 2018
- Predicting electricity consumption for commercial and residential buildings using deep recurrent neural networksApplied Energy, 2018
- A survey of methods for encrypted traffic classification and analysisInternational Journal of Network Management, 2015
- Independent comparison of popular DPI tools for traffic classificationComputer Networks, 2015
- Catalytic mechanism of F1-ATPaseBiochimica et Biophysica Acta (BBA) - Bioenergetics, 1997