IHUD-BERT: A Large-Scale Network Traffic Classification Method Based on Pre-Training Transformers and Knowledge Distillation
- 22 May 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive Communications and Networking
- Vol. 12, 8478-8491
Abstract
Accurate and efficient classification of large-scale network traffic is fundamental to ensuring network Quality of Service (QoS) and security. However, existing methodologies face critical challenges. While payload-based pre-trained models have achieved success, they often rely on memorizing session-specific encryption artifacts, leading to catastrophic performance degradation under realistic “inter-flow” splitting scenarios. Furthermore, the scarcity of labeled samples and the high computational cost of large models hinder their practical deployment. To address these limitations, this paper proposes an integrated framework, IHUD-BERT, which integrates robust feature representation with knowledge distillation. We first introduce the IP Header Unit (IHU) scheme, applying a sliding window over IP packet headers. Unlike payload-based approaches, this method captures invariant behavioral patterns that remain consistent across encrypted sessions, ensuring robustness against encryption randomization while augmenting limited training data. Subsequently, a pre-trained Transformer (IHU-BERT) is constructed as a teacher model, and its comprehensive knowledge is transferred to a lightweight student model (IHUD-BERT) via hierarchical knowledge distillation. Comprehensive evaluations on multiple datasets validate our approach. The results demonstrate that while payload-based baselines collapse under strict inter-flow evaluation on encrypted traffic, our method maintains high accuracy. Notably, the lightweight IHUD-BERT achieves only a marginal performance degradation compared to its teacher counterpart while delivering millisecond-level inference latency. These results establish the proposed framework as a robust, high-precision, and efficient solution for modern encrypted traffic classification.Keywords
Funding Information
- 2026 Enhancement of Independent Innovation - Disciplinary Interdisciplinary Innovation Project through the project Construction of Multi-mode Fusion Communication Platform for Mine Leaky Cable (81001203A56)
- Fundamental Research Funds for the Central Universities (2025ZKPYZN02)
- National Key Research and Development Program of China (2024YFB2908700)
- Chinese Academy of Sciences (2021168)
- Guoneng Zhishen Control Technology Co., Ltd. through the project Research and Development of Boundary-aware Intelligent Coal Sorting Technology Using Weakly Supervised Training
This publication has 25 references indexed in Scilit:
- Online network traffic classification based on external attention and convolution by IP packet headerComputer Networks, 2024
- Tree-RNN: Tree structural recurrent neural network for network traffic classificationExpert Systems with Applications, 2021
- A review on machine learning–based approaches for Internet traffic classificationAnnals of Telecommunications, 2020
- Semi-Supervised Encrypted Traffic Classification With Deep Convolutional Generative Adversarial NetworksIEEE Access, 2020
- Large-Scale Mobile App Identification Using Deep LearningIEEE Access, 2019
- A Traffic Classification Method Based on Packet Transport Layer Payload by Ensemble Learning2019
- Deep packet: a novel approach for encrypted traffic classification using deep learningSoft Computing, 2019
- Deep learning for time series classification: a reviewData Mining and Knowledge Discovery, 2019
- Toward Developing a Systematic Approach to Generate Benchmark Android Malware Datasets and Classification2018
- AppScanner: Automatic Fingerprinting of Smartphone Apps from Encrypted Network Traffic2016