Evaluation of Synthetic Data Generation Techniques in the Domain of Anonymous Traffic Classification
Open Access
- 12 December 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 129612-129625
Abstract
Anonymous network traffic is more pervasive than ever due to the accessibility of services such as virtual private networks (VPN) and The Onion Router (Tor). To address the need to identify and classify this traffic, machine and deep learning solutions have become the standard. However, high-performing classifiers often scale poorly when applied to real-world traffic classification due to the heavily skewed nature of network traffic data. Prior research has found synthetic data generation to be effective at alleviating concerns surrounding class imbalance, though a limited number of these techniques have been applied to the domain of anonymous network traffic detection. This work compares the ability of a Conditional Tabular Generative Adversarial Network (CTGAN), Copula Generative Adversarial Network (CopulaGAN), Variational Autoencoder (VAE), and Synthetic Minority Over-sampling Technique (SMOTE) to create viable synthetic anonymous network traffic samples. Moreover, we evaluate the performance of several shallow boosting and bagging classifiers as well as deep learning models on the synthetic data. Ultimately, we amalgamate the data generated by the GANs, VAE, and SMOTE into a comprehensive dataset dubbed CMU-SynTraffic-2022 for future research on this topic. Our findings show that SMOTE consistently outperformed the other upsampling techniques, improving classifiers’ F1-scores over the control by ~7.5% for application type characterization. Among the tested classifiers, Light Gradient Boosting Machine achieved the highest F1-score of 90.3% on eight application types.Keywords
Funding Information
- State of Colorado through funds appropriated for Cybersecurity Law Dubbed “Cyber Coding Cryptology for State Records”
This publication has 32 references indexed in Scilit:
- End-to-end encrypted traffic classification with one-dimensional convolution neural networks2017
- Characterization of Tor Traffic using Time based Features2017
- The Synthetic Data Vault2016
- Characterization of Encrypted and VPN Traffic using Time-related Features2016
- SMOTE for high-dimensional class-imbalanced dataBMC Bioinformatics, 2013
- Using chi-square statistics to measure similarities for text categorizationExpert Systems with Applications, 2011
- Propensity score estimation: neural networks, support vector machines, decision trees (CART), and meta-classifiers as alternatives to logistic regressionJournal of Clinical Epidemiology, 2010
- A Modified Kolmogorov–Smirnov Test for NormalityCommunications in Statistics - Simulation and Computation, 2010
- SMOTE: Synthetic Minority Over-sampling TechniqueJournal of Artificial Intelligence Research, 2002
- Using Iterated Bagging to Debias RegressionsMachine Learning, 2001