MoEVD: Enhancing Vulnerability Detection by Mixture-of-Experts (MoE)
Authors:
- Xu YangXYXu Yang
- Shaowei WangSWShaowei Wang
- Jiayuan ZhouJZJiayuan Zhou
- Wenhan ZhuWZWenhan Zhu
Preprint
Abstract
Deep Learning-based Vulnerability Detection (DLVD) techniques have garnered significant interest due to their ability to automatically learn vulnerability patterns from previously compromised code. Despite the notable accuracy demonstrated by pioneering tools, the broader application of DLVD methods in real-world scenarios is hindered by significant challenges. A primary issue is the "one-for-all" design, where a single model is trained to handle all types of vulnerabilities. This approach fails to capture the patterns of different vulnerability types, resulting in suboptimal performance, particularly for less common vulnerabilities that are often underrepresented in training datasets. To address these challenges, we propose MoEVD, which adopts the Mixture-of-Experts (MoE) framework for vulnerability detection. MoEVD decomposes vulnerability detection into two tasks, CWE type classification and CWE-specific vulnerability detection. By splitting the task, in vulnerability detection, MoEVD allows specific experts to handle distinct types of vulnerabilities instead of handling all vulnerabilities within one model. Our results show that MoEVD achieves an F1-score of 0.44, significantly outperforming all studied state-of-the-art (SOTA) baselines by at least 12.8%. MoEVD excels across almost all CWE types, improving recall over the best SOTA baseline by 9% to 77.8%. Notably, MoEVD does not sacrifice performance on long-tailed CWE types; instead, its MoE design enhances performance (F1-score) on these by at least 7.3%, addressing long-tailed issues effectively.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Bridging the Gap: A Study of AI-based Vulnerability Management between Industry and Academia2024
- LIVABLE: Exploring Long-Tailed Classification of Software Vulnerability TypesIEEE Transactions on Software Engineering, 2024
- VulExplainer: A Transformer-Based Hierarchical Distillation for Explaining Vulnerability TypesIEEE Transactions on Software Engineering, 2023
- ME-SFP: A Mixture-of-Experts-Based Approach for Software Fault PredictionIEEE Transactions on Reliability, 2023
- Daily Streamflow Forecasting in Mountainous Catchment Using XGBoost, LightGBM and CatBoostHydrology, 2022
- Deep Learning Based Vulnerability Detection: Are We There Yet?IEEE Transactions on Software Engineering, 2021
- SySeVR: A Framework for Using Deep Learning to Detect Software VulnerabilitiesIEEE Transactions on Dependable and Secure Computing, 2021
- A comparative analysis of gradient boosting algorithmsArtificial Intelligence Review, 2020
- A Comprehensive Survey on Graph Neural NetworksIEEE Transactions on Neural Networks and Learning Systems, 2020
- Predicting Vulnerable Software Components via Text MiningIEEE Transactions on Software Engineering, 2014