Sachin Lodha

Cyber Threat Intelligence (CTI) utilizes information from various sources, necessitating high-quality labeled datasets for effective application of machine learning. Our study addresses the often-overlooked issue of labeling errors in cybersecurity benchmarks, resulting in the creation of D-LADDER++, a curated version of the recently published LADDER dataset. We evaluated the performance of both an open-source model (Microsoft Phi-3) and a closed-source model (Google Gemini) on D-LADDER++. We assessed their zero-shot and few-shot capabilities and fine-tuned the Phi-3 model for enhanced adaptability. Our assessment of the impact of test errors on model performance emphasizes the critical need for robust benchmarks in cybersecurity to ensure accurate model evaluation and selection.