Farnaz Soltaniani
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Farnaz Soltaniani has a total of 4 co-authors. Here are the top co-authors by number of shared publications:
- Mohammad Ghafari
Technische Universität Clausthal Germany
2 shared publications - Mohammad Ghafari
Technische Universitt Clausthal,Germany
1 shared publication - Mohammed Sayagh
ETS Montreal, University of Quebec, Montreal, Canada
1 shared publication - Shoaib Razzaq
Technische Universität Clausthal Germany Technische Universität Clausthal Germany Technische Universität Clausthal Germany
1 shared publication
Latest Publications
- Open Access
Evaluating Large Language Models for Security Bug Report Prediction
Early detection of security bug reports (SBRs) is critical for timely vulnerability mitigation. We present an evaluation of prompt-based engineering and fine-tuning approaches for predicting SBRs using Large Language Models (LLMs). Our findings reveal a distinct trade-off between the two approaches. Prompted proprietary models demonstrate the highest sensitivity to SBRs, achieving a G-measure of 77% and a recall of 74% on average across all the datasets, albeit at the cost of a higher false-positive rate, resulting in an average precision of only 22%. Fine-tuned models, by contrast, exhibit the opposite behavior, attaining a lower overall G-measure of 51% but substantially higher precision of 75% at the cost of reduced recall of 36%. Though a one-time investment in building fine-tuned models is necessary, the inference on the largest dataset is up to 50 times faster than that of proprietary models. These findings suggest that further investigations to harness the power of LLMs for SBR prediction are necessary.
Open AccessEvaluating Large Language Models for Security Bug Report Prediction
Early detection of security bug reports (SBRs) is critical for timely vulnerability mitigation. We present an evaluation of prompt-based engineering and fine-tuning approaches for predicting SBRs using Large Language Models (LLMs). Our findings reveal a distinct trade-off between the two approaches. Prompted proprietary models demonstrate the highest sensitivity to SBRs, achieving a G-measure of 77% and a recall of 74% on average across all the datasets, albeit at the cost of a higher false-positive rate, resulting in an average precision of only 22%. Fine-tuned models, by contrast, exhibit the opposite behavior, attaining a lower overall G-measure of 51% but substantially higher precision of 75% at the cost of reduced recall of 36%. Though a one-time investment in building fine-tuned models is necessary, the inference on the largest dataset is up to 50 times faster than that of proprietary models. These findings suggest that further investigations to harness the power of LLMs for SBR prediction are necessary.
- Open Access
From Data Leak to Secret Misses: The Impact of Data Leakage on Secret Detection Models
Machine learning models are increasingly used for software security tasks. These models are commonly trained and evaluated on large Internet-derived datasets, which often contain duplicated or highly similar samples. When such samples are split across training and test sets, data leakage may occur, allowing models to memorize patterns instead of learning to generalize. We investigate duplication in a widely used benchmark dataset of hard coded secrets and show how data leakage can substantially inflate the reported performance of AI-based secret detectors, resulting in a misleading picture of their real-world effectiveness.
Open AccessFrom Data Leak to Secret Misses: The Impact of Data Leakage on Secret Detection Models
Machine learning models are increasingly used for software security tasks. These models are commonly trained and evaluated on large Internet-derived datasets, which often contain duplicated or highly similar samples. When such samples are split across training and test sets, data leakage may occur, allowing models to memorize patterns instead of learning to generalize. We investigate duplication in a widely used benchmark dataset of hard coded secrets and show how data leakage can substantially inflate the reported performance of AI-based secret detectors, resulting in a misleading picture of their real-world effectiveness.
- Open Access
- 26 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 85-94
Security Bug Report Prediction Within and Across Projects: A Comparative Study of BERT and Random Forest
- 26 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 85-94
Open AccessSecurity Bug Report Prediction Within and Across Projects: A Comparative Study of BERT and Random Forest