Cost-Stability Evaluation Framework for Reliable Deployment of Object Detectors in Industrial Applications
Open Access
- 26 November 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 207566-207580
Abstract
The practical deployment of object detectors is often hindered by a gap between standard academic metrics and real-world requirements—namely, performance at a specific operating threshold, cost-asymmetric errors, and performance stability under distribution shifts. To bridge this gap, we introduce the Cost-Stability Evaluation Framework (CSEF), a systematic workflow that transforms the selection of a model and its operating threshold from a heuristic practice to a rational, data-driven decision process. CSEF operationalizes user-defined requirements—an IoU threshold (τ) for localization accuracy; a cost asymmetry parameter (β) that serves as the recall-to-precision weight in the F-beta score; and a maximum tolerable performance degradation (σreq)—to algorithmically recommend the most operationally reliable model–threshold pair. CSEF first selects the threshold that maximizes the F-beta score on the validation set, and then quantifies robustness as the relative change in the F-beta score, σdeg, from the validation set to an unseen held-out test set. Candidates that pass a conservative stability gate—where the bootstrap upper bound of |σdeg| does not exceed σreq—are then ranked by their operational F-beta score at the fixed threshold. CSEF’s utility is demonstrated across four industrial scenarios, from medical imaging to semiconductor manufacturing, highlighting its adaptability to domain-specific requirements. Ultimately, CSEF provides a practical and auditable methodology for deploying object detectors with predictable performance and reliability, thereby helping close the long-standing gap between academic evaluation and industrial deployment.Keywords
Funding Information
- Samsung
This publication has 23 references indexed in Scilit:
- YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information2024
- High sensitivity methods for automated rib fracture detection in pediatric radiographsScientific Reports, 2024
- Localization and classification of space objects using EfficientDet detector for space situational awarenessScientific Reports, 2022
- Ransomware Classification and Detection With Machine Learning Algorithms2022
- Deep Learning-Based Workers Safety Helmet Wearing Detection on Construction Sites Using Multi-Scale FeaturesIEEE Access, 2021
- Polyp Detection in Colonoscopy Videos2021
- Robust and Accurate Object Detection via Adversarial Learning2021
- Tversky Loss Function for Image Segmentation Using 3D Fully Convolutional Deep Networks2017
- Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors2017
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009