De-Decay: Defusing Computer Vision Model Degradation through Scalable and Actionable Human-Data Alignment
Open Access
- 11 May 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Interactive Intelligent Systems
- Vol. 16 (2) , 1-34
Abstract
Computer Vision (CV) models can become outdated after deployment as real-world data evolves, requiring intensive attention from AI engineers to address degraded performance through tasks like data relabeling to update models with new human perceptions. Interactive human-in-the-loop systems have considerable potential to enhance model-steering practices. However, such workflows reveal two challenges: (1) scalability, where labor demands increase with data size, and (2) actionability, where human insights do not readily transform into model revisions. Based on our formative study (S1) on the current challenges faced by CV professionals, we developed De-Decay, an end-to-end Human-Data Alignment system offering scalable label-less assessment and actionable insight transformation . This enables engineers to investigate degradation and auto-retrain models with AI support, such as image clustering and regeneration. Our summative study (S2) showed that De-Decay helped engineers effectively identify and address CV degradation. We discuss how future research can enhance scalability and actionability in AI evaluation systems for aligning AI behaviors with human mental models.Keywords
This publication has 89 references indexed in Scilit:
- A survey of human-in-the-loop for machine learningFuture Generation Computer Systems, 2022
- From concept drift to model degradation: An overview on performance-aware drift detectorsKnowledge-Based Systems, 2022
- Machine learning and reasoning for predictive maintenance in Industry 4.0: Current status and challengesComputers in Industry, 2020
- Ada-boundary: accelerating DNN training via adaptive boundary batch selectionMachine Learning, 2020
- Validity of a low-cost Lichtenstein open inguinal hernia repair simulation model for surgical trainingHernia, 2019
- Initiating predictive maintenance for a conveyor motor in a bottling plant using industry 4.0 conceptsThe International Journal of Advanced Manufacturing Technology, 2018
- Characterizing concept driftData Mining and Knowledge Discovery, 2016
- Towards a comprehensive model of the cognitive process and mechanisms of individual sensemakingJournal of the Association for Information Science and Technology, 2014
- Microsoft COCO: Common Objects in Context2014
- The Hawthorne effect: A reconsideration of the methodological artifact.Journal of Applied Psychology, 1984