An Empirical Study on Real Bugs for Machine Learning Programs
Authors:
- Xiaobing SunXSXiaobing Sun
- Tianchi ZhouTZTianchi Zhou
- Gengjie LiGLGengjie Li
- Jiajun HuJHJiajun Hu
- Hui YangHYHui Yang
- Bin LiBLBin Li
- 1 December 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 348-357
Abstract
Due to the availability of various open source Machine Learning (ML) tools and libraries, developers nowadays can easily implement their purposes by just invoking machine learning APIs without knowing the details of the algorithm. However, the owners of ML tools and libraries usually pay more attention to the correctness and functionality of their algorithm, while spending much less effort on maintaining their code and keeping their code at a high quality level. Considering the popularity of machine learning in today's world, low quality ML tools and libraries can have a huge impact on the software products that use ML algorithms. So in this paper, we conduct an empirical study on real machine learning bugs to examine their patterns and how they evolve over time. We collect three popular machine learning projects on Github, and manually analyzed 329 closed bugs from the perspectives of their bug category, fix pattern, fix scale, fix duration, and type of software maintenance. The results show that (1) there are seven categories of bugs in machine learning programs; (2) twelve different fix patterns are commonly used to fix the bugs; (3) 63.83% of the patches belong to micro-scale-fix and small-scale-fix, and 68.39% of the bugs are fixed within one month; (4) 47.77% of the bug fixes belong to corrective activity from the view of software maintenance.Keywords
This publication has 26 references indexed in Scilit:
- Machine learning for finding bugs: An initial report2017
- IPSETFUL: an iterative process of selecting test cases for effective fault localization by exploring concept lattice of program spectraFrontiers of Computer Science, 2016
- Cross-project defect prediction using a connectivity-based unsupervised classifier2016
- Heterogeneous cross-company defect prediction by unified metric representation and CCA-based transfer learning2015
- Utilizing a multi-developer network-based developer recommendation algorithm to fix bugs effectively2014
- Automatic recovery of root causes from bug-fixing changes2013
- An Empirical Study of Bugs in Software Build Systems2013
- An Empirical Analysis of Bug Reports and Bug Fixing in Open Source Android Apps2013
- R2Fix: Automatically Generating Bug Fixes from Bug Reports2013
- Reducing the effort of bug report triageACM Transactions on Software Engineering and Methodology, 2011