Yuhua Qi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Yuhua Qi has a total of 9 co-authors. Here are the top co-authors by number of shared publications:
Xiaoguang Mao
College of Computer, National University of Defense Technology, Changsha, China
5 shared publicationsYan Lei
National University of Defense Technology, China
4 shared publicationsChengsong Wang
National University of Defense Technology, China
2 shared publicationsDeheng Yang
College of Computer, National University of Defense Technology, Changsha, China
2 shared publications- Xiaoguang Mao
National University of Defense Technology
2 shared publications - Yan Lei
Chongqing University
2 shared publications
Latest Publications
- Open Access
- 27 April 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 32 (3) , 1-37
Seeing the Whole Elephant: Systematically Understanding and Uncovering Evaluation Biases in Automated Program Repair
Evaluation is the foundation of automated program repair (APR), as it provides empirical evidence on strengths and weaknesses of APR techniques. However, the reliability of such evaluation is often threatened by various introduced biases. Consequently, bias exploration, which uncovers biases in the APR evaluation, has become a pivotal activity and performed since the early years when pioneer APR techniques were proposed. Unfortunately, there is still no methodology to support a systematic comprehension and discovery of evaluation biases in APR, which impedes the mitigation of such biases and threatens the evaluation of APR techniques. In this work, we propose to systematically understand existing evaluation biases by rigorously conducting the first systematic literature review on existing known biases and systematically uncover new biases by building a taxonomy that categorizes evaluation biases. As a result, we identify 17 investigated biases and uncover a new bias in the usage of patch validation strategies. To validate this new bias, we devise and implement an executable framework APRConfig , based on which we evaluate three typical patch validation strategies with four representative heuristic-based and constraint-based APR techniques on three bug datasets. Overall, this article distills 13 findings for bias understanding, discovery, and validation. The systematic exploration we performed and the open source executable framework we proposed in this article provide new insights as well as an infrastructure for future exploration and mitigation of biases in APR evaluation.
- 27 April 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 32 (3) , 1-37
Open AccessSeeing the Whole Elephant: Systematically Understanding and Uncovering Evaluation Biases in Automated Program Repair
Evaluation is the foundation of automated program repair (APR), as it provides empirical evidence on strengths and weaknesses of APR techniques. However, the reliability of such evaluation is often threatened by various introduced biases. Consequently, bias exploration, which uncovers biases in the APR evaluation, has become a pivotal activity and performed since the early years when pioneer APR techniques were proposed. Unfortunately, there is still no methodology to support a systematic comprehension and discovery of evaluation biases in APR, which impedes the mitigation of such biases and threatens the evaluation of APR techniques. In this work, we propose to systematically understand existing evaluation biases by rigorously conducting the first systematic literature review on existing known biases and systematically uncover new biases by building a taxonomy that categorizes evaluation biases. As a result, we identify 17 investigated biases and uncover a new bias in the usage of patch validation strategies. To validate this new bias, we devise and implement an executable framework APRConfig , based on which we evaluate three typical patch validation strategies with four representative heuristic-based and constraint-based APR techniques on three bug datasets. Overall, this article distills 13 findings for bias understanding, discovery, and validation. The systematic exploration we performed and the open source executable framework we proposed in this article provide new insights as well as an infrastructure for future exploration and mitigation of biases in APR evaluation.
- 29 September 2020
- Published by Springer Nature in Frontiers of Computer Science
- Vol. 15 (1) , 1-15
Evaluating the usage of fault localization in automated program repair: an empirical study
- 29 September 2020
- Published by Springer Nature in Frontiers of Computer Science
- Vol. 15 (1) , 1-15
Evaluating the usage of fault localization in automated program repair: an empirical study
- Open Access
- 20 November 2018
- Published by Springer Nature
- p. 33-48
Evaluating the Strategies of Statement Selection in Automated Program Repair
- 20 November 2018
- Published by Springer Nature
- p. 33-48
Open AccessEvaluating the Strategies of Statement Selection in Automated Program Repair
- Top Cited PapersOpen Access
- 31 May 2014
- Published by Association for Computing Machinery (ACM)
- p. 254-265
The strength of random search on automated program repair
- 31 May 2014
- Published by Association for Computing Machinery (ACM)
- p. 254-265
Top Cited PapersOpen AccessThe strength of random search on automated program repair
- 1 September 2013
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636773,p. 180-189
Efficient Automated Program Repair through Fault-Recorded Testing Prioritization
Most techniques for automated program repair use test cases to validate the effectiveness of the produced patches. The validation process can be time-consuming especially when the object programs ship with either lots of test cases or some long-running test cases. To alleviate the cost for testing, we first introduce regression test prioritization insight into the area of automated program repair, and present a novel prioritization technique called FRTP with the goal of reducing the number of test case executions in the repair process. Unlike most existing prioritization techniques frequently requiring additional cost for gathering previous test executions information, FRTP iteratively extracts that information just from the repair process, and thus incurs trivial performance lose. We also built a tool called TrpAutoRepair, which implements our FRTP technique and has the ability of automatically repairing C programs. To evaluate TrpAutoRepair, we compared it with GenProg, a state-of-the-art tool for automated C program repair. The experiment on the 5 subject programs with 16 real-life bugs provides evidence that TrpAutoRepair performs at least as good as GenProg in term of success rate, in most cases (15/16), TrpAutoRepair can significantly improve the repair efficiency by reducing efficiently the test case executions when searching a valid patch in the repair process.
- 1 September 2013
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636773,p. 180-189
Efficient Automated Program Repair through Fault-Recorded Testing Prioritization
Most techniques for automated program repair use test cases to validate the effectiveness of the produced patches. The validation process can be time-consuming especially when the object programs ship with either lots of test cases or some long-running test cases. To alleviate the cost for testing, we first introduce regression test prioritization insight into the area of automated program repair, and present a novel prioritization technique called FRTP with the goal of reducing the number of test case executions in the repair process. Unlike most existing prioritization techniques frequently requiring additional cost for gathering previous test executions information, FRTP iteratively extracts that information just from the repair process, and thus incurs trivial performance lose. We also built a tool called TrpAutoRepair, which implements our FRTP technique and has the ability of automatically repairing C programs. To evaluate TrpAutoRepair, we compared it with GenProg, a state-of-the-art tool for automated C program repair. The experiment on the 5 subject programs with 16 real-life bugs provides evidence that TrpAutoRepair performs at least as good as GenProg in term of success rate, in most cases (15/16), TrpAutoRepair can significantly improve the repair efficiency by reducing efficiently the test case executions when searching a valid patch in the repair process.
- Open Access
- 15 July 2013
- Published by Association for Computing Machinery (ACM)
- p. 191-201
Using automated program repair for evaluating the effectiveness of fault localization techniques
Many techniques on automated fault localization (AFL) have been introduced to assist developers in debugging. Prior studies evaluate the localization technique from the viewpoint of developers: measuring how many benefits that developers can obtain from the localization technique used when debugging. However, these evaluation approaches are not always suitable, because it is difficult to quantify precisely the benefits due to the complex debugging behaviors of developers. In addition, recent user studies have presented that developers working with AFL do not correct the defects more efficiently than ones working with only traditional debugging techniques such as breakpoints, even when the effectiveness of AFL is artificially improved. In this paper we attempt to propose a new research direction of developing AFL techniques from the viewpoint of fully automated debugging including the program repair of automation, for which the activity of AFL is necessary. We also introduce the NCP score as the evaluation measurement to assess and compare various techniques from this perspective. Our experiment on 15 popular AFL techniques with 11 subject programs shipping with real-life field failures presents the evidence that these AFL techniques performing well in prior studies do not have better localization effectiveness according to NCP score. We also observe that Jaccard has the better performance over other techniques in our experiment.
- 15 July 2013
- Published by Association for Computing Machinery (ACM)
- p. 191-201
Open AccessUsing automated program repair for evaluating the effectiveness of fault localization techniques
Many techniques on automated fault localization (AFL) have been introduced to assist developers in debugging. Prior studies evaluate the localization technique from the viewpoint of developers: measuring how many benefits that developers can obtain from the localization technique used when debugging. However, these evaluation approaches are not always suitable, because it is difficult to quantify precisely the benefits due to the complex debugging behaviors of developers. In addition, recent user studies have presented that developers working with AFL do not correct the defects more efficiently than ones working with only traditional debugging techniques such as breakpoints, even when the effectiveness of AFL is artificially improved. In this paper we attempt to propose a new research direction of developing AFL techniques from the viewpoint of fully automated debugging including the program repair of automation, for which the activity of AFL is necessary. We also introduce the NCP score as the evaluation measurement to assess and compare various techniques from this perspective. Our experiment on 15 popular AFL techniques with 11 subject programs shipping with real-life field failures presents the evidence that these AFL techniques performing well in prior studies do not have better localization effectiveness according to NCP score. We also observe that Jaccard has the better performance over other techniques in our experiment.
- 1 September 2012
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636773,p. 254-263
Making automatic repair for large-scale programs more efficient using weak recompilation
For large-scale programs, automatically repairing a bug by modifying source code is often a time-consuming process due to plenty of time spent on recompiling and reinstalling the patched program. To suppress the above time cost and make the repair process more efficient, a recompilation technique called weak recompilation is described in this paper. In weak recompilation, a program is assumed to be constructed from a set of components, and for each candidate patch only the changed code fragment in term of one component is recompiled to a shared library; the behaviors of patched program are observed by executing the original program with an instrumentation tool which can wrap specified function. The advantage of weak recompilation is that redundant recompilation cost can be suppressed, and reinstallation cost will be cut down completely. We also built WAutoRepair, a system which enables scalability to fix bugs in large-scale C programs with high efficiency. The experiments confirm that our repair system significantly outperforms Genprog, a famous approach for automatic program repair. For the wireshark program containing over 2 millions lines of code, WAutoRepair spent only 0.222 seconds in recompiling one candidate patch and 8.035 seconds in totally repairing the bug, compared to Genprog separately taking about 20.484 and 75.493 seconds, on average.
- 1 September 2012
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636773,p. 254-263
Making automatic repair for large-scale programs more efficient using weak recompilation
For large-scale programs, automatically repairing a bug by modifying source code is often a time-consuming process due to plenty of time spent on recompiling and reinstalling the patched program. To suppress the above time cost and make the repair process more efficient, a recompilation technique called weak recompilation is described in this paper. In weak recompilation, a program is assumed to be constructed from a set of components, and for each candidate patch only the changed code fragment in term of one component is recompiled to a shared library; the behaviors of patched program are observed by executing the original program with an instrumentation tool which can wrap specified function. The advantage of weak recompilation is that redundant recompilation cost can be suppressed, and reinstallation cost will be cut down completely. We also built WAutoRepair, a system which enables scalability to fix bugs in large-scale C programs with high efficiency. The experiments confirm that our repair system significantly outperforms Genprog, a famous approach for automatic program repair. For the wireshark program containing over 2 millions lines of code, WAutoRepair spent only 0.222 seconds in recompiling one candidate patch and 8.035 seconds in totally repairing the bug, compared to Genprog separately taking about 20.484 and 75.493 seconds, on average.