Dong Huang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Dong Huang has a total of 35 co-authors. Here are the top co-authors by number of shared publications:
- Heming Cui
The University of Hong Kong, Hong Kong, Hong Kong
4 shared publications - Jie M. Zhang
King’s College London, London, United Kingdom of Great Britain and Northern Ireland
3 shared publications - Mingzhe Du
National University of Singapore, Singapore, Singapore
3 shared publications - Junjie Chen
Tianjin University, Tianjin, China
2 shared publications - Mark Harman
University College London, London, United Kingdom
2 shared publications - Qianru Zhang
University of Hong Kong
2 shared publications
Latest Publications
- Top Cited PapersOpen Access
- 18 November 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35 (12) , 1-33
Benchmarking LLMs for Unit Test Generation from Real-World Functions
Recently, large language models (LLMs) have shown great promise in automating unit test generation, significantly reducing the manual effort required by developers. To effectively evaluate the capabilities of LLMs in this domain, it is crucial to have a well-designed benchmark that accurately reflects real-world scenarios and mitigates common pitfalls. Existing LLM test generation benchmarks are limited by two critical drawbacks: data contamination and structurally simple function code. As a result, scientific conclusions drawn from empirical studies using these benchmarks may be compromised. The evidence presented may be biased due to contamination and may fail to generalize beyond toy programs due to structural simplicity. To address these problems, we introduce ULT (UnLeakedTestbench), a new benchmark specifically designed for function-level unit test generation from real-world Python functions. ULT is constructed through a multi-stage curation process that ensures high cyclomatic complexity and mitigates test case contamination. With 3,909 carefully selected function-level tasks, ULT provides a more realistic and challenging evaluation of LLMs’ test generation capabilities. We also provide PLT (PreLeakedTestbench), a pair of ULT benchmarks with leaked tests designed to enable a controlled analysis of memorization versus reasoning in test generation. Based on the two datasets, we conduct a large-scale empirical study involving 12 state-of-the-art LLMs, comparing their performance against established benchmarks. Our evaluation results demonstrate that ULT is significantly more challenging. For example, test cases generated by LLMs only achieve 41.32%, 45.10%, 30.22%, and 40.21% for accuracy, statement coverage, branch coverage, and mutation score on average for all LLMs, respectively. These results are substantially lower than the corresponding metrics on TestEval (91.79%, 92.18%, 82.04%, and 49.69%) and PLT (47.07%, 55.13%, 40.07%, and 50.80%). In addition, different from existing benchmarks, ULT shows a strong correlation between test generation performance and code generation performance. For example, the correlation coefficient between the coding ability and test generation performance ( \(Pass@1\) ) on ULT is 0.79 (p = 0.002), while it is only 0.56 (p = 0.059) and 0.52 (p = 0.080) on TestEval and PLT, respectively. This indicates that ULT more effectively measures the generalization ability of LLMs. We also make ULT and evaluation results publicly available to foster further research (To preserve the integrity of the benchmark and prevent test case leakage into future LLM training sets, we do not release the ground-truth tests. Instead, we provide the complete evaluation results for all models benchmarked in this article. This allows researchers to compare new models against our findings without compromising the benchmark). ULT is available at https://github.com/huangd1999/UnLeakedTestBench .- 18 November 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35 (12) , 1-33
Top Cited PapersOpen AccessBenchmarking LLMs for Unit Test Generation from Real-World Functions
Recently, large language models (LLMs) have shown great promise in automating unit test generation, significantly reducing the manual effort required by developers. To effectively evaluate the capabilities of LLMs in this domain, it is crucial to have a well-designed benchmark that accurately reflects real-world scenarios and mitigates common pitfalls. Existing LLM test generation benchmarks are limited by two critical drawbacks: data contamination and structurally simple function code. As a result, scientific conclusions drawn from empirical studies using these benchmarks may be compromised. The evidence presented may be biased due to contamination and may fail to generalize beyond toy programs due to structural simplicity. To address these problems, we introduce ULT (UnLeakedTestbench), a new benchmark specifically designed for function-level unit test generation from real-world Python functions. ULT is constructed through a multi-stage curation process that ensures high cyclomatic complexity and mitigates test case contamination. With 3,909 carefully selected function-level tasks, ULT provides a more realistic and challenging evaluation of LLMs’ test generation capabilities. We also provide PLT (PreLeakedTestbench), a pair of ULT benchmarks with leaked tests designed to enable a controlled analysis of memorization versus reasoning in test generation. Based on the two datasets, we conduct a large-scale empirical study involving 12 state-of-the-art LLMs, comparing their performance against established benchmarks. Our evaluation results demonstrate that ULT is significantly more challenging. For example, test cases generated by LLMs only achieve 41.32%, 45.10%, 30.22%, and 40.21% for accuracy, statement coverage, branch coverage, and mutation score on average for all LLMs, respectively. These results are substantially lower than the corresponding metrics on TestEval (91.79%, 92.18%, 82.04%, and 49.69%) and PLT (47.07%, 55.13%, 40.07%, and 50.80%). In addition, different from existing benchmarks, ULT shows a strong correlation between test generation performance and code generation performance. For example, the correlation coefficient between the coding ability and test generation performance ( \(Pass@1\) ) on ULT is 0.79 (p = 0.002), while it is only 0.56 (p = 0.059) and 0.52 (p = 0.080) on TestEval and PLT, respectively. This indicates that ULT more effectively measures the generalization ability of LLMs. We also make ULT and evaluation results publicly available to foster further research (To preserve the integrity of the benchmark and prevent test case leakage into future LLM training sets, we do not release the ground-truth tests. Instead, we provide the complete evaluation results for all models benchmarked in this article. This allows researchers to compare new models against our findings without compromising the benchmark). ULT is available at https://github.com/huangd1999/UnLeakedTestBench . - Open Access
- 6 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 8260-8267
Paper Espresso: From Paper Overload to Research Insight
The accelerating pace of scientific publishing makes it increasingly difficult for researchers to stay current. We present Paper Espresso , an open-source platform that automatically discovers, summarizes, and analyzes trending arXiv papers. The system uses large language models (LLMs) to generate structured summaries with topical labels and keywords, and provides multi-granularity trend analysis at daily, monthly, and lifecycle scales through LLM-driven topic consolidation. Over 36~months of continuous deployment, Paper Espresso has processed 13,388 papers and publicly released all structured metadata, revealing rich dynamics in the AI research landscape: a mid-2025 surge in reinforcement learning for LLM reasoning, non-saturating topic emergence (6,673 unique topics), and a positive correlation between topic novelty and community engagement (2.0× median upvotes for the most novel papers). The live demo and code are available at https://huggingface.co/spaces/Elfsong/Paper_Espresso.
- 6 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 8260-8267
Open AccessPaper Espresso: From Paper Overload to Research Insight
The accelerating pace of scientific publishing makes it increasingly difficult for researchers to stay current. We present Paper Espresso , an open-source platform that automatically discovers, summarizes, and analyzes trending arXiv papers. The system uses large language models (LLMs) to generate structured summaries with topical labels and keywords, and provides multi-granularity trend analysis at daily, monthly, and lifecycle scales through LLM-driven topic consolidation. Over 36~months of continuous deployment, Paper Espresso has processed 13,388 papers and publicly released all structured metadata, revealing rich dynamics in the AI research landscape: a mid-2025 surge in reinforcement learning for LLM reasoning, non-saturating topic emergence (6,673 unique topics), and a positive correlation between topic novelty and community engagement (2.0× median upvotes for the most novel papers). The live demo and code are available at https://huggingface.co/spaces/Elfsong/Paper_Espresso.
- Open Access
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 156-168
Measuring the Influence of Incorrect Code on Test Generation
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 156-168
Open AccessMeasuring the Influence of Incorrect Code on Test Generation
- Top Cited PapersOpen Access
- 11 December 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35 (1) , 1-31
Bias Testing and Mitigation in LLM-based Code Generation
As the adoption of LLMs becomes more widespread in software coding ecosystems, a pressing issue has emerged: does the generated code contain social bias and unfairness, such as those related to age, gender, and race? This issue concerns the integrity, fairness, and ethical foundation of software applications that depend on the code generated by these models but are underexplored in the literature. This article presents a novel bias testing framework that is specifically designed for code generation tasks. Based on this framework, we conduct an extensive empirical study on the biases in code generated by five widely studied LLMs (i.e., PALM-2-CodeChat-bison, Claude-instant-1, GPT-3.5-turbo, GPT-4-turbo, and GPT-4). Our findings reveal that biases are prevalent. For example, 13.47% to 49.10% of the codes generated by these LLMs have biased behaviors towards gender. Moreover, we study five bias mitigation prompt strategies that are commonly used in current code generation scenarios, i.e., zero-shot, one-shot, few-shot, and two Chain-of-Thought (CoT) prompts, with and without provided feedback-driven refinement. Our evaluation results illustrate that using direct prompt engineering strategies has limited effectiveness in mitigating bias, but our test execution feedback can help to reduce the ratio of code biases to a large extent (e.g., from 59.88% to 4.79% for GPT-4).
- 11 December 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35 (1) , 1-31
Top Cited PapersOpen AccessBias Testing and Mitigation in LLM-based Code Generation
As the adoption of LLMs becomes more widespread in software coding ecosystems, a pressing issue has emerged: does the generated code contain social bias and unfairness, such as those related to age, gender, and race? This issue concerns the integrity, fairness, and ethical foundation of software applications that depend on the code generated by these models but are underexplored in the literature. This article presents a novel bias testing framework that is specifically designed for code generation tasks. Based on this framework, we conduct an extensive empirical study on the biases in code generated by five widely studied LLMs (i.e., PALM-2-CodeChat-bison, Claude-instant-1, GPT-3.5-turbo, GPT-4-turbo, and GPT-4). Our findings reveal that biases are prevalent. For example, 13.47% to 49.10% of the codes generated by these LLMs have biased behaviors towards gender. Moreover, we study five bias mitigation prompt strategies that are commonly used in current code generation scenarios, i.e., zero-shot, one-shot, few-shot, and two Chain-of-Thought (CoT) prompts, with and without provided feedback-driven refinement. Our evaluation results illustrate that using direct prompt engineering strategies has limited effectiveness in mitigating bias, but our test execution feedback can help to reduce the ratio of code biases to a large extent (e.g., from 59.88% to 4.79% for GPT-4).
- Open Access
- 10 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 4139-4148
HGAurban: Heterogeneous Graph Autoencoding for Urban Spatial-Temporal Learning
- 10 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 4139-4148
Open AccessHGAurban: Heterogeneous Graph Autoencoding for Urban Spatial-Temporal Learning
- 21 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 25 (1) , 992-1008
State-Aware Perturbation Optimization for Robust Deep Reinforcement Learning
Recently, deep reinforcement learning (DRL) has emerged as a promising approach for robotic control. However, the deployment of DRL in real-world robots is hindered by its sensitivity to environmental perturbations. While existing white-box adversarial attacks rely on local gradient information and apply uniform perturbations across all states to evaluate DRL robustness, they fail to account for temporal dynamics and state-specific vulnerabilities. To combat the above challenge, we first conduct a theoretical analysis of white-box attacks in DRL by establishing the Adversarial Victim Dynamics Markov Decision Process (AVD-MDP), to derive the necessary and sufficient conditions for a successful attack. Based on this, we propose the Selective State-Aware Reinforcement adversarial attack (STAR), to optimize perturbation stealthiness and state visitation dispersion. STAR first employs a soft mask-based state-targeting mechanism to minimize redundant perturbations, enhancing stealthiness and attack effectiveness. Then, it incorporates an information-theoretic optimization objective to maximize mutual information between perturbations, environmental states, and victim actions, ensuring a dispersed state-visitation distribution that steers the victim agent into vulnerable states for maximum return reduction. Extensive experiments demonstrate that STAR outperforms state-of-the-art benchmarks.
- 21 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 25 (1) , 992-1008
State-Aware Perturbation Optimization for Robust Deep Reinforcement Learning
Recently, deep reinforcement learning (DRL) has emerged as a promising approach for robotic control. However, the deployment of DRL in real-world robots is hindered by its sensitivity to environmental perturbations. While existing white-box adversarial attacks rely on local gradient information and apply uniform perturbations across all states to evaluate DRL robustness, they fail to account for temporal dynamics and state-specific vulnerabilities. To combat the above challenge, we first conduct a theoretical analysis of white-box attacks in DRL by establishing the Adversarial Victim Dynamics Markov Decision Process (AVD-MDP), to derive the necessary and sufficient conditions for a successful attack. Based on this, we propose the Selective State-Aware Reinforcement adversarial attack (STAR), to optimize perturbation stealthiness and state visitation dispersion. STAR first employs a soft mask-based state-targeting mechanism to minimize redundant perturbations, enhancing stealthiness and attack effectiveness. Then, it incorporates an information-theoretic optimization objective to maximize mutual information between perturbations, environmental states, and victim actions, ensuring a dispersed state-visitation distribution that steers the victim agent into vulnerable states for maximum return reduction. Extensive experiments demonstrate that STAR outperforms state-of-the-art benchmarks.
- Open Access
- 27 September 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 33 (7) , 1-32
Neuron Sensitivity-Guided Test Case Selection
Deep neural networks (DNNs) have been widely deployed in software to address various tasks (e.g., autonomous driving, medical diagnosis). However, they can also produce incorrect behaviors that result in financial losses and even threaten human safety. To reveal and repair incorrect behaviors in DNNs, developers often collect rich, unlabeled datasets from the natural world and label them to test DNN models. However, properly labeling a large number of datasets is a highly expensive and time-consuming task. To address the above-mentioned problem, we propose neuron sensitivity-guided test case selection (NSS), which can reduce the labeling time by selecting valuable test cases from unlabeled datasets. NSS leverages the information of the internal neuron induced by the test cases to select valuable test cases, which have high confidence in causing the model to behave incorrectly. We evaluated NSS with four widely used datasets and four well-designed DNN models compared to the state-of-the-art (SOTA) baseline methods. The results show that NSS performs well in assessing the probability of failure triggering in test cases and in the improvement capabilities of the model. Specifically, compared to the baseline approaches, NSS achieves a higher fault detection rate (e.g., when selecting 5% of the test cases from the unlabeled dataset in the MNIST and LeNet1 experiment, NSS can obtain an 81.8% fault detection rate, which is a 20% increase compared with SOTA baseline strategies).
- 27 September 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 33 (7) , 1-32
Open AccessNeuron Sensitivity-Guided Test Case Selection
Deep neural networks (DNNs) have been widely deployed in software to address various tasks (e.g., autonomous driving, medical diagnosis). However, they can also produce incorrect behaviors that result in financial losses and even threaten human safety. To reveal and repair incorrect behaviors in DNNs, developers often collect rich, unlabeled datasets from the natural world and label them to test DNN models. However, properly labeling a large number of datasets is a highly expensive and time-consuming task. To address the above-mentioned problem, we propose neuron sensitivity-guided test case selection (NSS), which can reduce the labeling time by selecting valuable test cases from unlabeled datasets. NSS leverages the information of the internal neuron induced by the test cases to select valuable test cases, which have high confidence in causing the model to behave incorrectly. We evaluated NSS with four widely used datasets and four well-designed DNN models compared to the state-of-the-art (SOTA) baseline methods. The results show that NSS performs well in assessing the probability of failure triggering in test cases and in the improvement capabilities of the model. Specifically, compared to the baseline approaches, NSS achieves a higher fault detection rate (e.g., when selecting 5% of the test cases from the unlabeled dataset in the MNIST and LeNet1 experiment, NSS can obtain an 81.8% fault detection rate, which is a 20% increase compared with SOTA baseline strategies).
- 4 August 2019
- Published by Taylor & Francis in Applied Economics
- Vol. 52 (6) , 537-551
Is there evidence of “fear of appreciation” in China’s foreign exchange market intervention?
Using the data after the exchange rate regime reform in July 2005, this paper systematically studied the issue whether China has conducted ‘fear of appreciation’ asymmetric intervention. The results show that China’s intervention is discontinuous and asymmetric, rather than oversimplified ‘fear of appreciation’. China has a target range with respect to exchange rate deviation, and most observations are concentrated in this range, in which the exchange rate deviation can be tolerated. Outside the target range, the width of the range suggests that the tolerance level for appreciation is much higher than that for depreciation, whereas the estimated coefficients show that the monetary authority responds more vigorously to substantial appreciation than to depreciation. In addition, it can be expected that with the marketization of RMB exchange rate, China’s intervention degree will further decline.
- 4 August 2019
- Published by Taylor & Francis in Applied Economics
- Vol. 52 (6) , 537-551
Is there evidence of “fear of appreciation” in China’s foreign exchange market intervention?
Using the data after the exchange rate regime reform in July 2005, this paper systematically studied the issue whether China has conducted ‘fear of appreciation’ asymmetric intervention. The results show that China’s intervention is discontinuous and asymmetric, rather than oversimplified ‘fear of appreciation’. China has a target range with respect to exchange rate deviation, and most observations are concentrated in this range, in which the exchange rate deviation can be tolerated. Outside the target range, the width of the range suggests that the tolerance level for appreciation is much higher than that for depreciation, whereas the estimated coefficients show that the monetary authority responds more vigorously to substantial appreciation than to depreciation. In addition, it can be expected that with the marketization of RMB exchange rate, China’s intervention degree will further decline.