Qianru Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Qianru Zhang has a total of 32 co-authors. Here are the top co-authors by number of shared publications:
- Siu-Ming Yiu
The University of Hong Kong
6 shared publications - Dong Huang
National University of Singapore, Singapore, Singapore
2 shared publications - Hongzhi Yin
The University of Queensland
2 shared publications - Zheng Wang
Nanyang technological university, Singapore, Singapore
2 shared publications - A-Long Jin
Department of Communications and Networking, Xian Jiaotong-Liverpool University, Suzhou, China
1 shared publication - Chao Huang
The University of Hong Kong, Hong Kong, China
1 shared publication
Latest Publications
- Top Cited PapersOpen Access
- 18 November 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35 (12) , 1-33
Benchmarking LLMs for Unit Test Generation from Real-World Functions
Recently, large language models (LLMs) have shown great promise in automating unit test generation, significantly reducing the manual effort required by developers. To effectively evaluate the capabilities of LLMs in this domain, it is crucial to have a well-designed benchmark that accurately reflects real-world scenarios and mitigates common pitfalls. Existing LLM test generation benchmarks are limited by two critical drawbacks: data contamination and structurally simple function code. As a result, scientific conclusions drawn from empirical studies using these benchmarks may be compromised. The evidence presented may be biased due to contamination and may fail to generalize beyond toy programs due to structural simplicity. To address these problems, we introduce ULT (UnLeakedTestbench), a new benchmark specifically designed for function-level unit test generation from real-world Python functions. ULT is constructed through a multi-stage curation process that ensures high cyclomatic complexity and mitigates test case contamination. With 3,909 carefully selected function-level tasks, ULT provides a more realistic and challenging evaluation of LLMs’ test generation capabilities. We also provide PLT (PreLeakedTestbench), a pair of ULT benchmarks with leaked tests designed to enable a controlled analysis of memorization versus reasoning in test generation. Based on the two datasets, we conduct a large-scale empirical study involving 12 state-of-the-art LLMs, comparing their performance against established benchmarks. Our evaluation results demonstrate that ULT is significantly more challenging. For example, test cases generated by LLMs only achieve 41.32%, 45.10%, 30.22%, and 40.21% for accuracy, statement coverage, branch coverage, and mutation score on average for all LLMs, respectively. These results are substantially lower than the corresponding metrics on TestEval (91.79%, 92.18%, 82.04%, and 49.69%) and PLT (47.07%, 55.13%, 40.07%, and 50.80%). In addition, different from existing benchmarks, ULT shows a strong correlation between test generation performance and code generation performance. For example, the correlation coefficient between the coding ability and test generation performance ( \(Pass@1\) ) on ULT is 0.79 (p = 0.002), while it is only 0.56 (p = 0.059) and 0.52 (p = 0.080) on TestEval and PLT, respectively. This indicates that ULT more effectively measures the generalization ability of LLMs. We also make ULT and evaluation results publicly available to foster further research (To preserve the integrity of the benchmark and prevent test case leakage into future LLM training sets, we do not release the ground-truth tests. Instead, we provide the complete evaluation results for all models benchmarked in this article. This allows researchers to compare new models against our findings without compromising the benchmark). ULT is available at https://github.com/huangd1999/UnLeakedTestBench .- 18 November 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35 (12) , 1-33
Top Cited PapersOpen AccessBenchmarking LLMs for Unit Test Generation from Real-World Functions
Recently, large language models (LLMs) have shown great promise in automating unit test generation, significantly reducing the manual effort required by developers. To effectively evaluate the capabilities of LLMs in this domain, it is crucial to have a well-designed benchmark that accurately reflects real-world scenarios and mitigates common pitfalls. Existing LLM test generation benchmarks are limited by two critical drawbacks: data contamination and structurally simple function code. As a result, scientific conclusions drawn from empirical studies using these benchmarks may be compromised. The evidence presented may be biased due to contamination and may fail to generalize beyond toy programs due to structural simplicity. To address these problems, we introduce ULT (UnLeakedTestbench), a new benchmark specifically designed for function-level unit test generation from real-world Python functions. ULT is constructed through a multi-stage curation process that ensures high cyclomatic complexity and mitigates test case contamination. With 3,909 carefully selected function-level tasks, ULT provides a more realistic and challenging evaluation of LLMs’ test generation capabilities. We also provide PLT (PreLeakedTestbench), a pair of ULT benchmarks with leaked tests designed to enable a controlled analysis of memorization versus reasoning in test generation. Based on the two datasets, we conduct a large-scale empirical study involving 12 state-of-the-art LLMs, comparing their performance against established benchmarks. Our evaluation results demonstrate that ULT is significantly more challenging. For example, test cases generated by LLMs only achieve 41.32%, 45.10%, 30.22%, and 40.21% for accuracy, statement coverage, branch coverage, and mutation score on average for all LLMs, respectively. These results are substantially lower than the corresponding metrics on TestEval (91.79%, 92.18%, 82.04%, and 49.69%) and PLT (47.07%, 55.13%, 40.07%, and 50.80%). In addition, different from existing benchmarks, ULT shows a strong correlation between test generation performance and code generation performance. For example, the correlation coefficient between the coding ability and test generation performance ( \(Pass@1\) ) on ULT is 0.79 (p = 0.002), while it is only 0.56 (p = 0.059) and 0.52 (p = 0.080) on TestEval and PLT, respectively. This indicates that ULT more effectively measures the generalization ability of LLMs. We also make ULT and evaluation results publicly available to foster further research (To preserve the integrity of the benchmark and prevent test case leakage into future LLM training sets, we do not release the ground-truth tests. Instead, we provide the complete evaluation results for all models benchmarked in this article. This allows researchers to compare new models against our findings without compromising the benchmark). ULT is available at https://github.com/huangd1999/UnLeakedTestBench . - 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Dependable and Secure Computing
- Vol. 23 (6) , 12673-12686
TUBench: Benchmark Large Vision-Language Models on Trustworthiness With Unanswerable Questions
- 1 November 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Dependable and Secure Computing
- Vol. 23 (6) , 12673-12686
TUBench: Benchmark Large Vision-Language Models on Trustworthiness With Unanswerable Questions
- Open Access
- 23 May 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
- Vol. 44 (5) , 1-27
HMamba : Hyperbolic Mamba for Sequential Recommendation
Sequential recommendation systems require both temporal efficiency to handle long interaction histories and hierarchical representation to model complex user–item relationships. Existing approaches face a fundamental tension: Mamba-based methods offer linear-time efficiency ( \(\mathcal{O}(L)\) ) but operate in Euclidean space, which distorts hierarchical patterns; hyperbolic models capture taxonomies well but suffer quadratic complexity ( \(\mathcal{O}(L^{2})\) ). To solve this dual challenge , we propose Hyperbolic Mamba (HMamba), the first architecture that unifies curvature-aware state spaces with hyperbolic geometry. Our key insight is that hyperbolic curvature \(\kappa\) simultaneously governs: (1) state transition granularity through \(\mathbf{\bar{A}}=\exp(\Delta\mathbf{A}\odot\mathbf{K}(\kappa))\) and (2) hierarchical distance preservation via \(d_{\mathcal{L}}\propto\sqrt{\kappa}\log(\cdot)\) . This enables joint optimization of efficiency and hierarchy—addressing the previously unsolved problem of deep-long modeling . Experiments show HMamba achieves 3–11% accuracy gains while maintaining 3.2 \(\times\) faster training than attention-based models, establishing a new paradigm for hierarchy-aware sequential recommendation. The code and datasets accompanying our paper are publicly available at https://github.com/CoderPowerBeyond/HMamba .- 23 May 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
- Vol. 44 (5) , 1-27
Open AccessHMamba : Hyperbolic Mamba for Sequential Recommendation
Sequential recommendation systems require both temporal efficiency to handle long interaction histories and hierarchical representation to model complex user–item relationships. Existing approaches face a fundamental tension: Mamba-based methods offer linear-time efficiency ( \(\mathcal{O}(L)\) ) but operate in Euclidean space, which distorts hierarchical patterns; hyperbolic models capture taxonomies well but suffer quadratic complexity ( \(\mathcal{O}(L^{2})\) ). To solve this dual challenge , we propose Hyperbolic Mamba (HMamba), the first architecture that unifies curvature-aware state spaces with hyperbolic geometry. Our key insight is that hyperbolic curvature \(\kappa\) simultaneously governs: (1) state transition granularity through \(\mathbf{\bar{A}}=\exp(\Delta\mathbf{A}\odot\mathbf{K}(\kappa))\) and (2) hierarchical distance preservation via \(d_{\mathcal{L}}\propto\sqrt{\kappa}\log(\cdot)\) . This enables joint optimization of efficiency and hierarchy—addressing the previously unsolved problem of deep-long modeling . Experiments show HMamba achieves 3–11% accuracy gains while maintaining 3.2 \(\times\) faster training than attention-based models, establishing a new paradigm for hierarchy-aware sequential recommendation. The code and datasets accompanying our paper are publicly available at https://github.com/CoderPowerBeyond/HMamba . - Top Cited Papers
- 7 May 2026
- Published by American Chemical Society (ACS) in Journal of Chemical Theory and Computation
- Vol. 22 (10) , 4866-4887
A Systematic Survey and Benchmark of Deep Learning for Molecular Property Prediction in the Foundation Model Era
Molecular property prediction integrates quantum chemistry, cheminformatics, and deep learning to connect molecular structure with physicochemical and biological behavior. This survey traces four complementary paradigms, including Quantum, Descriptor Machine Learning, Geometric Deep Learning, and Foundation Models, and outlines a unified taxonomy linking molecular representations, model architectures, and interdisciplinary applications. Benchmark analyses integrate evidence from both widely used data sets and data sets reflecting industry perspectives, encompassing quantum, physicochemical, physiological, and biophysical domains. The survey examines current standards in data curation, splitting strategies, and evaluation protocols, highlighting challenges including inconsistent stereochemistry, heterogeneous assay sources, and reproducibility limitations under random or poorly defined splits. These observations motivate the modernization of benchmark design toward more transparent, time- and scaffold-aware methodologies. We further propose three forward-looking directions: (i) physics-aware learning embedding quantum consistency, (ii) uncertainty-calibrated foundation models for trustworthy inference, and (iii) realistic multimodal benchmark ecosystems integrating computational and experimental data. Repository: https://github.com/Zongru-Li/Survey-and-Benchmarks-of-DL-for-Molecular-Property-Prediction-in-the-Foundation-Model-Era.
- 7 May 2026
- Published by American Chemical Society (ACS) in Journal of Chemical Theory and Computation
- Vol. 22 (10) , 4866-4887
Top Cited PapersA Systematic Survey and Benchmark of Deep Learning for Molecular Property Prediction in the Foundation Model Era
Molecular property prediction integrates quantum chemistry, cheminformatics, and deep learning to connect molecular structure with physicochemical and biological behavior. This survey traces four complementary paradigms, including Quantum, Descriptor Machine Learning, Geometric Deep Learning, and Foundation Models, and outlines a unified taxonomy linking molecular representations, model architectures, and interdisciplinary applications. Benchmark analyses integrate evidence from both widely used data sets and data sets reflecting industry perspectives, encompassing quantum, physicochemical, physiological, and biophysical domains. The survey examines current standards in data curation, splitting strategies, and evaluation protocols, highlighting challenges including inconsistent stereochemistry, heterogeneous assay sources, and reproducibility limitations under random or poorly defined splits. These observations motivate the modernization of benchmark design toward more transparent, time- and scaffold-aware methodologies. We further propose three forward-looking directions: (i) physics-aware learning embedding quantum consistency, (ii) uncertainty-calibrated foundation models for trustworthy inference, and (iii) realistic multimodal benchmark ecosystems integrating computational and experimental data. Repository: https://github.com/Zongru-Li/Survey-and-Benchmarks-of-DL-for-Molecular-Property-Prediction-in-the-Foundation-Model-Era.
- Open Access
- 10 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 4139-4148
HGAurban: Heterogeneous Graph Autoencoding for Urban Spatial-Temporal Learning
- 10 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 4139-4148
Open AccessHGAurban: Heterogeneous Graph Autoencoding for Urban Spatial-Temporal Learning
- Open Access
- 14 October 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Knowledge Discovery From Data
- Vol. 18 (9) , 1-27
Billiards Sports Analytics: Datasets and Tasks
Nowadays, it becomes a common practice to capture some data of sports games with devices such as GPS sensors and cameras and then use the data to perform various analyses on sports games, including tactics discovery, similar game retrieval, performance study, and so forth. While this practice has been conducted to many sports such as basketball and soccer, it remains largely unexplored on the billiards sports, which is mainly due to the lack of publicly available datasets. Motivated by this, we collect a dataset of billiards sports, which includes the layouts (i.e., locations) of billiards balls after performing break shots, called break shot layouts, the traces of the balls as a result of strikes (in the form of trajectories), and detailed statistics and performance indicators. We then study and develop techniques for three tasks on the collected dataset, including (1) prediction and (2) generation on the layouts data, and (3) similar billiards layout retrieval on the layouts data, which can serve different users such as coaches, players and fans. We conduct extensive experiments on the collected dataset and the results show that our methods perform effectively and efficiently.
- 14 October 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Knowledge Discovery From Data
- Vol. 18 (9) , 1-27
Open AccessBilliards Sports Analytics: Datasets and Tasks
Nowadays, it becomes a common practice to capture some data of sports games with devices such as GPS sensors and cameras and then use the data to perform various analyses on sports games, including tactics discovery, similar game retrieval, performance study, and so forth. While this practice has been conducted to many sports such as basketball and soccer, it remains largely unexplored on the billiards sports, which is mainly due to the lack of publicly available datasets. Motivated by this, we collect a dataset of billiards sports, which includes the layouts (i.e., locations) of billiards balls after performing break shots, called break shot layouts, the traces of the balls as a result of strikes (in the form of trajectories), and detailed statistics and performance indicators. We then study and develop techniques for three tasks on the collected dataset, including (1) prediction and (2) generation on the layouts data, and (3) similar billiards layout retrieval on the layouts data, which can serve different users such as coaches, players and fans. We conduct extensive experiments on the collected dataset and the results show that our methods perform effectively and efficiently.
- Open Access
- 30 April 2023
- Published by Association for Computing Machinery (ACM)
- p. 295-305
Automated Spatio-Temporal Graph Contrastive Learning
Among various region embedding methods, graph-based region relation learning models stand out, owing to their strong structure representation ability for encoding spatial correlations with graph neural networks. Despite their effectiveness, several key challenges have not been well addressed in existing methods: i) Data noise and missing are ubiquitous in many spatio-temporal scenarios due to a variety of factors. ii) Input spatio-temporal data (e.g., mobility traces) usually exhibits distribution heterogeneity across space and time. In such cases, current methods are vulnerable to the quality of the generated region graphs, which may lead to suboptimal performance. In this paper, we tackle the above challenges by exploring the Automated Spatio-Temporal graph contrastive learning paradigm (AutoST) over the heterogeneous region graph generated from multi-view data sources. Our AutoST framework is built upon a heterogeneous graph neural architecture to capture the multi-view region dependencies with respect to POI semantics, mobility flow patterns and geographical positions. To improve the robustness of our GNN encoder against data noise and distribution issues, we design an automated spatio-temporal augmentation scheme with a parameterized contrastive view generator. AutoST can adapt to the spatio-temporal heterogeneous graph with multi-view semantics well preserved. Extensive experiments for three downstream spatio-temporal mining tasks on several real-world datasets demonstrate the significant performance gain achieved by our AutoST over a variety of baselines. The code is publicly available at https://github.com/HKUDS/AutoST.
- 30 April 2023
- Published by Association for Computing Machinery (ACM)
- p. 295-305
Open AccessAutomated Spatio-Temporal Graph Contrastive Learning
Among various region embedding methods, graph-based region relation learning models stand out, owing to their strong structure representation ability for encoding spatial correlations with graph neural networks. Despite their effectiveness, several key challenges have not been well addressed in existing methods: i) Data noise and missing are ubiquitous in many spatio-temporal scenarios due to a variety of factors. ii) Input spatio-temporal data (e.g., mobility traces) usually exhibits distribution heterogeneity across space and time. In such cases, current methods are vulnerable to the quality of the generated region graphs, which may lead to suboptimal performance. In this paper, we tackle the above challenges by exploring the Automated Spatio-Temporal graph contrastive learning paradigm (AutoST) over the heterogeneous region graph generated from multi-view data sources. Our AutoST framework is built upon a heterogeneous graph neural architecture to capture the multi-view region dependencies with respect to POI semantics, mobility flow patterns and geographical positions. To improve the robustness of our GNN encoder against data noise and distribution issues, we design an automated spatio-temporal augmentation scheme with a parameterized contrastive view generator. AutoST can adapt to the spatio-temporal heterogeneous graph with multi-view semantics well preserved. Extensive experiments for three downstream spatio-temporal mining tasks on several real-world datasets demonstrate the significant performance gain achieved by our AutoST over a variety of baselines. The code is publicly available at https://github.com/HKUDS/AutoST.