Junxiao Han
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Junxiao Han has a total of 82 co-authors. Here are the top co-authors by number of shared publications:
- Shuiguang Deng
Zhejiang University, Hangzhou, China
11 shared publications - Chen Zhi
Zhejiang University, Ningbo, China
6 shared publications - Yishan Chen
Jiangxi University of Science and Technology
6 shared publications - Jianwei Yin
School of Software Technology, Zhejiang University, Ningbo, China
4 shared publications David Lo
School of Computing and Information Systems Singapore Management University Singapore Singapore
3 shared publications- Jiakun Liu
Harbin Institute of Technology, Harbin, China
3 shared publications
Latest Publications
- 7 October 2026
- Published by Wiley in Journal of Software: Evolution and Process
- Vol. 38 (10) , e70170
What Makes a Good Model Card?
Hugging Face (HF) is an increasingly popular platform for sharing and discovering machine learning (ML) and artificial intelligence (AI) models. Model cards document these models, and good model cards serve as a vital communication bridge between model developers and users. Without good model cards, it is challenging for users to understand the models and for developers to maintain them. Prior research focused on the documentation process and automatic generation of model cards, and identified the need for high‐quality datasets. Understanding the characteristics of good model cards can help improve the quality of model cards, which is essential for model developers and users. To address these gaps, we study the characteristics of high‐quality model cards. We collect 758 sampled models along with their model cards and adapt a prior framework to manually identify seven categories of content (i.e., categories of “What,” “Why,” “How,” “When,” “Who,” “References,” and “Other”) and their corresponding themes (e.g., “Model Description” in the “What” category) for these model cards. Then, we conduct a survey to explore what constitutes a high‐quality model card. After validation, we define a high‐quality model card as one that incorporates the “Model Overview” and “Model Evaluation” content themes from the “What” category, along with the “Model Usage Guide” content theme from the “How” category, that is, a model card that provides both a brief or detailed introduction to the model and instructions for its usage. Then, we train classifiers to investigate whether high‐quality model cards can be automatically identified. The random forest model achieves the best performance, with an ROC‐AUC of 94.57%. We also investigate the determinants of the quality of model cards. We find that the total word count of the content of model cards, the number of tables that show the key evaluation metrics and results, external links that show the references of external resources, and code blocks that are used to show examples of how to use the model are several of the most significant features that determine the quality of model cards. Finally, we explore the relationship between model card quality and model survival, and find that models with high‐quality model cards tend to remain active significantly longer than those with low‐quality model cards. Our findings provide insights into high‐quality model cards and practical suggestions for model developers, users, researchers, and the HF platform.
- 7 October 2026
- Published by Wiley in Journal of Software: Evolution and Process
- Vol. 38 (10) , e70170
What Makes a Good Model Card?
Hugging Face (HF) is an increasingly popular platform for sharing and discovering machine learning (ML) and artificial intelligence (AI) models. Model cards document these models, and good model cards serve as a vital communication bridge between model developers and users. Without good model cards, it is challenging for users to understand the models and for developers to maintain them. Prior research focused on the documentation process and automatic generation of model cards, and identified the need for high‐quality datasets. Understanding the characteristics of good model cards can help improve the quality of model cards, which is essential for model developers and users. To address these gaps, we study the characteristics of high‐quality model cards. We collect 758 sampled models along with their model cards and adapt a prior framework to manually identify seven categories of content (i.e., categories of “What,” “Why,” “How,” “When,” “Who,” “References,” and “Other”) and their corresponding themes (e.g., “Model Description” in the “What” category) for these model cards. Then, we conduct a survey to explore what constitutes a high‐quality model card. After validation, we define a high‐quality model card as one that incorporates the “Model Overview” and “Model Evaluation” content themes from the “What” category, along with the “Model Usage Guide” content theme from the “How” category, that is, a model card that provides both a brief or detailed introduction to the model and instructions for its usage. Then, we train classifiers to investigate whether high‐quality model cards can be automatically identified. The random forest model achieves the best performance, with an ROC‐AUC of 94.57%. We also investigate the determinants of the quality of model cards. We find that the total word count of the content of model cards, the number of tables that show the key evaluation metrics and results, external links that show the references of external resources, and code blocks that are used to show examples of how to use the model are several of the most significant features that determine the quality of model cards. Finally, we explore the relationship between model card quality and model survival, and find that models with high‐quality model cards tend to remain active significantly longer than those with low‐quality model cards. Our findings provide insights into high‐quality model cards and practical suggestions for model developers, users, researchers, and the HF platform.
- Open Access
- 1 October 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 3 (ISSTA) , 1033-1055
Better Call Grep: Evaluating and Improving Grep-Like Lexical Retrieval for Repository-Level Code Completion
Repository-level code completion remains challenging for large language models (LLMs), as it requires reasoning over cross-file dependencies while under limited context windows. To address this challenge, prior work has adopted Retrieval-Augmented Generation (RAG) frameworks based on semantic indexing or structure-aware graph analysis. Although effective, these approaches introduce substantial computational overhead for index construction and maintenance, which hinders their practicality in real-world development. Motivated by common developer workflows that rely on lightweight search utilities (e.g., ripgrep) to locate relevant code, we revisit a fundamental yet underexplored question: how far can simple, index-free lexical retrieval go in supporting repository-level code completion before more complex retrieval mechanisms become necessary? To answer this question, we systematically explore the potential of lightweight, index-free, intent-aware lexical retrieval through extensive empirical analysis. We first introduce Naive GrepRAG, a baseline framework where LLMs autonomously generate ripgrep commands to localize relevant context. Our preliminary experiments show that even this basic implementation achieves performance comparable to sophisticated graph-based baselines. Further analysis reveals that its effectiveness stems from retrieving code fragments that are lexically precise and spatially closer to the completion site. However, we identify key limitations of this approach, including sensitivity to noisy matches caused by high-frequency ambiguous keywords and context fragmentation due to rigid truncation boundaries. To address these issues, we propose GrepRAG, which augments lexical retrieval with a lightweight post-processing pipeline featuring identifier-weighted re-ranking and structure-aware deduplication. Extensive evaluation on CrossCodeEval and RepoEval_Updated demonstrates that GrepRAG consistently outperforms state-of-the-art (SOTA) methods. In particular, on CrossCodeEval, GrepRAG achieves 7.04–15.58% relative improvement in code exact match (EM) over the best baseline.
- 1 October 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 3 (ISSTA) , 1033-1055
Open AccessBetter Call Grep: Evaluating and Improving Grep-Like Lexical Retrieval for Repository-Level Code Completion
Repository-level code completion remains challenging for large language models (LLMs), as it requires reasoning over cross-file dependencies while under limited context windows. To address this challenge, prior work has adopted Retrieval-Augmented Generation (RAG) frameworks based on semantic indexing or structure-aware graph analysis. Although effective, these approaches introduce substantial computational overhead for index construction and maintenance, which hinders their practicality in real-world development. Motivated by common developer workflows that rely on lightweight search utilities (e.g., ripgrep) to locate relevant code, we revisit a fundamental yet underexplored question: how far can simple, index-free lexical retrieval go in supporting repository-level code completion before more complex retrieval mechanisms become necessary? To answer this question, we systematically explore the potential of lightweight, index-free, intent-aware lexical retrieval through extensive empirical analysis. We first introduce Naive GrepRAG, a baseline framework where LLMs autonomously generate ripgrep commands to localize relevant context. Our preliminary experiments show that even this basic implementation achieves performance comparable to sophisticated graph-based baselines. Further analysis reveals that its effectiveness stems from retrieving code fragments that are lexically precise and spatially closer to the completion site. However, we identify key limitations of this approach, including sensitivity to noisy matches caused by high-frequency ambiguous keywords and context fragmentation due to rigid truncation boundaries. To address these issues, we propose GrepRAG, which augments lexical retrieval with a lightweight post-processing pipeline featuring identifier-weighted re-ranking and structure-aware deduplication. Extensive evaluation on CrossCodeEval and RepoEval_Updated demonstrates that GrepRAG consistently outperforms state-of-the-art (SOTA) methods. In particular, on CrossCodeEval, GrepRAG achieves 7.04–15.58% relative improvement in code exact match (EM) over the best baseline.
- 1 October 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 13 (20) , 47287-47301
Hierarchical Hybrid Aggregation for Secrecy-Constrained Federated Task Offloading in End-Edge–Cloud Collaborative Networks
- 1 October 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 13 (20) , 47287-47301
Hierarchical Hybrid Aggregation for Secrecy-Constrained Federated Task Offloading in End-Edge–Cloud Collaborative Networks
- Open Access
- 3 August 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
Code Refinement with Repository Context: How Far are We?
Code refinement, a process of revising flawed code changes based on the code review comments, is a fundamental practice in software development for ensuring software quality and maintainability. Recent approaches using large language models (LLMs) show promise in automating this process, yet they typically rely only on review comments and a single code snippet, neglecting broader repository context. When a code refinement task requires invoking reviewer-unspecified identifiers (e.g., variables, functions, or classes) that are defined elsewhere in the repository, the LLM must accurately predict the exact identifier signature and usage, otherwise it could easily result in fatal errors. Prior work demonstrates that 90.59% of such knowledge-conflicting hallucinations cause all test cases to fail. Notably, over 60% of real-world code refinement scenarios in most popular Github repositories depend on repository-level knowledge, yet none of the existing code refinement works leverage this information. Moreover, current datasets do not distinguish between context-dependent and context-independent instances, contain many low-quality instances, and rely on lexical metrics (e.g., BLEU) that poorly correlate with human judgment. These limitations hinder reliable evaluation of code refinement tools in real-world, context-dependent scenarios. To address this gap, we introduce ReCoRe-Bench , a high-quality benchmark of 1,000 code refinement instances from 328 Python, Java, and JavaScript repositories that focused on one of the most challenging code refinement scenarios that strictly requires repository-level knowledge reasoning. To ensure repository-level context is required and to achieve strong review-revision alignment, we applied a strict identifier-based heuristic filtering pipeline and manually filtered low quality instances containing vague comments or unrelated review comments. To better evaluate whether an LLM correctly retrieves and resolves identifiers defined in the repository, instead of inferring them from limited local context, we propose a new evaluation metric, ReDIs Match. Additionally, to address the weaknesses of current LLM-powered code refinement methods identified by ReCoRe-Bench , we propose a straightforward method, RepoRefiner , which retrieves repository-level context by collecting the full file content, extracting definitions of its identifiers, and summarizing these contexts to support code refinement. The experiment results show that the RepoRefiner outperforms current code refinement baselines, and the repository-level context is essential for the reliable code refinement. Our findings underscore both the promise and current limitations of LLMs in repository-level code refinement scenarios, highlighting the need for more effective static analysis techniques and prompt engineering for broader repository-level context exploration.
- 3 August 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
Open AccessCode Refinement with Repository Context: How Far are We?
Code refinement, a process of revising flawed code changes based on the code review comments, is a fundamental practice in software development for ensuring software quality and maintainability. Recent approaches using large language models (LLMs) show promise in automating this process, yet they typically rely only on review comments and a single code snippet, neglecting broader repository context. When a code refinement task requires invoking reviewer-unspecified identifiers (e.g., variables, functions, or classes) that are defined elsewhere in the repository, the LLM must accurately predict the exact identifier signature and usage, otherwise it could easily result in fatal errors. Prior work demonstrates that 90.59% of such knowledge-conflicting hallucinations cause all test cases to fail. Notably, over 60% of real-world code refinement scenarios in most popular Github repositories depend on repository-level knowledge, yet none of the existing code refinement works leverage this information. Moreover, current datasets do not distinguish between context-dependent and context-independent instances, contain many low-quality instances, and rely on lexical metrics (e.g., BLEU) that poorly correlate with human judgment. These limitations hinder reliable evaluation of code refinement tools in real-world, context-dependent scenarios. To address this gap, we introduce ReCoRe-Bench , a high-quality benchmark of 1,000 code refinement instances from 328 Python, Java, and JavaScript repositories that focused on one of the most challenging code refinement scenarios that strictly requires repository-level knowledge reasoning. To ensure repository-level context is required and to achieve strong review-revision alignment, we applied a strict identifier-based heuristic filtering pipeline and manually filtered low quality instances containing vague comments or unrelated review comments. To better evaluate whether an LLM correctly retrieves and resolves identifiers defined in the repository, instead of inferring them from limited local context, we propose a new evaluation metric, ReDIs Match. Additionally, to address the weaknesses of current LLM-powered code refinement methods identified by ReCoRe-Bench , we propose a straightforward method, RepoRefiner , which retrieves repository-level context by collecting the full file content, extracting definitions of its identifiers, and summarizing these contexts to support code refinement. The experiment results show that the RepoRefiner outperforms current code refinement baselines, and the repository-level context is essential for the reliable code refinement. Our findings underscore both the promise and current limitations of LLMs in repository-level code refinement scenarios, highlighting the need for more effective static analysis techniques and prompt engineering for broader repository-level context exploration.
- Open Access
- 30 June 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 3 (FSE) , 2976-2999
Hallucinations in LLM-Based Code Summarization: Unveiling, Detection, and Mitigation
Code summarization plays a vital role in program comprehension and software maintenance by generating natural language descriptions to summarize the semantics of code. While Large Language Models (LLMs) have shown remarkable performance in this area, recent empirical studies reveal a critical limitation: LLMs are prone to hallucinations, producing summaries that are factually inaccurate or unfaithful to the source code, potentially misleading developers. In this paper, we propose to unveil, detect, and mitigate hallucinations in LLM-based code summarization. First, we construct Hallu-Eval, a novel dataset for unveiling hallucination phenomena and rigorously evaluating the effectiveness of hallucination detection and mitigation in LLM-based code summarization. It comprises both original code snippets to capture naturally occurring hallucinations and their semantically perturbed counterparts, which are designed to systematically induce challenging logical hallucinations, all complemented with manual hallucination annotations on a curated testbed of 800 code-summary pairs. Next, we propose Hallu-Det, a synergistic approach that combines direct entity-level detection to identify explicit hallucinations with a synonymous mutation-based refinement to reliably confirm or refute more ambiguous cases. Finally, we introduce Hallu-Shield, an inference-time mitigation approach that leverages an external value model to guide LLMs toward producing more faithful summaries without costly retraining of the LLM itself. Extensive experiments show that Hallu-Eval effectively triggers hallucinations, increasing the hallucination rate of models such as Qwen2.5-Coder-7B from 17% to 97% on perturbed code. Our detection approach, Hallu-Det, achieves the best performance among baselines, reaching an F1-score of 0.95 for summaries generated by Qwen2.5-Coder-7B. Moreover, our mitigation method, Hallu-Shield, reduces hallucination rates. For example, it lowers the rate from 66% to 59%, a 10.6% relative reduction, on DeepSeek-Coder-6.7B, while simultaneously improving summary quality, achieving a 74.0% win rate evaluated by an LLM-as-a-judge majority vote ensemble.
- 30 June 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 3 (FSE) , 2976-2999
Open AccessHallucinations in LLM-Based Code Summarization: Unveiling, Detection, and Mitigation
Code summarization plays a vital role in program comprehension and software maintenance by generating natural language descriptions to summarize the semantics of code. While Large Language Models (LLMs) have shown remarkable performance in this area, recent empirical studies reveal a critical limitation: LLMs are prone to hallucinations, producing summaries that are factually inaccurate or unfaithful to the source code, potentially misleading developers. In this paper, we propose to unveil, detect, and mitigate hallucinations in LLM-based code summarization. First, we construct Hallu-Eval, a novel dataset for unveiling hallucination phenomena and rigorously evaluating the effectiveness of hallucination detection and mitigation in LLM-based code summarization. It comprises both original code snippets to capture naturally occurring hallucinations and their semantically perturbed counterparts, which are designed to systematically induce challenging logical hallucinations, all complemented with manual hallucination annotations on a curated testbed of 800 code-summary pairs. Next, we propose Hallu-Det, a synergistic approach that combines direct entity-level detection to identify explicit hallucinations with a synonymous mutation-based refinement to reliably confirm or refute more ambiguous cases. Finally, we introduce Hallu-Shield, an inference-time mitigation approach that leverages an external value model to guide LLMs toward producing more faithful summaries without costly retraining of the LLM itself. Extensive experiments show that Hallu-Eval effectively triggers hallucinations, increasing the hallucination rate of models such as Qwen2.5-Coder-7B from 17% to 97% on perturbed code. Our detection approach, Hallu-Det, achieves the best performance among baselines, reaching an F1-score of 0.95 for summaries generated by Qwen2.5-Coder-7B. Moreover, our mitigation method, Hallu-Shield, reduces hallucination rates. For example, it lowers the rate from 66% to 59%, a 10.6% relative reduction, on DeepSeek-Coder-6.7B, while simultaneously improving summary quality, achieving a 74.0% win rate evaluated by an LLM-as-a-judge majority vote ensemble.
- Open Access
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 257-269
CodeGlance: Understanding Code Reasoning Challenges in LLMs through Multi-Dimensional Feature Analysis
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 257-269
Open AccessCodeGlance: Understanding Code Reasoning Challenges in LLMs through Multi-Dimensional Feature Analysis
- Open Access
- 14 February 2026
- Published by Springer Nature in Journal of Cloud Computing
- Vol. 15 (1)
Cloud-assisted LLM-enhanced datasets for AST hierarchy-aware code summarization model
- 14 February 2026
- Published by Springer Nature in Journal of Cloud Computing
- Vol. 15 (1)
Open AccessCloud-assisted LLM-enhanced datasets for AST hierarchy-aware code summarization model
- 23 January 2026
- Published by Elsevier in Applied Soft Computing
- Vol. 191
Bi-directional distillation enabling progressive lightweight model training in edge computing-based federated learning
- 23 January 2026
- Published by Elsevier in Applied Soft Computing
- Vol. 191
Bi-directional distillation enabling progressive lightweight model training in edge computing-based federated learning
- Open Access
From LLMs to Agents: A Comparative Evaluation of LLMs and LLM-based Agents in Security Patch Detection
The widespread adoption of open-source software (OSS) has accelerated software innovation but also increased security risks due to the rapid propagation of vulnerabilities and silent patch releases. In recent years, large language models (LLMs) and LLM-based agents have demonstrated remarkable capabilities in various software engineering (SE) tasks, enabling them to effectively address software security challenges such as vulnerability detection. However, systematic evaluation of the capabilities of LLMs and LLM-based agents in security patch detection remains limited. To bridge this gap, we conduct a comprehensive evaluation of the performance of LLMs and LLM-based agents for security patch detection. Specifically, we investigate three methods: Plain LLM (a single LLM with a system prompt), Data-Aug LLM (data augmentation based on the Plain LLM), and the ReAct Agent (leveraging the thought-action-observation mechanism). We also evaluate the performance of both commercial and open-source LLMs under these methods and compare these results with those of existing baselines. Furthermore, we analyze the detection performance of these methods across various vulnerability types, and examine the impact of different prompting strategies and context window sizes on the results. Our findings reveal that the Data-Aug LLM achieves the best overall performance, whereas the ReAct Agent demonstrates the lowest false positive rate (FPR). Although baseline methods exhibit strong accuracy, their false positive rates are significantly higher. In contrast, our evaluated methods achieve comparable accuracy while substantially reducing the FPR. These findings provide valuable insights into the practical applications of LLMs and LLM-based agents in security patch detection, highlighting their advantage in maintaining robust performance while minimizing false positive rates.
Open AccessFrom LLMs to Agents: A Comparative Evaluation of LLMs and LLM-based Agents in Security Patch Detection
The widespread adoption of open-source software (OSS) has accelerated software innovation but also increased security risks due to the rapid propagation of vulnerabilities and silent patch releases. In recent years, large language models (LLMs) and LLM-based agents have demonstrated remarkable capabilities in various software engineering (SE) tasks, enabling them to effectively address software security challenges such as vulnerability detection. However, systematic evaluation of the capabilities of LLMs and LLM-based agents in security patch detection remains limited. To bridge this gap, we conduct a comprehensive evaluation of the performance of LLMs and LLM-based agents for security patch detection. Specifically, we investigate three methods: Plain LLM (a single LLM with a system prompt), Data-Aug LLM (data augmentation based on the Plain LLM), and the ReAct Agent (leveraging the thought-action-observation mechanism). We also evaluate the performance of both commercial and open-source LLMs under these methods and compare these results with those of existing baselines. Furthermore, we analyze the detection performance of these methods across various vulnerability types, and examine the impact of different prompting strategies and context window sizes on the results. Our findings reveal that the Data-Aug LLM achieves the best overall performance, whereas the ReAct Agent demonstrates the lowest false positive rate (FPR). Although baseline methods exhibit strong accuracy, their false positive rates are significantly higher. In contrast, our evaluated methods achieve comparable accuracy while substantially reducing the FPR. These findings provide valuable insights into the practical applications of LLMs and LLM-based agents in security patch detection, highlighting their advantage in maintaining robust performance while minimizing false positive rates.