Shidong Pan
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Shidong Pan has a total of 31 co-authors. Here are the top co-authors by number of shared publications:
- Zhenchang Xing
CSIRO, Canberra, Australia
8 shared publications - Xiaoyu Sun
The Australian National University, Canberra, Australia
4 shared publications - Dianshu Liao
The School of Computing, The Australian National University, Canberra, Australia
3 shared publications - Jiamou Sun
Data61, CSIRO, Australia
3 shared publications - Linyi Han
College of Intelligence and Computing, Tianjin University, Tianjin, China
3 shared publications - Qing Huang
Jiangxi Normal University, Nanchang, China
3 shared publications
Latest Publications
- Open Access
- 11 July 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35 (7) , 1-32
A Comparative Study of Android Performance Issues in Real-World Applications and Literature
Performance issues in Android applications significantly undermine users’ experience, engagement, and retention, which is a long-lasting research topic in academia. Unlike functionality issues, performance issues are more difficult to diagnose and resolve due to their complex root causes, which often emerge only under specific conditions or payloads. Although many efforts have attempted to mitigate the impact of performance issues by developing methods to automatically identify and resolve them, it remains unclear if this objective has been fulfilled, and the existing approaches indeed targeted on the most critical performance issues encountered in real-world settings. To this end, we conduct a large-scale comparative study of Android performance issues in real-world applications and literature. Specifically, we started by investigating real-world performance issues, their underlying root causes (i.e., contributing factors), and common code patterns. We then took an additional step to empirically summarize existing approaches and datasets through a literature review, assessing how well academic research reflects the real-world challenges faced by developers and users. Our comparison results show a substantial divergence exists in the primary performance concerns of researchers, developers, and users. Among all the identified factors, 57.14% have not been examined in academic research, while a substantial 63.41% remain unaddressed by existing tools, and 70.73% lack corresponding datasets. This stark contrast underscores a substantial gap in our understanding and management of performance issues. Consequently, it is crucial for our community to intensify efforts to bridge these gaps and achieve comprehensive detection and resolution of performance issues.
- 11 July 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 35 (7) , 1-32
Open AccessA Comparative Study of Android Performance Issues in Real-World Applications and Literature
Performance issues in Android applications significantly undermine users’ experience, engagement, and retention, which is a long-lasting research topic in academia. Unlike functionality issues, performance issues are more difficult to diagnose and resolve due to their complex root causes, which often emerge only under specific conditions or payloads. Although many efforts have attempted to mitigate the impact of performance issues by developing methods to automatically identify and resolve them, it remains unclear if this objective has been fulfilled, and the existing approaches indeed targeted on the most critical performance issues encountered in real-world settings. To this end, we conduct a large-scale comparative study of Android performance issues in real-world applications and literature. Specifically, we started by investigating real-world performance issues, their underlying root causes (i.e., contributing factors), and common code patterns. We then took an additional step to empirically summarize existing approaches and datasets through a literature review, assessing how well academic research reflects the real-world challenges faced by developers and users. Our comparison results show a substantial divergence exists in the primary performance concerns of researchers, developers, and users. Among all the identified factors, 57.14% have not been examined in academic research, while a substantial 63.41% remain unaddressed by existing tools, and 70.73% lack corresponding datasets. This stark contrast underscores a substantial gap in our understanding and management of performance issues. Consequently, it is crucial for our community to intensify efforts to bridge these gaps and achieve comprehensive detection and resolution of performance issues.
- Open Access
- 30 June 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 3 (FSE) , 321-343
Deployability-Centric Infrastructure-as-Code Generation: Fail, Learn, Refine, and Succeed through LLM-Empowered DevOps Simulation
Infrastructure-as-Code (IaC) generation holds significant promise for automating the provisioning of cloud infrastructure. Recent advances in Large Language Models (LLMs) present a promising opportunity to democratize IaC development by generating deployable infrastructure templates from natural language descriptions. However, current evaluation focuses on syntactic correctness while ignoring deployability, the critical measure of the utility of IaC configuration files. Six state-of-the-art LLMs performed poorly on deployability, achieving only 20.8∼30.2% deployment success rate on the first attempt. In this paper, we construct DPIaC-Eval, the first deployability-centric IaC template benchmark consisting of 153 real-world scenarios across 58 unique services. Also, we propose an LLM-based deployability-centric framework, dubbed IaCGen, that uses an iterative feedback mechanism encompassing format verification, syntax checking, and live deployment stages, thereby closely mirroring the real DevOps workflows. Results show that IaCGen can make 54.6∼91.6% generated IaC templates from all evaluated models deployable in the first 10 iterations. Additionally, human-in-the-loop feedback that provide direct guidance for the deployability errors, can further boost the performance to over 90% passItr@25 on all evaluated LLMs. Furthermore, we explore the trustworthiness of the generated IaC templates on user intent alignment and security compliance. The poor performance (25.2% user requirement coverage and 8.4% security compliance rate) indicates a critical need for continued research in this domain.
- 30 June 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 3 (FSE) , 321-343
Open AccessDeployability-Centric Infrastructure-as-Code Generation: Fail, Learn, Refine, and Succeed through LLM-Empowered DevOps Simulation
Infrastructure-as-Code (IaC) generation holds significant promise for automating the provisioning of cloud infrastructure. Recent advances in Large Language Models (LLMs) present a promising opportunity to democratize IaC development by generating deployable infrastructure templates from natural language descriptions. However, current evaluation focuses on syntactic correctness while ignoring deployability, the critical measure of the utility of IaC configuration files. Six state-of-the-art LLMs performed poorly on deployability, achieving only 20.8∼30.2% deployment success rate on the first attempt. In this paper, we construct DPIaC-Eval, the first deployability-centric IaC template benchmark consisting of 153 real-world scenarios across 58 unique services. Also, we propose an LLM-based deployability-centric framework, dubbed IaCGen, that uses an iterative feedback mechanism encompassing format verification, syntax checking, and live deployment stages, thereby closely mirroring the real DevOps workflows. Results show that IaCGen can make 54.6∼91.6% generated IaC templates from all evaluated models deployable in the first 10 iterations. Additionally, human-in-the-loop feedback that provide direct guidance for the deployability errors, can further boost the performance to over 90% passItr@25 on all evaluated LLMs. Furthermore, we explore the trustworthiness of the generated IaC templates on user intent alignment and security compliance. The poor performance (25.2% user requirement coverage and 8.4% security compliance rate) indicates a critical need for continued research in this domain.
- Open Access
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1018-1029
When AI Takes the Wheel: Security Analysis of Framework-Constrained Program Generation
In recent years, the AI wave has grown rapidly in software development. Even novice developers can now design and generate complex framework-constrained software systems based on their high-level requirements with the help of Large Language Models (LLMs). However, when LLMs gradually "take the wheel" of software development, developers may only check whether the program works. They often miss security problems hidden in how the generated programs are implemented. In this work, we investigate the security properties of framework-constrained programs generated by state-of-the-art LLMs. We focus specifically on Chrome extensions due to their complex security model involving multiple privilege boundaries and isolated components. To achieve this, we built ChromeSecBench, a dataset with 140 prompts based on known vulnerable extensions. We used these prompts to instruct nine state-of-the-art LLMs to generate complete Chrome extensions, and then analyzed them for vulnerabilities across three dimensions: scenario types, model differences, and vulnerability categories. Our results show that LLMs produced vulnerable programs at alarmingly high rates (18%-50%), particularly in Authentication & Identity and Cookie Management scenarios (up to 83% and 78% respectively). Most vulnerabilities exposed sensitive browser data like cookies, history, or bookmarks to untrusted code. Interestingly, we found that advanced reasoning models performed worse, generating more vulnerabilities than simpler models. These findings highlight a critical gap between LLMs’ coding skills and their ability to write secure framework-constrained programs.
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1018-1029
Open AccessWhen AI Takes the Wheel: Security Analysis of Framework-Constrained Program Generation
In recent years, the AI wave has grown rapidly in software development. Even novice developers can now design and generate complex framework-constrained software systems based on their high-level requirements with the help of Large Language Models (LLMs). However, when LLMs gradually "take the wheel" of software development, developers may only check whether the program works. They often miss security problems hidden in how the generated programs are implemented. In this work, we investigate the security properties of framework-constrained programs generated by state-of-the-art LLMs. We focus specifically on Chrome extensions due to their complex security model involving multiple privilege boundaries and isolated components. To achieve this, we built ChromeSecBench, a dataset with 140 prompts based on known vulnerable extensions. We used these prompts to instruct nine state-of-the-art LLMs to generate complete Chrome extensions, and then analyzed them for vulnerabilities across three dimensions: scenario types, model differences, and vulnerability categories. Our results show that LLMs produced vulnerable programs at alarmingly high rates (18%-50%), particularly in Authentication & Identity and Cookie Management scenarios (up to 83% and 78% respectively). Most vulnerabilities exposed sensitive browser data like cookies, history, or bookmarks to untrusted code. Interestingly, we found that advanced reasoning models performed worse, generating more vulnerabilities than simpler models. These findings highlight a critical gap between LLMs’ coding skills and their ability to write secure framework-constrained programs.
- Open Access
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 34-37
JUnitGenie: A Framework for Path-Sensitive Unit Test Generation with Large Language Models
Unit testing is essential for software quality assurance, yet writing and maintaining tests remains time-consuming and error-prone. To address this challenge, researchers have proposed techniques for automating unit test generation, including heuristic-based methods and more recent approaches that leverage large language models (LLMs). However, these approaches are inherently path-insensitive because they rely on fixed heuristics or limited contextual information and fail to reason about deep control-flow structures, which leads to inadequate coverage, especially for deep execution paths. JUnitGenie extracts code knowledge from Java projects, distills it into structured prompts, and guides LLMs to generate high-coverage tests. We evaluate JUnitGenie on 2,258 complex focal methods from ten real-world Java projects. Results show that JUnitGenie improves branch coverage by an average of 29.60% and line coverage by 31.00% over both heuristic and LLM-based baselines, and generalizes effectively across both closed- and open-source foundation models. JUnitGenie is available at https://github.com/Dianshu-Liao/JUnitGenie-ICSE-Demo, and the demo video is available at https://www.youtube.com/watch?v=-jWJaf661mw.
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 34-37
Open AccessJUnitGenie: A Framework for Path-Sensitive Unit Test Generation with Large Language Models
Unit testing is essential for software quality assurance, yet writing and maintaining tests remains time-consuming and error-prone. To address this challenge, researchers have proposed techniques for automating unit test generation, including heuristic-based methods and more recent approaches that leverage large language models (LLMs). However, these approaches are inherently path-insensitive because they rely on fixed heuristics or limited contextual information and fail to reason about deep control-flow structures, which leads to inadequate coverage, especially for deep execution paths. JUnitGenie extracts code knowledge from Java projects, distills it into structured prompts, and guides LLMs to generate high-coverage tests. We evaluate JUnitGenie on 2,258 complex focal methods from ten real-world Java projects. Results show that JUnitGenie improves branch coverage by an average of 29.60% and line coverage by 31.00% over both heuristic and LLM-based baselines, and generalizes effectively across both closed- and open-source foundation models. JUnitGenie is available at https://github.com/Dianshu-Liao/JUnitGenie-ICSE-Demo, and the demo video is available at https://www.youtube.com/watch?v=-jWJaf661mw.
- 27 November 2025
- Published by Springer Nature in Automated Software Engineering
- Vol. 33 (1) , 35
Domain-constrained synthesis of inconsistent key aspects in textual vulnerability descriptions
- 27 November 2025
- Published by Springer Nature in Automated Software Engineering
- Vol. 33 (1) , 35
Domain-constrained synthesis of inconsistent key aspects in textual vulnerability descriptions
- Open Access
Domain-constrained Synthesis of Inconsistent Key Aspects in Textual Vulnerability Descriptions
Textual Vulnerability Descriptions (TVDs) are crucial for security analysts to understand and address software vulnerabilities. However, the key aspect inconsistencies in TVDs from different repositories pose challenges for achieving a comprehensive understanding of vulnerabilities. Existing approaches aim to mitigate inconsistencies by aligning TVDs with external knowledge bases, but they often discard valuable information and fail to synthesize comprehensive representations. In this paper, we propose a domain-constrained LLM-based synthesis framework for unifying key aspects of TVDs. Our framework consists of three stages: 1) Extraction, guided by rule-based templates to ensure all critical details are captured; 2) Self-evaluation, using domain-specific anchor words to assess semantic variability across sources; and 3) Fusion, leveraging information entropy to reconcile inconsistencies and prioritize relevant details. This framework improves synthesis performance, increasing the F1 score for key aspect augmentation from 0.82 to 0.87, while enhancing comprehension and efficiency by over 30\%. We further develop Digest Labels, a practical tool for visualizing TVDs, which human evaluations show significantly boosts usability.
Open AccessDomain-constrained Synthesis of Inconsistent Key Aspects in Textual Vulnerability Descriptions
Textual Vulnerability Descriptions (TVDs) are crucial for security analysts to understand and address software vulnerabilities. However, the key aspect inconsistencies in TVDs from different repositories pose challenges for achieving a comprehensive understanding of vulnerabilities. Existing approaches aim to mitigate inconsistencies by aligning TVDs with external knowledge bases, but they often discard valuable information and fail to synthesize comprehensive representations. In this paper, we propose a domain-constrained LLM-based synthesis framework for unifying key aspects of TVDs. Our framework consists of three stages: 1) Extraction, guided by rule-based templates to ensure all critical details are captured; 2) Self-evaluation, using domain-specific anchor words to assess semantic variability across sources; and 3) Fusion, leveraging information entropy to reconcile inconsistencies and prioritize relevant details. This framework improves synthesis performance, increasing the F1 score for key aspect augmentation from 0.82 to 0.87, while enhancing comprehension and efficiency by over 30\%. We further develop Digest Labels, a practical tool for visualizing TVDs, which human evaluations show significantly boosts usability.
- Open Access
Towards Context-aware Mobile Privacy Notice: Implementation of A Deployable Contextual Privacy Policies Generator
Lengthy and legally phrased privacy policies impede users' understanding of how mobile applications collect and process personal data. Prior work proposed Contextual Privacy Policies (CPPs) for mobile apps to display shorter policy snippets only in the corresponding user interface contexts, but the pipeline could not be deployable in real-world mobile environments. In this paper, we present PrivScan, the first deployable CPP Software Development Kit (SDK) for Android. It captures live app screenshots to identify GUI elements associated with types of personal data and displays CPPs in a concise, user-facing format. We provide a lightweight floating button that offers low-friction, on-demand control. The architecture leverages remote deployment to decouple the multimodal backend pipeline from a mobile client comprising five modular components, thereby reducing on-device resource demands and easing cross-platform portability. A feasibility-oriented evaluation shows an average execution time of 9.15\,s, demonstrating the practicality of our approach. The source code of PrivScan is available at https://github.com/buyanghc/PrivScan and the demo video can be found at https://www.youtube.com/watch?v=ck-25otfyHc.
Open AccessTowards Context-aware Mobile Privacy Notice: Implementation of A Deployable Contextual Privacy Policies Generator
Lengthy and legally phrased privacy policies impede users' understanding of how mobile applications collect and process personal data. Prior work proposed Contextual Privacy Policies (CPPs) for mobile apps to display shorter policy snippets only in the corresponding user interface contexts, but the pipeline could not be deployable in real-world mobile environments. In this paper, we present PrivScan, the first deployable CPP Software Development Kit (SDK) for Android. It captures live app screenshots to identify GUI elements associated with types of personal data and displays CPPs in a concise, user-facing format. We provide a lightweight floating button that offers low-friction, on-demand control. The architecture leverages remote deployment to decouple the multimodal backend pipeline from a mobile client comprising five modular components, thereby reducing on-device resource demands and easing cross-platform portability. A feasibility-oriented evaluation shows an average execution time of 9.15\,s, demonstrating the practicality of our approach. The source code of PrivScan is available at https://github.com/buyanghc/PrivScan and the demo video can be found at https://www.youtube.com/watch?v=ck-25otfyHc.
- Open Access
- 29 April 2025
- Published by Elsevier in Information and Software Technology
- Vol. 184
An analysis of privacy regulations and user concerns of finance mobile applications
- 29 April 2025
- Published by Elsevier in Information and Software Technology
- Vol. 184
Open AccessAn analysis of privacy regulations and user concerns of finance mobile applications
- Open Access
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-8
Privacy Meets Explainability: Managing Confidential Data and Transparency Policies in LLM-Empowered Science
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-8
Open AccessPrivacy Meets Explainability: Managing Confidential Data and Transparency Policies in LLM-Empowered Science
- 27 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 51 (2) , 466-483
Do Chase Your Tail! Missing Key Aspects Augmentation in Textual Vulnerability Descriptions of Long-Tail Software Through Feature Inference
Augmenting missing key aspects in Textual Vulnerability Descriptions (TVDs) is crucial for effective vulnerability analysis. For instance, in TVDs, key aspects include Attack Vector, Vulnerability Type , among others. These key aspects help security engineers understand and address the vulnerability in a timely manner. For software with a large user base (non-long-tail software), augmenting these missing key aspects has significantly advanced vulnerability analysis and software security research. However, software instances with a limited user base (long-tail software) often get overlooked due to inconsistency software names, TVD limited availability, and domain-specific jargon, which complicates vulnerability analysis and software repairs. In this paper, we introduce a novel software feature inference framework designed to augment the missing key aspects of TVDs for long-tail software. Firstly, we tackle the issue of nonstandard software names found in community-maintained vulnerability databases by cross-referencing government databases with Common Vulnerabilities and Exposures (CVEs). Next, we employ Large Language Models (LLMs) to generate the missing key aspects. However, the limited availability of historical TVDs restricts the variety of examples. To overcome this limitation, we utilize the Common Weakness Enumeration (CWE) to classify all TVDs and select cluster centers as representative examples. To ensure accuracy, we present Natural Language Inference (NLI) models specifically designed for long-tail software. These models identify and eliminate incorrect responses. Additionally, we use a wiki repository to provide explanations for proprietary terms. Our evaluations demonstrate that our approach significantly improves the accuracy of augmenting missing key aspects of TVDs for log-tail software from 0.27 to 0.56 (+107%). Interestingly, the accuracy of non-long-tail software also increases from 64% to 71%. As a result, our approach can be useful in various downstream tasks that require complete TVD information.
- 27 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 51 (2) , 466-483
Do Chase Your Tail! Missing Key Aspects Augmentation in Textual Vulnerability Descriptions of Long-Tail Software Through Feature Inference
Augmenting missing key aspects in Textual Vulnerability Descriptions (TVDs) is crucial for effective vulnerability analysis. For instance, in TVDs, key aspects include Attack Vector, Vulnerability Type , among others. These key aspects help security engineers understand and address the vulnerability in a timely manner. For software with a large user base (non-long-tail software), augmenting these missing key aspects has significantly advanced vulnerability analysis and software security research. However, software instances with a limited user base (long-tail software) often get overlooked due to inconsistency software names, TVD limited availability, and domain-specific jargon, which complicates vulnerability analysis and software repairs. In this paper, we introduce a novel software feature inference framework designed to augment the missing key aspects of TVDs for long-tail software. Firstly, we tackle the issue of nonstandard software names found in community-maintained vulnerability databases by cross-referencing government databases with Common Vulnerabilities and Exposures (CVEs). Next, we employ Large Language Models (LLMs) to generate the missing key aspects. However, the limited availability of historical TVDs restricts the variety of examples. To overcome this limitation, we utilize the Common Weakness Enumeration (CWE) to classify all TVDs and select cluster centers as representative examples. To ensure accuracy, we present Natural Language Inference (NLI) models specifically designed for long-tail software. These models identify and eliminate incorrect responses. Additionally, we use a wiki repository to provide explanations for proprietary terms. Our evaluations demonstrate that our approach significantly improves the accuracy of augmenting missing key aspects of TVDs for log-tail software from 0.27 to 0.56 (+107%). Interestingly, the accuracy of non-long-tail software also increases from 64% to 71%. As a result, our approach can be useful in various downstream tasks that require complete TVD information.