Weifeng Sun

Unit testing is essential for software quality assurance, where a test case typically consists of a test prefix and an oracle, expressed as assertions. In practice, crafting high-quality assertions is non-trivial and time-consuming, as it requires developers to reason carefully about program states and expected behaviors. While recent advances in Large Language Models (LLMs) have shown promise for automating assertion generation (AG), current AG methods often rely on two unrealistic assumptions: (1) the Single-Assertion Formulation (A1), which assumes tests contain only one assertion, and (2) the Known-Position Formulation (A2), which treats AG as a "fill-in-the-blanks" task with pre-defined insertion points. Despite being widely adopted, the realism and implications of these assumptions have not been systematically examined. This paper revisits AG under a realistic Mixed-Assertion Scenario, where tests may contain one or multiple assertions and insertion positions are unavailable at inference time. To examine A1, we first conduct a large-scale empirical study of 358,117 developer-written tests from 7,061 projects. The results show that multi-assertion tests are prevalent, accounting for 40.32% of all tests and appearing in 92.87% of projects. Through manual analysis, we derive a taxonomy comprising ten fine-grained assertion patterns, showing that assertions in multi-assertion tests are rarely independent checks (4.69%) and instead coordinate to validate a unified test objective. To examine A2, we remove ground-truth insertion cues and observe substantial performance degradation, with Exact Match dropping by 11.80%--23.18% overall. This suggests that position cues affect not only where assertions are placed, but also the quality of what to assert. Motivated by these findings, we propose DA-AG, a two-stage framework designed for the realistic Mixed-Assertion Scenario with unknown insertion positions. In the first stage, it predicts assertion insertion positions to construct an assertion skeleton with explicit insertion cues. In the second stage, it generates assertion content conditioned on the resulting skeleton and retrieved exemplar assertion sequences. Extensive experiments across 13 diverse LLMs show that DA-AG consistently outperforms the corresponding one-stage baselines, which directly generate the completed test from the focal method and raw test prefix. DA-AG improves Exact Match by 32.24%--78.08% and CodeBLEU by 2.80%--8.63%, increases real-bug detection on Defects4J by 4--37 exposed bugs and 5--21 unique exposed bugs, and further improves other execution-based metrics, including compilability, bug-finding quality, and mutation scores. Moreover, DA-AG outperforms closed-source LLMs evaluated in a prompt-only setting without task-specific fine-tuning (e.g., GPT-4o and Claude-3.5) in similarity-based quality and real-bug detection.

LLM-based systems increasingly incorporate long-term memory to improve cross-session continuity. However, once insecure coding preferences are stored, they may silently influence security-critical decisions in subsequent generations. In this study, we conduct the first systematic empirical study on the impact of insecure coding preferences stored in long-term memory on the security of LLM-based code generation. We evaluate four LLMs (ChatGPT, Gemini, Qwen, and Grok) across five programming languages (Python, C, C++, Go, and JavaScript). Our results show that insecure memories significantly increase the risk of generating vulnerable code by 2.7-50.3 percentage points (pp). Moreover, they create a 5.4-14.0 percentage-point risk-warning gap, where warning-rate increases lag behind vulnerability-rate increases. Further analysis reveals that insecure memories are difficult to overwrite through normal interactions and can broadly influence model outputs even when prompts are phrased differently. Finally, we evaluate three mitigation strategies: security-requirement appending and memory storage reduce vulnerability rates by 19.7-33.6 pp but may degrade functional correctness by up to 15.9 pp; memory-level safety filtering achieves a 100% detection rate on our evaluated risky memory entries and restores generation behavior to the without-memory baseline. Based on these findings, we provide actionable suggestions to improve the security of long-term memory in LLM-based code generation.

Large language models (LLMs) are widely used for code generation but may reproduce vulnerabilities learned from insecure training data. Prior work mainly relies on inference-time hardening, which introduces auxiliary components and runtime overhead. We present the first systematic study of model editing as a model-level hardening mechanism for secure code generation, directly injecting security knowledge through localized parameter updates. We evaluate three state-of-the-art editing methods across diverse LLM families against CoSec, a representative inference-time approach, examining security effectiveness and robustness, generalization to unseen vulnerabilities, and functional correctness. Model editing substantially outperforms CoSec on seen vulnerabilities, improving security ratios by 15

Deep neural networks (DNNs) have been widely used in many different fields (including for medical diagnosis and autonomous driving). A challenge, however, is that internal defects may cause incorrect DNN behavior, potentially resulting in financial losses and even posing safety threats: DNN reliability and safety require comprehensive, in-depth testing and evaluation. Since DNN testing research began in 2017, various methods and tools have been proposed. This paper systematically surveys advances in DNN testing over the past 9 years, covering topics such as key technologies for testing objectives, noise-sample construction, test-case generation, addressing the test-oracle problem in DNN testing, test datasets, and evaluation metrics. We also examine open-source testing tools, and analyze typical applications and domains for DNN testing. Finally, we provide an in-depth discussion of the challenges faced in DNN testing, and discuss potential future work directions.

Unit testing is a cornerstone of software quality assurance, yet writing high-quality tests remains labor-intensive and cognitively demanding, especially when developers must infer testing objectives directly from code. Recent advances in large language models (LLMs) have enabled automated unit test generation, but these methods take a focal method as input and generate executable test code without explicitly specifying what behavior is being tested or how its correctness should be assessed. This mismatch with real-world developer workflows introduces cognitive overhead, hampers maintainability, and reduces trust in generated tests. In this paper, we propose ReaTSG , a framework for generating natural language test suggestions that explicitly describe from the developer's perspective what behavior should be tested and how it can be validated. ReaTSG contains two key components. First, the Test Suggestion Oracle Construction module leverages LLM-based self-verification to automatically generate high-quality test suggestions for given focal methods. Second, the Retrieval-Augmented Suggestion Learning module couples a dense retriever and a salience-aware generator within a joint training strategy, enabling the retriever to select examples that are not only relevant but beneficial for generation, and guiding the generator to incorporate TF-IDF-based salience weights into its attention mechanism to emphasize informationally important tokens. We construct the first large-scale dataset comprising 55,467 pairs of focal methods and their corresponding suggestions. Extensive experiments across diverse LLMs demonstrate that ReaTSG achieves the best overall performance. Relative to the fine-tuned baselines, it yields average improvements of 17.37% (C-BLEU), 13.15% (S-BLEU), 5.14% (METEOR), 8.21% (ROUGE-L), and 93.04% (CIDEr). Furthermore, we demonstrate that the test suggestions generated by ReaTSG can significantly facilitate the performance of multiple downstream testing tasks.