Yun Lin

Large language models show promise for code generation, yet their effectiveness in producing high-performance GPU kernels remains insufficiently understood. Recent agentic workflows improve over direct prompting through specialized generation roles and execution feedback, but our characterization of a representative workflow reveals failures in optimization-strategy selection, hardware-specific instruction realization, and shape-dependent parameter selection. We present AlphaCu, a transformation-driven synthesis framework for LLM-based GPU kernel generation. AlphaCu treats kernel generation as structured synthesis over explicit optimization decisions rather than end-to-end code emission. Specifically, AlphaCu represents these decisions as scoped transformation passes maintained across agent-coordinated synthesis stages, enabling compatibility checking and failure attribution. AlphaCu binds selected hardware-sensitive operations to pre-validated microkernels and uses target-device measurements to revise transformation choices and tune runtime configurations. We evaluate AlphaCu on 104 test cases covering single and fused operators across four GPU architectures: NVIDIA H100, A100, V100, and AMD MI210. Under comparable generation budgets, AlphaCu achieves 100% functional correctness on our benchmark suite and delivers an average speedup of 5.64× over prior LLM workflows, while remaining competitive with vendor libraries and tensor compilers.

Large Language Models (LLMs) have significantly improved programming efficiency by parsing natural language into code snippets. However, their performance degrades significantly as requirements scale; when faced with multi-modal documents containing hundreds of scenarios, LLMs often produce incorrect implementations or omit crucial constraints. Observing LLMs' ever-evolving capability and their persistent stochastic hallucination, we raise a question: whether it is possible to make LLM-based agentic programming go beyond "code generation" to "requirement compilation", i.e., whether programmers can produce a runnable system by only accomplishing (non-trivial) requirement documents? In this work, we take a first step by proposing the ARC (Agentic Requirement Compilation) technique to parse a multi-modal requirement document, describing hundreds of scenarios in a DSL format, into a runnable software system. In addition to the source code, ARC also generates software engineering artifacts including (1) a modular design that spans the user interface, API interface, and database, (2) enriched test cases for each interface (including unit tests, modular tests, and integration tests), and (3) detailed traceability across all artifacts for software maintenance. Our approach employs a bidirectional test-driven agentic loop: (1) a top-down architecture phase that decomposes requirements into UI, API, and database interfaces, each of which is equipped with verifiable test suites, and (2) a bottom-up implementation phase where agents generate code that must satisfy the generated tests. Throughout this process, ARC maintains strict traceability across requirements, design, and code to facilitate intelligent asset reuse and follow-up maintenance. We evaluate ARC on two complementary benchmarks, i.e., a depth-oriented benchmark of 6 runnable web systems spanning 50-200 requirement scenarios, and the breadth-oriented AppForge benchmark comprising 101 Android app generation tasks. Across 3 independent trials, ARC outperforms all state-of-the-art LLM-based baselines, with the generated web systems passing on average 50.6% more GUI tests, and achieving 100% compile success and 68.3% test case pass rate on AppForge. In addition, a user study with 21 participants shows that participants with limited programming experience successfully write DSL-based documents consisting of 50 to 174 scenarios, within 5.6 hours on average, to generate a runnable system such as a real-world ticket-booking system of around 10K lines of code with maintainable architecture.

Test cases are valuable assets for maintaining software quality. State-of-the-art automated test generation techniques typically focus on maximizing program branch coverage or translating focal methods into test code. However, in contrast to branch coverage or code-to-test translation, practical tests are written out of the need to validate whether a requirement has been fulfilled. Specifically, a test usually reflects a developer’s validation intention for a particular scenario of a program function, regarding (1) what is the test scenario of a program function? and (2) what is the expected behavior under such a scenario? Without taking such intention into account, generated tests are less likely to be adopted in practice. In this work, we propose IntentionTest, which generates project-specific tests given the description of validation intention. The design is motivated by two insights: (1) rationale insight : the description of validation intention regarding scenario description and behavioral expectation, compared to coverage and focal code, carries more crucial information about what to test ; and (2) technical insight : practical test code exhibits high duplication, indicating that existing tests are highly reusable for how to test . Therefore, IntentionTest adopts a retrieval-and-edit manner. First, given a focal code and a description of validation intention consisting of a test objective with test precondition and expected results, IntentionTest retrieves a reusable test in the project as the test reference. Then, IntentionTest edits the test reference with an LLM regarding the validation intention toward the target test. To help the target test include a project-specific test prefix and a relevant assertion, IntentionTest further explores the software project to identify crucial code facts (i.e., relevant API/code to call and global variables to refer to in the test) as important context for the test generation. We extensively evaluate IntentionTest against four baselines (TELPA, DA, ChatTester, and EvoSuite) on 3,680 test cases from 12 open-source projects. Compared to state-of-the-art baselines, with a given validation intention, IntentionTest can (1) generate tests far more semantically relevant to ground-truth tests by (i) achieving common mutation scores 28.1% to 37.6% higher and (ii) achieving common coverage ratios 16.9% to 23.9% higher; and (2) achieve successful-pass rates 23.7% to 49.0% higher.

With the emergence of deep learning, semantic code search has been widely adopted in both academia and industry. These approaches embed natural-language queries and code snippets into a shared embedding space and retrieve results based on vector similarity. Despite their strong performance on benchmark datasets, they often suffer from poor explainability and generalization. Retrieved code may appear semantically similar yet miss critical functional requirements of the query, while providing no explanation of why the result was retrieved. Moreover, such failures become more severe under distribution shift, where models struggle to generalize to unseen benchmarks. In this work, we propose XSearch, an intrinsically explainable code search framework. Our key insight is that, by relying on global embedding similarity, all existing retrievers inherently take an inductive view. They learn statistical patterns, rather than truly understand the query's functional requirements. Therefore, we address the problem by reformulating code search as a deductive concept alignment problem. At a high level, XSearch (i) identifies functional concepts in the query and (ii) explicitly aligns them with corresponding code statements. This explain-then-predict design not only produces inherent concept-level explanations, but also mitigates shortcut learning that harms out-of-distribution generalization. We train an encoder with explicit concept-alignment objectives and perform retrieval through explicit matching between query concepts and code statements. Experiments show that, when trained on CodeSearchNet with a small model size (GraphCodeBERT with 125M parameters), XSearch improves performance on out-of-distribution benchmarks from 0.02 to 0.33 (15×) over eight state-of-the-art retrievers, and consistently outperforms both encoder-based and decoder-based baselines with up to 7B parameters. A controlled user study further demonstrates that concept-alignment explanations enable users to accept or reject retrieved results both faster and more accurately. The source code is publicly available at https://github.com/code-philia/Xsearch.

Issue localization, which identifies code locations requiring modification from issue descriptions, is a critical step in automated software maintenance. Existing approaches predominantly attempt to directly align issue descriptions with code elements, yet often struggle due to the inherent abstraction gap between the issue description and code implementation. Seeking alternative signals, our theoretical analysis suggests that test suites can serve as executable proxies for requirements, reducing localization uncertainty by 7.73 bits of entropy on average. A large-scale empirical study on 18 repositories validates this premise: existing tests cover 96.98% of ground-truth files, and the two-hop pathway yields stronger semantic connectivity than direct matching in 82.4% of cases. Despite their potential, leveraging tests for localization faces two key challenges: the semantic gap separating issue descriptions from test identifiers, and the substantial noise in execution traces from infrastructure code. To address these, we propose IssueExec, which bridges the semantic gap through domain-knowledge-enhanced test representations and filters noise via hierarchical trace analysis. Experiments on SWE-bench Lite show that IssueExec achieves state-of-the-art performance, improving function-level Recall@1 by 41.57% over the strongest baseline. When integrated into the Agentless pipeline, IssueExec resolves 17.72% more issues, demonstrating practical downstream benefits.

Visual language model (VLM) agents show great promise in automating graphical user interface (GUI) testing against requirements in natural language. However, the probabilistic nature of language models can have inherent hallucinations. Therefore, given a detected inconsistency between the requirement and the web application, it is hard to distinguish whether it stems from the hallucination or a real application bug. Addressing this issue presents two core technical challenges: (1) limited capability and accuracy in deriving implicit test oracles, where the agent must act as its own oracle to implicitly decide if the application’s behavior is correct without guidance, and (2) limited reliability due to probabilistic inference, where an LLM’s inconsistent reasoning undermines its trustworthiness as an oracle. We introduce WebTestPilot, a neurosymbolic LLM-based approach that addresses both challenges through symbolization. WebTestPilot detects and abstracts critical GUI elements of a web application into symbolic variables. This design improves reliability by constraining assertion generation to operations grounded in explicitly defined symbols, thereby reducing unconstrained or inconsistent reasoning. At the same time, it improves accuracy by representing application states and their relationships in a structured symbolic form, which increases the likelihood of the agent recognizing data, causal, and temporal dependencies across states. Together, these capabilities enable WebTestPilot to generate reliable and accurate test oracles that capture meaningful implicit expectations derived from test requirements. To advance research in this area, we build a benchmark of bug-injected web apps for evaluating NL-to-E2E testing. The results show that WebTestPilot achieves a task completion rate of 99%, with 96% precision and 96% recall in bug detection, outperforming the best baseline (+70 precision, +27 recall). The agent generalizes across diverse natural language inputs (i.e., those containing typos, grammatical errors, redundant sentences, stylistic restyling, or abbreviations) and model scales (3B–72B). In a real-world deployment with a no-code platform, WebTestPilot discovered 8 bugs during development, including data binding, UI, and navigation issues.

Test cases are essential for software development and maintenance. In practice, developers derive multiple test cases from an implicit pattern based on their understanding of requirements and inference of diverse test scenarios, each validating a specific behavior of the focal method. However, producing comprehensive tests is time-consuming and error-prone: many important tests that should have accompanied the initial test are added only after a significant delay, sometimes only after bugs are triggered. Existing automated test generation techniques largely focus on code coverage. Yet in real projects, practical tests are seldom driven by code coverage alone, since test scenarios do not necessarily align with control-flow branches. Instead, test scenarios originate from requirements, which are often undocumented and implicitly embedded in a project's design and implementation. However, developer-written tests are frequently treated as executable specifications; thus, even a single initial test that reflects the developer's intent can reveal the underlying requirement and the diverse scenarios that should be validated. In this work, we propose TestGeneralizer, a framework for generalizing test cases to comprehensively cover test scenarios. TestGeneralizer orchestrates three stages: (1) enhancing the understanding of the requirement and scenario behind the focal method and initial test; (2) generating a test scenario template and crystallizing it into various test scenario instances; and (3) generating and refining executable test cases from these instances. To ensure accuracy and completeness, TestGeneralizer combines rule-based prompts, automatically optimized via a prompt auto-tuning technique, with crucial project knowledge retrieved through program analysis. We evaluate TestGeneralizer against three state-of-the-art baselines (EvoSuite, gpt-o4-mini, and ChatTester) on 12 open-source Java projects, covering 506 multi-test focal methods and 1,637 test scenarios. TestGeneralizer achieves significant improvements: +57.67% and +59.62% over EvoSuite, +37.44% and +32.82% over gpt-o4-mini, and +31.66% and +23.08% over ChatTester, in mutation-based and LLM-assessed scenario coverage, respectively. In a field study, we submitted 27 generalized tests overlooked by developers; 16 were accepted and merged into official repositories, demonstrating the practical usefulness of TestGeneralizer.