TLTianlin Li

Tianlin Li

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Tianlin Li has a total of 98 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    KGMACG: Knowledge-Guided Multi-Agent Orchestration for Scalable Application-Level Code Generation

    by Bo Yang, Xiao Zhang, Weisong Sun, Guohao Ma, Han Liu, Xiaowei Yang, Qian Xiong, Yiran Zhang, Tianlin Li, Yang Liu, et al.

    Automated code generation driven by Large Language Models (LLMs) has enhanced development efficiency, yet generating complex application-level software code remains challenging. Multi-agent frameworks show potential, but existing methods perform inadequately in large-scale application-level software code generation: they fail to maintain a semantically reasonable project structure and generate fragmented outputs that lack traceable requirement-to-code mappings, making them difficult to extend or scale. To address these limitations, this article proposes KGMACG , a K nowledge- G uided M ulti- A gent framework for scalable C ode G eneration. KGMACG orchestrates three specialized agents in a closed-loop: the Code Organization and Planning Agent (COPA) transforms software requirements specification (SRS) and architectural design document (ADD) into a modular build plan and project skeleton; the Coding Agent (CA) synthesizes repository-level code guided by a five-pillar knowledge base; and the Testing Agent (TA) continuously generates unit tests and feeds failure traces back for rectification. The loop terminates only when the project compiles and achieves \(\geq\) 95% requirement coverage, guaranteeing both syntactic correctness and functional completeness. We evaluate KGMACG on three industrial-scale case studies (E-Commerce, Campus Security, Stock Trading) against six state-of-the-art multi-agent baselines: MetaGPT, AutoGen, CAMEL, CrewAI, ChatDev, and CodeAgent. With the same backbone LLMs (DeepSeek R1 and gpt-5-codex-medium), the results indicate that KGMACG advances the automation of application-level software development.


  • Open Access

    Not as Sweet by Another Name: An Empirical Study of Format Robustness in LLM Document Workflows

    by Xiaoyu Zhang, Xianyun Cheng, Tianlin Li, Yuwei Zheng, Yue Yang, Yang Liu

    LLM-driven software systems are rapidly evolving from plain-text conversations to document-centric end-to-end workflows, where the same semantic content can be delivered in diverse document formats (e.g., CSV) through file upload interfaces. Yet existing testing work focuses on the robustness and reliability of models and systems whose input is a single prompt string, leaving a critical question unanswered: Can these document workflows maintain robust behaviors when the same content arrives in a different document format? To fill the gap, in this paper, we propose a format-aware metamorphic testing framework with three metamorphic relations to comprehensively evaluate the format robustness of end-to-end LLM document workflows. Based on this framework, we conduct a large-scale empirical study spanning four representative LLM workflows, four real-world tasks, and four document formats, comprising a total of 48,000 workflow executions. Our findings reveal that format variation poses a systematic and serious threat. Merely switching formats can cause accuracy to drop by up to 53.63% and trigger decision drifts in over 41% of instances. We further design lightweight mitigation strategies from the users’ perspective that recover up to 44.21% of format-induced decision drift without model retraining. Our study demonstrates that document format is not a neutral wrapper but a critical factor affecting the reliability of LLM software systems, calling for corresponding testing and safeguards in the deployment in real-world high-stakes scenarios.

  • Open Access

    Bugs in Modern LLM Agent Frameworks: An Empirical Study

    by Xinxue Zhu, Jiacong Wu, Xiaoyu Zhang, Tianlin Li, Yanzhou Mu, Juan Zhai, Chao Shen, Chunrong Fang, Yang Liu


  • Open Access

    Fairness Testing of Large Language Models in Role-Playing

    by Xinyue Li, Zhenpeng Chen, Jie M. Zhang, Ying Xiao, Tianlin Li, Weisong Sun, Yang Liu, Yiling Lou, Xuanzhe Liu

    Large Language Models (LLMs) have become foundational in modern language-driven software applications, profoundly influencing daily life. A critical technique in leveraging their potential is role-playing, where LLMs simulate diverse roles to enhance their real-world utility. However, while research has highlighted the presence of social biases in LLM outputs, it remains unclear whether and to what extent these biases emerge during role-playing scenarios. In this paper, we conduct an empirical study on fairness testing of LLMs in role-playing scenarios. To enable this testing, we use LLMs to generate 550 social roles spanning a comprehensive set of 11 demographic attributes, producing 33,000 role-specific questions that target various forms of bias. These questions, covering Yes/No, multiple-choice, and open-ended formats, are designed to prompt LLMs to adopt specific roles and respond accordingly. We employ a combination of rule-based and LLM-based strategies to identify biased responses, rigorously validated through human evaluation. Using the generated questions as the test cases, we conduct extensive evaluations of 10 advanced LLMs. The evaluation reveal 107,580 biased responses across the studied LLMs, with individual models yielding between 7,579 and 16,963 biased responses, underscoring the prevalence of bias in role-playing contexts. To support future research, we have publicly released the dataset, along with all scripts and experimental results.


  • Enhancing Stereo Matching Domain Generalization With Adversarial Domain Alignment

    by Shiqian Zhao, Meiqing Wu, Kangjie Chen, Yi Xie, Tianlin Li, Siew-Kei Lam, Guowen Xu, Anran Li

    Recently, state-of-the-art stereo-matching networks trained on large-scale synthetic data have shown remarkable performance. However, their capacity to extrapolate effectively to unseen real-world data, i.e. different domains, remains a challenge. The major difficulty resides in the unforeseeable domain gap when generalizing from synthetic data to real-world data. In this paper, we introduce ADASM, an approach using adversarial domain alignment, designed to enhance the robustness and generalization of stereo-matching networks. It mainly consists of two modules: an end-to-end robustness optimizer and a domain-invariant feature learner. First, we adapt adversarial training into the stereo-matching task to reduce models' sensitivity to the perturbation in real-world samples. By introducing worst cases into the training space, we take unseen data into account and achieve robust disparity estimation for the end-to-end model. Then, via simulating the real-world noise with gradient-based perturbation, we construct a fictitious domain, which is taken as a referential distribution of the real-world noisy data, for further domain alignment. Specifically, we propose to utilize Maximum Mean Discrepancy to realize domain regularization between the original domain and the fictitious one. Finally, we fuse all aforementioned objectives and propose a unified, simple but effective loss function that can be adapted to all stereo-matching networks. The extensive experiments show that our method achieves a superior disparity estimation performance on various real-world benchmarks, including KITTI, Middlebury, and DrivingStereo. More importantly, ADASM obtains competitive or even better performance than the fine-tuning strategy, revealing its fine-tuning-free character.

  • Open Access

    Unveiling the Potential of Diffusion Large Language Models in Software Engineering Tasks: An Empirical Study

    by Jingyao Zhang, Tianlin Li, Xiaoyu Zhang, Qiang Hu, Bin Shi

    Autoregressive Large Language Models (AR-LLMs) are widely used in software engineering (SE) but face limitations in processing code structure information and suffer from high inference latency. Diffusion LLMs (DLLMs) offer a promising alternative with global bidirectional encoding and decoupled generation steps. This work presents the first comprehensive evaluation of DLLMs across the software development lifecycle, including code generation, defect detection, and program repair. On a large-scale benchmark of 52,937 tasks, 7B-parameter DLLMs outperform AR-LLMs with a 30% average accuracy improvement, achieving a 113% gain on cross-file repair, while maintaining superior efficiency and reduced latency. Our results establish DLLMs as a superior paradigm for SE tasks. The code is publicly available at https://anonymous.4open.science/r/all_datasets-B4A6

  • Open Access

    NeuSemSlice: Towards Effective DNN Model Maintenance via Neuron-Level Semantic Slicing

    by Shide Zhou, Tianlin Li, Yihao Huang, Ling Shi, Kailong Wang, Yang Liu, Haoyu Wang

    Deep Neural Networks (DNNs), extensively applied across diverse disciplines, are characterized by their integrated and monolithic architectures, setting them apart from conventional software systems. This architectural difference introduces particular challenges to maintenance tasks, such as model restructure (e.g., model compression), re-adaptation (e.g., fitting new samples), and incremental development (e.g., continual knowledge accumulation). Prior research addresses these challenges by identifying task-critical neuron layers and dividing neural networks into semantically similar sequential modules. However, such layer-level approaches fail to precisely identify and manipulate neuron-level semantic components, restricting their applicability to finer-grained model maintenance tasks. In this work, we implement NeuSemSlice, a novel framework that introduces the semantic slicing technique to effectively identify critical neuron-level semantic components in DNN models for semantic-aware model maintenance tasks. Specifically, semantic slicing identifies, categorizes, and merges critical neurons across different categories and layers according to their semantic similarity, enabling their flexibility and effectiveness in the subsequent tasks. For semantic-aware model maintenance tasks, we provide a series of novel strategies based on semantic slicing to enhance NeuSemSlice. They include semantic components (i.e., critical neurons) preservation for model restructure, critical neuron tuning for model re-adaptation, and non-critical neuron training for model incremental development. A thorough evaluation has demonstrated that NeuSemSlice significantly outperforms baselines in all three tasks.


  • Top Cited Papers
    Open Access

    JailGuard : A Universal Detection Framework for Prompt-based Attacks on LLM Systems

    by Xiaoyu Zhang, Cen Zhang, Tianlin Li, Yihao Huang, Xiaojun Jia, Ming Hu, Jie Zhang, Yang Liu, Shiqing Ma, Chao Shen

    The systems and software powered by Large Language Models (LLMs) and Multi-Modal Large Language Models (MLLMs) have played a critical role in numerous scenarios. However, current LLM systems are vulnerable to prompt-based attacks, with jailbreaking attacks enabling the LLM system to generate harmful content, while hijacking attacks manipulate the LLM system to perform attacker-desired tasks, underscoring the necessity for detection tools. Unfortunately, existing detecting approaches are usually tailored to specific attacks, resulting in poor generalization in detecting various attacks across different modalities. To address it, we propose JailGuard , a universal detection framework deployed on top of LLM systems for prompt-based attacks across text and image modalities. JailGuard operates on the principle that attacks are inherently less robust than benign ones. Specifically, JailGuard mutates untrusted inputs to generate variants and leverages the discrepancy of the variants’ responses on the target model to distinguish attack samples from benign samples. We implement 18 mutators for text and image inputs and design a mutator combination policy to further improve detection generalization. The evaluation on the dataset containing 15 known attack types suggests that JailGuard achieves the best detection accuracy of 86.14%/82.90% on text and image inputs, outperforming state-of-the-art methods by 11.81–25.73% and 12.20–21.40%.


  • FOOLSDEDIT: Deceptively Steering Your Edits Towards Targeted Attribute-Aware Distribution

    by Qi Zhou, Dongxia Wang, Tianlin Li, Zhihong Xu, Yang Liu, Kui Ren, Wenhai Wang, Qing Guo

    Guided image synthesis methods, like SDEdit based on the diffusion model, excel at creating realistic images from user inputs such as stroke paintings. However, existing efforts mainly focus on image quality, often overlooking a key point: the diffusion model represents a data distribution, not individual images. This introduces a low but critical chance of generating images that contradict user intentions, raising ethical concerns. For example, a user inputting a stroke painting with female characteristics might, with some probability, get male faces from SDEdit. To expose this potential vulnerability, we propose the Targeted Attribute Generative Attack (TAGA), whose objective is to force SDEdit to generate data distributions aligned with a specified attribute (i.e., targeted attribute like male), without changing the attribute of the input image. Empirical studies reveal that traditional adversarial noise struggles to achieve TAGA, while natural perturbations such as exposure and motion blur can easily influence attributes of the generated images. Inspired by the observation, we design attack method FOOLSDEDIT to achieve effective TAGA against SDEdit. It aims to search for an optimized strategy to execute attacks within a weighted graph-based attack architecture, which is formulated to model diverse strategies derived from both exposure and motion blur perturbations. Comprehensive experiments on two commonly used datasets and three social attributes present that FOOLSDEDIT forces SDEdit to generate targeted attribute-aware distributions, achieving significantly more effective TAGA than the baselines. We also validated empirically that FOOLSDEDIT could induce bias in downstream tasks of SDEdit which rely on the generated data under attack. Our work reveals critical vulnerabilities in diffusion-based image generation models and paves the way for future research on model auditing and bias mitigation.


  • An Analytical Perspective on Software Engineering for Large Language Models

    by Tianlin Li, Qiang Hu, Chong Wang, Jian Zhang, Wei Ma, Aishan Liu, Jingyi Wang, Yang Liu