Jian-Guang Lou
Jian-Guang Lou has a total of 31 co-authors. Here are the top co-authors by number of shared publications:
- Dongmei Zhang
Microsoft Research Asia, Beijing, China
3 shared publications - Haoyi Xiong
Baidu Inc., Beijing, China
2 shared publications - Jiaqi Guo
Microsoft Corporation, Beijing, China
2 shared publications - Ting Liu
Xi’an Jiaotong University
2 shared publications - Zhaoyun Jiang
Xi'an Jiaotong University, Xi'an, China
2 shared publications - Bin Benjamin Zhu
Microsoft Research Asia, Beijing, China
1 shared publication
Latest Publications
- Open Access
- 20 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 819-830
HyFunc: Accelerating LLM-based Function Calls for Agentic AI through Hybrid-Model Cascade and Dynamic Templating
- 20 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 819-830
Open AccessHyFunc: Accelerating LLM-based Function Calls for Agentic AI through Hybrid-Model Cascade and Dynamic Templating
- Top Cited PapersOpen Access
- 14 February 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
Evaluating LLM-based Agents for Multi-Turn Conversations: A Survey
This survey examines evaluation methods for large language model (LLM)-based agents in multi-turn conversational settings. Using a PRISMA-inspired framework, we systematically reviewed nearly 250 scholarly sources, capturing the state of the art from various venues of publication, and establishing a solid foundation for our analysis. Our study offers a structured approach by developing two interrelated taxonomy systems: one that defines what to evaluate and another that explains how to evaluate . The first taxonomy identifies key components of LLM-based agents for multi-turn conversations and their evaluation dimensions, including task completion, response quality, user experience, memory and context retention, as well as planning and tool integration. These components ensure that the performance of conversational agents is assessed in a holistic and meaningful manner. The second taxonomy system focuses on the evaluation methodologies. It categorizes approaches into annotation-based evaluations, automated metrics, hybrid strategies that combine human assessments with quantitative measures, and self-judging methods utilizing LLMs. This framework not only captures traditional metrics derived from language understanding, such as BLEU and ROUGE scores, but also incorporates advanced techniques that reflect the dynamic, interactive nature of multi-turn dialogues. Together, these frameworks summarize the current status quo, expose limitations in traditional practices, and provide a structured blueprint for improvement. Based on the summarization of existing studies, we identify several challenges and propose future directions, including the development of scalable, real-time evaluation pipelines, enhanced privacy-preserving mechanisms, and robust metrics that capture dynamic multi-turn interactions. Our contributions bridge historical insights with modern practices, paving the way for next-generation, reliably evaluated conversational AI systems and offering a comprehensive guide for researchers and practitioners.
- 14 February 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
Top Cited PapersOpen AccessEvaluating LLM-based Agents for Multi-Turn Conversations: A Survey
This survey examines evaluation methods for large language model (LLM)-based agents in multi-turn conversational settings. Using a PRISMA-inspired framework, we systematically reviewed nearly 250 scholarly sources, capturing the state of the art from various venues of publication, and establishing a solid foundation for our analysis. Our study offers a structured approach by developing two interrelated taxonomy systems: one that defines what to evaluate and another that explains how to evaluate . The first taxonomy identifies key components of LLM-based agents for multi-turn conversations and their evaluation dimensions, including task completion, response quality, user experience, memory and context retention, as well as planning and tool integration. These components ensure that the performance of conversational agents is assessed in a holistic and meaningful manner. The second taxonomy system focuses on the evaluation methodologies. It categorizes approaches into annotation-based evaluations, automated metrics, hybrid strategies that combine human assessments with quantitative measures, and self-judging methods utilizing LLMs. This framework not only captures traditional metrics derived from language understanding, such as BLEU and ROUGE scores, but also incorporates advanced techniques that reflect the dynamic, interactive nature of multi-turn dialogues. Together, these frameworks summarize the current status quo, expose limitations in traditional practices, and provide a structured blueprint for improvement. Based on the summarization of existing studies, we identify several challenges and propose future directions, including the development of scalable, real-time evaluation pipelines, enhanced privacy-preserving mechanisms, and robust metrics that capture dynamic multi-turn interactions. Our contributions bridge historical insights with modern practices, paving the way for next-generation, reliably evaluated conversational AI systems and offering a comprehensive guide for researchers and practitioners.
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 5365-5374
Illustration Layout Generation for Slide Enhancement with Pixel-based Diffusion Model
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 5365-5374
Open AccessIllustration Layout Generation for Slide Enhancement with Pixel-based Diffusion Model
- Top Cited PapersOpen Access
- 20 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 496-507
AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation
- 20 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 496-507
Top Cited PapersOpen AccessAgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 156-165
IconDM: Text-Guided Icon Set Expansion Using Diffusion Models
Icons are ubiquitous visual elements in graphic design, yet their creation is often complex and time-consuming. To resolve this problem, we draw inspiration from the booming text-to-image field and propose Text-Guided Icon Set Expansion, a novel task that helps users design high-quality icons using textual descriptions. Besides, users can control the style consistency of the created icons by inputting a few hand-crafted icons as style reference. Despite its practicality, the task poses two unique challenges. (i) Abstract Concept Visualization. Abstract concepts like technology and health are frequently encountered in icon creation, but their visualization is not straightforward and requires a grounding process that translates them into physical, easy-to-depict objects. (ii) Fine-grained Style Transfer. Unlike ordinary images, icons exhibit richer fine-grained stylistic elements, including tones, line widths, shapes, shadow effects, etc., which puts higher demands on capturing and preserving detailed styles during icon generation. To address the challenges, we propose IconDM, a method based on pre-trained text-to-image (T2I) diffusion models. Our approach incorporates a one-time domain adaptation process and an online style transfer process. In domain adaptation, we enhance the existing T2I model's capability to understand abstract concepts by fine-tuning it on high-quality icon-text pairs. To achieve this, we construct a large-scale dataset IconBank containing 2.3 million icon samples, and leverage a state-of-the-art vision-language model to generate textual descriptions for each icon. In style transfer, we introduce a Style Enhancement Module into the T2I model. It explicitly extracts the fine-grained style features from the given reference icons and is jointly optimized with the T2I model during DreamBooth tuning. To assess IconDM, we present IconBench, a structured evaluation suite with 30 icon sets and 100 concepts (including 50 abstract concepts). Quantitative results, qualitative analysis, and extensive ablation studies demonstrate the effectiveness of IconDM.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 156-165
Open AccessIconDM: Text-Guided Icon Set Expansion Using Diffusion Models
Icons are ubiquitous visual elements in graphic design, yet their creation is often complex and time-consuming. To resolve this problem, we draw inspiration from the booming text-to-image field and propose Text-Guided Icon Set Expansion, a novel task that helps users design high-quality icons using textual descriptions. Besides, users can control the style consistency of the created icons by inputting a few hand-crafted icons as style reference. Despite its practicality, the task poses two unique challenges. (i) Abstract Concept Visualization. Abstract concepts like technology and health are frequently encountered in icon creation, but their visualization is not straightforward and requires a grounding process that translates them into physical, easy-to-depict objects. (ii) Fine-grained Style Transfer. Unlike ordinary images, icons exhibit richer fine-grained stylistic elements, including tones, line widths, shapes, shadow effects, etc., which puts higher demands on capturing and preserving detailed styles during icon generation. To address the challenges, we propose IconDM, a method based on pre-trained text-to-image (T2I) diffusion models. Our approach incorporates a one-time domain adaptation process and an online style transfer process. In domain adaptation, we enhance the existing T2I model's capability to understand abstract concepts by fine-tuning it on high-quality icon-text pairs. To achieve this, we construct a large-scale dataset IconBank containing 2.3 million icon samples, and leverage a state-of-the-art vision-language model to generate textual descriptions for each icon. In style transfer, we introduce a Style Enhancement Module into the T2I model. It explicitly extracts the fine-grained style features from the given reference icons and is jointly optimized with the T2I model during DreamBooth tuning. To assess IconDM, we present IconBench, a structured evaluation suite with 30 icon sets and 100 concepts (including 50 abstract concepts). Quantitative results, qualitative analysis, and extensive ablation studies demonstrate the effectiveness of IconDM.
- Open Access
- 2 February 2024
- Published by Royal Society of Chemistry (RSC) in Digital Discovery
- Vol. 3 (3) , 482-490
Retro-BLEU: quantifying chemical plausibility of retrosynthesis routes through reaction template sequence analysis
Retro-BLEU is a statistical metric to evaluate the plausibility of model-generated retrosynthesis routes based on reaction template sequences analysis.
- 2 February 2024
- Published by Royal Society of Chemistry (RSC) in Digital Discovery
- Vol. 3 (3) , 482-490
Open AccessRetro-BLEU: quantifying chemical plausibility of retrosynthesis routes through reaction template sequence analysis
Retro-BLEU is a statistical metric to evaluate the plausibility of model-generated retrosynthesis routes based on reaction template sequences analysis.
- Open Access
- 21 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 276-285
Hadamard Adapter: An Extreme Parameter-Efficient Adapter Tuning Method for Pre-trained Language Models
Recent years, Pre-trained Language models (PLMs) have swept into various fields of artificial intelligence and achieved great success. However, most PLMs, such as T5 and GPT3, have a huge amount of parameters, fine-tuning them is often expensive and time consuming, and storing them takes up a lot of space. Therefore, it is necessary to adopt a parameter-efficient approach to reduce parameters of PLMs in fine-tuning without compromising their performance in downstream tasks. In this paper, we design a novel adapter which only acts on self-attention outputs in PLMs. This adapter adopts element-wise linear transformation using Hadamard product, hence named as Hadamard adapter, requires the fewest parameters compared to previous parameter-efficient adapters. In addition, we also summarize some tuning patterns for Hadamard adapter shared by various downstream tasks, expecting to provide some guidance for further parameter reduction with shared adapters in future studies. The experiments conducted on the widely-used GLUE benchmark with several SOTA PLMs prove that the Hadamard adapter achieves competitive performance with only 0.033% parameters compared with full fine-tuning, and it has the fewest parameters compared with other adapters. Moreover, we further find that there is also some redundant layers in the Hadamard adapter which can be removed to achieve more parameter efficiency with only 0.022% parameters.
- 21 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 276-285
Open AccessHadamard Adapter: An Extreme Parameter-Efficient Adapter Tuning Method for Pre-trained Language Models
Recent years, Pre-trained Language models (PLMs) have swept into various fields of artificial intelligence and achieved great success. However, most PLMs, such as T5 and GPT3, have a huge amount of parameters, fine-tuning them is often expensive and time consuming, and storing them takes up a lot of space. Therefore, it is necessary to adopt a parameter-efficient approach to reduce parameters of PLMs in fine-tuning without compromising their performance in downstream tasks. In this paper, we design a novel adapter which only acts on self-attention outputs in PLMs. This adapter adopts element-wise linear transformation using Hadamard product, hence named as Hadamard adapter, requires the fewest parameters compared to previous parameter-efficient adapters. In addition, we also summarize some tuning patterns for Hadamard adapter shared by various downstream tasks, expecting to provide some guidance for further parameter reduction with shared adapters in future studies. The experiments conducted on the widely-used GLUE benchmark with several SOTA PLMs prove that the Hadamard adapter achieves competitive performance with only 0.033% parameters compared with full fine-tuning, and it has the fewest parameters compared with other adapters. Moreover, we further find that there is also some redundant layers in the Hadamard adapter which can be removed to achieve more parameter efficiency with only 0.022% parameters.