Xiaoyuan Yi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Xiaoyuan Yi has a total of 39 co-authors. Here are the top co-authors by number of shared publications:
- Xing Xie
Microsoft Research Asia, Beijing, China
3 shared publications - Zhenghao Liu
Northeastern University, Shenyang, China
3 shared publications - Ge Yu
Northeastern University, Shenyang, China
2 shared publications - Jia Jia
Department of Computer Science and Technology, BNRist, Tsinghua University, Beijing, China
2 shared publications - Jianxun Lian
Microsoft Research Asia, Beijing, China
2 shared publications - Xing Xie
Microsoft Research Asia, Beijing, China
2 shared publications
Latest Publications
- 23 June 2026
- Published by Springer Nature
- p. 64-77
ValueLex: Revealing the Value Structures of Large Language Models
- 23 June 2026
- Published by Springer Nature
- p. 64-77
ValueLex: Revealing the Value Structures of Large Language Models
- Open Access
NAMeGEn: Creative Name Generation via A Novel Agent-based Multiple Personalized Goal Enhancement Framework
Trained on diverse human-authored texts, Large Language Models (LLMs) unlocked the potential for Creative Natural Language Generation (CNLG), benefiting various applications like advertising and storytelling. Nevertheless, CNLG still remains difficult due to two main challenges. (1) Multi-objective flexibility: user requirements are often personalized, fine-grained, and pluralistic, which LLMs struggle to satisfy simultaneously; (2) Interpretive complexity: beyond generation, creativity also involves understanding and interpreting implicit meaning to enhance users' perception. These challenges significantly limit current methods, especially in short-form text generation, in generating creative and insightful content. To address this, we focus on Chinese baby naming, a representative short-form CNLG task requiring adherence to explicit user constraints (e.g., length, semantics, anthroponymy) while offering meaningful aesthetic explanations. We propose NAMeGEn, a novel multi-agent optimization framework that iteratively alternates between objective extraction, name generation, and evaluation to meet diverse requirements and generate accurate explanations. To support this task, we further construct a classical Chinese poetry corpus with 17k+ poems to enhance aesthetics, and introduce CBNames, a new benchmark with tailored metrics. Extensive experiments demonstrate that NAMeGEn effectively generates creative names that meet diverse, personalized requirements while providing meaningful explanations, outperforming six baseline methods spanning various LLM backbones without any training.
Open AccessNAMeGEn: Creative Name Generation via A Novel Agent-based Multiple Personalized Goal Enhancement Framework
Trained on diverse human-authored texts, Large Language Models (LLMs) unlocked the potential for Creative Natural Language Generation (CNLG), benefiting various applications like advertising and storytelling. Nevertheless, CNLG still remains difficult due to two main challenges. (1) Multi-objective flexibility: user requirements are often personalized, fine-grained, and pluralistic, which LLMs struggle to satisfy simultaneously; (2) Interpretive complexity: beyond generation, creativity also involves understanding and interpreting implicit meaning to enhance users' perception. These challenges significantly limit current methods, especially in short-form text generation, in generating creative and insightful content. To address this, we focus on Chinese baby naming, a representative short-form CNLG task requiring adherence to explicit user constraints (e.g., length, semantics, anthroponymy) while offering meaningful aesthetic explanations. We propose NAMeGEn, a novel multi-agent optimization framework that iteratively alternates between objective extraction, name generation, and evaluation to meet diverse requirements and generate accurate explanations. To support this task, we further construct a classical Chinese poetry corpus with 17k+ poems to enhance aesthetics, and introduce CBNames, a new benchmark with tailored metrics. Extensive experiments demonstrate that NAMeGEn effectively generates creative names that meet diverse, personalized requirements while providing meaningful explanations, outperforming six baseline methods spanning various LLM backbones without any training.
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 12304-12313
Specify Privacy Yourself: Assessing Inference-Time Personalized Privacy Preservation Ability of Large Vision-Language Models
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 12304-12313
Open AccessSpecify Privacy Yourself: Assessing Inference-Time Personalized Privacy Preservation Ability of Large Vision-Language Models
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 4817-4826
Benchmarking Retrieval-Augmented Generation in Multi-Modal Contexts
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 4817-4826
Open AccessBenchmarking Retrieval-Augmented Generation in Multi-Modal Contexts
- Open Access
- 25 June 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
Neural Recommendation Reasoning with Logic Rules
Explainability is critical for recommender systems to ensure good user experience and facilitate designers to debug. However, generating explanations in recommender systems usually requires large efforts due to the dependency on additional data and case-by-case model design. One possible solution to these challenges is reasoning with logic rules, whose validity or confidence can automatically indicate high-quality explanations and formats are general. However, pioneer methods can be hardly applied in recommendation due to the high sparsity of interaction data, which raises the difficulty in accurately computing the rule validity, and the specific ranking-oriented task. To bridge this gap, we propose a general framework for Reco mmendation with lo gic r ule reasoning ( Recolor ) that satisfies three desirable properties. First, we explicitly estimate the rule validity to ensure well-grounded decisions, where a fuzzy logic validity module is designed for accurate estimation on highly sparse recommendation data. Second, we ensure the generality for both the types of input data and model architectures by designing a neural logic generation module, which decouples the user-item representation learning from the rule construction. Third, we integrate the two above-mentioned modules with a ranking-oriented BPR loss and achieve a unified optimization of explainability and accuracy. For any given neural recommendation model, our proposed logic rule reasoning framework can upgrade it to a self-explainable version. Numerical experiments and user studies on four public recommendation datasets with different levels of sparsity demonstrate that our framework shows high-validity rule explanations, generality in architecture and data, and high recommendation accuracy.
- 25 June 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
Open AccessNeural Recommendation Reasoning with Logic Rules
Explainability is critical for recommender systems to ensure good user experience and facilitate designers to debug. However, generating explanations in recommender systems usually requires large efforts due to the dependency on additional data and case-by-case model design. One possible solution to these challenges is reasoning with logic rules, whose validity or confidence can automatically indicate high-quality explanations and formats are general. However, pioneer methods can be hardly applied in recommendation due to the high sparsity of interaction data, which raises the difficulty in accurately computing the rule validity, and the specific ranking-oriented task. To bridge this gap, we propose a general framework for Reco mmendation with lo gic r ule reasoning ( Recolor ) that satisfies three desirable properties. First, we explicitly estimate the rule validity to ensure well-grounded decisions, where a fuzzy logic validity module is designed for accurate estimation on highly sparse recommendation data. Second, we ensure the generality for both the types of input data and model architectures by designing a neural logic generation module, which decouples the user-item representation learning from the rule construction. Third, we integrate the two above-mentioned modules with a ranking-oriented BPR loss and achieve a unified optimization of explainability and accuracy. For any given neural recommendation model, our proposed logic rule reasoning framework can upgrade it to a self-explainable version. Numerical experiments and user studies on four public recommendation datasets with different levels of sparsity demonstrate that our framework shows high-validity rule explanations, generality in architecture and data, and high recommendation accuracy.
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 3558-3567
Embedding an Ethical Mind: Aligning Text-to-Image Synthesis via Lightweight Value Optimization
Recent advancements in diffusion models trained on large-scale data have enabled the generation of indistinguishable human-level images, yet they often produce harmful content misaligned with human values, e.g., social bias, and offensive content. Despite extensive research on Large Language Models (LLMs), the challenge of Text-to-Image (T2I) model alignment remains largely unexplored. Addressing this problem, we propose LiVO (Lightweight Value Optimization), a novel lightweight method for aligning T2I models with human values. LiVO only optimizes a plug-and-play value encoder to integrate a specified value principle with the input prompt, allowing the control of generated images over both semantics and values. Specifically, we design a diffusion model-tailored preference optimization loss, which theoretically approximates the Bradley-Terry model used in LLM alignment but provides a more flexible trade-off between image quality and value conformity. To optimize the value encoder, we also develop a framework to automatically construct a text-image preference dataset of 86k (prompt, aligned image, violating image, value principle) samples. Without updating most model parameters and through adaptive value selection from the input prompt, LiVO significantly reduces harmful outputs and achieves faster convergence, surpassing several strong baselines and taking an initial step towards ethically aligned T2I models. Warning: This paper involves descriptions and images depicting discriminatory, pornographic, bloody, and horrific scenes.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 3558-3567
Open AccessEmbedding an Ethical Mind: Aligning Text-to-Image Synthesis via Lightweight Value Optimization
Recent advancements in diffusion models trained on large-scale data have enabled the generation of indistinguishable human-level images, yet they often produce harmful content misaligned with human values, e.g., social bias, and offensive content. Despite extensive research on Large Language Models (LLMs), the challenge of Text-to-Image (T2I) model alignment remains largely unexplored. Addressing this problem, we propose LiVO (Lightweight Value Optimization), a novel lightweight method for aligning T2I models with human values. LiVO only optimizes a plug-and-play value encoder to integrate a specified value principle with the input prompt, allowing the control of generated images over both semantics and values. Specifically, we design a diffusion model-tailored preference optimization loss, which theoretically approximates the Bradley-Terry model used in LLM alignment but provides a more flexible trade-off between image quality and value conformity. To optimize the value encoder, we also develop a framework to automatically construct a text-image preference dataset of 86k (prompt, aligned image, violating image, value principle) samples. Without updating most model parameters and through adaptive value selection from the input prompt, LiVO significantly reduces harmful outputs and achieves faster convergence, surpassing several strong baselines and taking an initial step towards ethically aligned T2I models. Warning: This paper involves descriptions and images depicting discriminatory, pornographic, bloody, and horrific scenes.
- 20 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Big Data
- Vol. 11 (2) , 426-437
Multi-Evidence Based Fact Verification via A Confidential Graph Neural Network
Fact verification tasks aim to identify the integrity of textual contents according to the truthful corpus. Existing fact verification models usually build a fully connected reasoning graph, which regards claim-evidence pairs as nodes and connects them with edges. They employ the graph to propagate the semantics of the nodes. Nevertheless, the noisy nodes usually propagate their semantics via the edges of the reasoning graph, which misleads the semantic representations of other nodes and amplifies the noise signals. To mitigate the propagation of noisy semantic information, we introduce a Confidential Graph Attention Network (CO-GAT), which proposes a node masking mechanism for modeling the nodes. Specifically, CO-GAT calculates the node confidence score by estimating the relevance between the claim and evidence pieces. Then, the node masking mechanism uses the node confidence scores to control the noise information flow from the vanilla node to the other graph nodes. CO-GAT achieves a 73.59% FEVER score on the FEVER dataset and shows the generalization ability by broadening the effectiveness to the science-specific domain.
- 20 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Big Data
- Vol. 11 (2) , 426-437
Multi-Evidence Based Fact Verification via A Confidential Graph Neural Network
Fact verification tasks aim to identify the integrity of textual contents according to the truthful corpus. Existing fact verification models usually build a fully connected reasoning graph, which regards claim-evidence pairs as nodes and connects them with edges. They employ the graph to propagate the semantics of the nodes. Nevertheless, the noisy nodes usually propagate their semantics via the edges of the reasoning graph, which misleads the semantic representations of other nodes and amplifies the noise signals. To mitigate the propagation of noisy semantic information, we introduce a Confidential Graph Attention Network (CO-GAT), which proposes a node masking mechanism for modeling the nodes. Specifically, CO-GAT calculates the node confidence score by estimating the relevance between the claim and evidence pieces. Then, the node masking mechanism uses the node confidence scores to control the noise information flow from the vanilla node to the other graph nodes. CO-GAT achieves a 73.59% FEVER score on the FEVER dataset and shows the generalization ability by broadening the effectiveness to the science-specific domain.
- Top Cited PapersOpen Access
- 29 March 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
- Vol. 15 (3) , 1-45
A Survey on Evaluation of Large Language Models
Large language models (LLMs) are gaining increasing popularity in both academia and industry, owing to their unprecedented performance in various applications. As LLMs continue to play a vital role in both research and daily use, their evaluation becomes increasingly critical, not only at the task level, but also at the society level for better understanding of their potential risks. Over the past years, significant efforts have been made to examine LLMs from various perspectives. This paper presents a comprehensive review of these evaluation methods for LLMs, focusing on three key dimensions: what to evaluate , where to evaluate , and how to evaluate . Firstly, we provide an overview from the perspective of evaluation tasks, encompassing general natural language processing tasks, reasoning, medical usage, ethics, education, natural and social sciences, agent applications, and other areas. Secondly, we answer the ‘where’ and ‘how’ questions by diving into the evaluation methods and benchmarks, which serve as crucial components in assessing the performance of LLMs. Then, we summarize the success and failure cases of LLMs in different tasks. Finally, we shed light on several future challenges that lie ahead in LLMs evaluation. Our aim is to offer invaluable insights to researchers in the realm of LLMs evaluation, thereby aiding the development of more proficient LLMs. Our key point is that evaluation should be treated as an essential discipline to better assist the development of LLMs. We consistently maintain the related open-source materials at: https://github.com/MLGroupJLU/LLM-eval-survey
- 29 March 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
- Vol. 15 (3) , 1-45
Top Cited PapersOpen AccessA Survey on Evaluation of Large Language Models
Large language models (LLMs) are gaining increasing popularity in both academia and industry, owing to their unprecedented performance in various applications. As LLMs continue to play a vital role in both research and daily use, their evaluation becomes increasingly critical, not only at the task level, but also at the society level for better understanding of their potential risks. Over the past years, significant efforts have been made to examine LLMs from various perspectives. This paper presents a comprehensive review of these evaluation methods for LLMs, focusing on three key dimensions: what to evaluate , where to evaluate , and how to evaluate . Firstly, we provide an overview from the perspective of evaluation tasks, encompassing general natural language processing tasks, reasoning, medical usage, ethics, education, natural and social sciences, agent applications, and other areas. Secondly, we answer the ‘where’ and ‘how’ questions by diving into the evaluation methods and benchmarks, which serve as crucial components in assessing the performance of LLMs. Then, we summarize the success and failure cases of LLMs in different tasks. Finally, we shed light on several future challenges that lie ahead in LLMs evaluation. Our aim is to offer invaluable insights to researchers in the realm of LLMs evaluation, thereby aiding the development of more proficient LLMs. Our key point is that evaluation should be treated as an essential discipline to better assist the development of LLMs. We consistently maintain the related open-source materials at: https://github.com/MLGroupJLU/LLM-eval-survey