Xing Xie
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Xing Xie has a total of 206 co-authors. Here are the top co-authors by number of shared publications:
- Xiaoyuan Yi
Microsoft Research Asia
11 shared publications - Jing Yao
Microsoft Research Asia
8 shared publications Xiting Wang
Microsoft Research
8 shared publicationsXiaoyuan Yi
Microsoft Research Asia
7 shared publications- Fangzhao Wu
Microsoft Research Asia Beijing China
6 shared publications - Xiting Wang
Renmin University of China
5 shared publications
Latest Publications
- 26 October 2026
- Published by Springer Nature in Nature Human Behaviour
- p. 1-17
Large language models for computational social science
- 26 October 2026
- Published by Springer Nature in Nature Human Behaviour
- p. 1-17
Large language models for computational social science
- Open Access
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 13423-13424
The 1st International Workshop on AI Data Scientist
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 13423-13424
Open AccessThe 1st International Workshop on AI Data Scientist
- 23 June 2026
- Published by Springer Nature
- p. 64-77
ValueLex: Revealing the Value Structures of Large Language Models
- 23 June 2026
- Published by Springer Nature
- p. 64-77
ValueLex: Revealing the Value Structures of Large Language Models
- Top Cited PapersOpen Access
- 24 April 2026
- Published by Association for Computing Machinery (ACM) in Communications of the ACM
- Vol. 69 (5) , 92-102
Evaluating General-Purpose AI with Psychometrics
Rigorous evaluation of general-purpose AI systems such as large language models should allow for deepened understanding of their capabilities and effective mitigation of their risks. The current evaluation paradigm, mostly reliant on benchmarks aggregating scores on one or more tasks, lacks the scientific machinery for predicting performance on unforeseen tasks and explaining the variability of results. Moreover, existing benchmarks raise growing concerns about their reliability and validity. To tackle these challenges, we vindicate psychometrics, the science of psychological measurement, as a methodology for identifying and measuring constructs that underlie AI performance across multiple tasks. To raise awareness, we first identify the key advantages of adapting psychometric principles to AI evaluation through concrete examples; second, we distinguish sound applications of psychometric techniques from oversimplified ones and warn against common pitfalls; and third, to encourage general use, we introduce a systematic psychometric framework and an operational evaluation pipeline, which provide practical implementation guidance. In the end, we discuss underexplored avenues and societal implications that open new research directions for the use of psychometrics in broader AI research.
- 24 April 2026
- Published by Association for Computing Machinery (ACM) in Communications of the ACM
- Vol. 69 (5) , 92-102
Top Cited PapersOpen AccessEvaluating General-Purpose AI with Psychometrics
Rigorous evaluation of general-purpose AI systems such as large language models should allow for deepened understanding of their capabilities and effective mitigation of their risks. The current evaluation paradigm, mostly reliant on benchmarks aggregating scores on one or more tasks, lacks the scientific machinery for predicting performance on unforeseen tasks and explaining the variability of results. Moreover, existing benchmarks raise growing concerns about their reliability and validity. To tackle these challenges, we vindicate psychometrics, the science of psychological measurement, as a methodology for identifying and measuring constructs that underlie AI performance across multiple tasks. To raise awareness, we first identify the key advantages of adapting psychometric principles to AI evaluation through concrete examples; second, we distinguish sound applications of psychometric techniques from oversimplified ones and warn against common pitfalls; and third, to encourage general use, we introduce a systematic psychometric framework and an operational evaluation pipeline, which provide practical implementation guidance. In the end, we discuss underexplored avenues and societal implications that open new research directions for the use of psychometrics in broader AI research.
- Open Access
- 20 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 621-632
HumanLLM: Towards Personalized Understanding and Simulation of Human Nature
Motivated by the remarkable progress of large language models (LLMs) in objective tasks like mathematics and coding, there is growing interest in their potential to simulate human behavior—a capability with profound implications for transforming social science research and customer-centric business insights. However, LLMs often lack a nuanced understanding of human cognition and behavior, limiting their effectiveness in social simulation and personalized applications. We posit that this limitation stems from a fundamental misalignment: standard LLM pretraining on vast, uncontextualized web data does not capture the continuous, situated context of an individual's decisions, thoughts, and behaviors over time. To bridge this gap, we introduce HumanLLM, a foundation model designed for personalized understanding and simulation of individuals. We first construct the Cognitive Genome Dataset, a large-scale corpus curated from real-world user data on platforms like Reddit, Twitter, Blogger, and Amazon. Through a rigorous, multi-stage pipeline involving data filtering, synthesis, and quality control, we automatically extract over 5.5 million user logs to distill rich profiles, behaviors, and thinking patterns. We then formulate diverse learning tasks and perform supervised fine-tuning to empower the model to predict a wide range of individualized human behaviors, thoughts, and experiences. Comprehensive evaluations demonstrate that HumanLLM achieves superior performance in predicting user actions and inner thoughts, more accurately mimics user writing styles and preferences, and generates more authentic user profiles compared to base models. Furthermore, HumanLLM shows significant gains on out-of-domain social intelligence benchmarks, indicating enhanced generalization. This work paves the way for more human-centric AI systems by advancing research in social simulation, developing personalized companions, enabling marketing intelligence through simulated customer feedback, and powering more realistic user simulation for recommender systems.
- 20 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 621-632
Open AccessHumanLLM: Towards Personalized Understanding and Simulation of Human Nature
Motivated by the remarkable progress of large language models (LLMs) in objective tasks like mathematics and coding, there is growing interest in their potential to simulate human behavior—a capability with profound implications for transforming social science research and customer-centric business insights. However, LLMs often lack a nuanced understanding of human cognition and behavior, limiting their effectiveness in social simulation and personalized applications. We posit that this limitation stems from a fundamental misalignment: standard LLM pretraining on vast, uncontextualized web data does not capture the continuous, situated context of an individual's decisions, thoughts, and behaviors over time. To bridge this gap, we introduce HumanLLM, a foundation model designed for personalized understanding and simulation of individuals. We first construct the Cognitive Genome Dataset, a large-scale corpus curated from real-world user data on platforms like Reddit, Twitter, Blogger, and Amazon. Through a rigorous, multi-stage pipeline involving data filtering, synthesis, and quality control, we automatically extract over 5.5 million user logs to distill rich profiles, behaviors, and thinking patterns. We then formulate diverse learning tasks and perform supervised fine-tuning to empower the model to predict a wide range of individualized human behaviors, thoughts, and experiences. Comprehensive evaluations demonstrate that HumanLLM achieves superior performance in predicting user actions and inner thoughts, more accurately mimics user writing styles and preferences, and generates more authentic user profiles compared to base models. Furthermore, HumanLLM shows significant gains on out-of-domain social intelligence benchmarks, indicating enhanced generalization. This work paves the way for more human-centric AI systems by advancing research in social simulation, developing personalized companions, enabling marketing intelligence through simulated customer feedback, and powering more realistic user simulation for recommender systems.
- Top Cited PapersOpen Access
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-19
How Do Human Creators Embrace Human-AI Co-Creation? A Perspective on Human Agency of Screenwriters
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-19
Top Cited PapersOpen AccessHow Do Human Creators Embrace Human-AI Co-Creation? A Perspective on Human Agency of Screenwriters
- Open Access
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-20
DuoDrama: Supporting Screenplay Refinement Through LLM-Assisted Human Reflection
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-20
Open AccessDuoDrama: Supporting Screenplay Refinement Through LLM-Assisted Human Reflection
- Open Access
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-6
From Passive Consumption to Active Interaction: Exploring Interactive LLM Scaffolding to Support Learning Engagement
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Open AccessFrom Passive Consumption to Active Interaction: Exploring Interactive LLM Scaffolding to Support Learning Engagement
- Open Access
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-21
Interaction-Augmented Instruction: Modeling the Synergy of Prompts and Interactions in Human-GenAI Collaboration
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-21
Open AccessInteraction-Augmented Instruction: Modeling the Synergy of Prompts and Interactions in Human-GenAI Collaboration
- Top Cited PapersOpen Access
- 1 April 2026
- Published by Springer Nature in Nature
- Vol. 652 (8108) , 58-67
General scales unlock AI evaluation with explanatory and predictive power
Ensuring safe and effective use of artificial intelligence (AI) requires understanding and anticipating its performance on new tasks, from advanced scientific challenges to transformed workplace activities 1–3 . So far, benchmarking has guided progress in AI but has offered limited explanatory and predictive power for general-purpose AI systems 4–8 , attributed to limited transferability across specific tasks 9–11 . Here we introduce general scales for AI evaluation that elicit demand profiles explaining what capabilities common AI benchmarks truly measure, extract ability profiles quantifying the general strengths and limits of AI systems and robustly predict AI performance for new task instances. Our fully automated methodology builds on 18 rubrics, capturing a broad range of cognitive and intellectual demands, which place different task instances on the same general scales, illustrated on 15 large language models (LLMs) and 63 tasks. Both the demand and the ability profiles on these scales bring new insights such as construct validity through benchmark sensitivity and specificity and explain conflicting claims about whether AI has reasoning capabilities. Ultimately, high predictive power at the instance level becomes possible using the general scales, providing superior estimates over strong black-box baseline predictors, especially in out-of-distribution settings (new tasks and benchmarks). The scales, rubrics, battery, techniques and results presented here constitute a solid foundation for a science of AI evaluation, underpinning the reliable deployment of AI in the years ahead.
- 1 April 2026
- Published by Springer Nature in Nature
- Vol. 652 (8108) , 58-67
Top Cited PapersOpen AccessGeneral scales unlock AI evaluation with explanatory and predictive power
Ensuring safe and effective use of artificial intelligence (AI) requires understanding and anticipating its performance on new tasks, from advanced scientific challenges to transformed workplace activities 1–3 . So far, benchmarking has guided progress in AI but has offered limited explanatory and predictive power for general-purpose AI systems 4–8 , attributed to limited transferability across specific tasks 9–11 . Here we introduce general scales for AI evaluation that elicit demand profiles explaining what capabilities common AI benchmarks truly measure, extract ability profiles quantifying the general strengths and limits of AI systems and robustly predict AI performance for new task instances. Our fully automated methodology builds on 18 rubrics, capturing a broad range of cognitive and intellectual demands, which place different task instances on the same general scales, illustrated on 15 large language models (LLMs) and 63 tasks. Both the demand and the ability profiles on these scales bring new insights such as construct validity through benchmark sensitivity and specificity and explain conflicting claims about whether AI has reasoning capabilities. Ultimately, high predictive power at the instance level becomes possible using the general scales, providing superior estimates over strong black-box baseline predictors, especially in out-of-distribution settings (new tasks and benchmarks). The scales, rubrics, battery, techniques and results presented here constitute a solid foundation for a science of AI evaluation, underpinning the reliable deployment of AI in the years ahead.