Haoyu Liu
Research Subjects
Haoyu Liu has a total of 94 co-authors. Here are the top co-authors by number of shared publications:
- Jiming Chen
Zhejiang University, Hangzhou, China
10 shared publications - Changjie Fan
NetEase Fuxi AI Lab, Hangzhou, China
9 shared publications - Runze Wu
NetEase Fuxi AI Lab, Hangzhou, China
9 shared publications - Shibo He
Zhejiang University
8 shared publications - Tangjie Lv
NetEase Fuxi AI Lab, Hangzhou, China
8 shared publications - Zhipeng Hu
Fuxi AI Lab, NetEase Inc., Hangzhou, China
5 shared publications
Latest Publications
- Open Access
- 20 January 2026
- Published by Springer Nature in Journal of King Saud University Computer and Information Sciences
- Vol. 38 (2)
Online label aggregation with incomplete crowd responses
Crowdsourcing delivers responses that are asynchronous and incomplete, making offline aggregators that assume complete response sets impractical. Prior online methods often either require per-step completeness or repeatedly reload historical responses, which is storage- and privacy-unfriendly and susceptible to forgetting. We present OLA-Incomplete, an online label-aggregation framework designed for incomplete response streams. It integrates a variational-inference aggregator with a generative replay module that preserves historical information without reloading prior responses and explicitly models unknown worker reliability. At each update step, the generator replays cumulative responses and side information for previously observed instances to mitigate catastrophic forgetting, while the aggregator infers current truths by maximizing the evidence lower bound over a mixture of replayed and newly received labels. Across three public datasets—Duck, RTE, and PostSent—OLA-Incomplete attains final accuracies of 90.74%, 92.50%, and 95.99%, respectively, delivering at least 7.79% relative improvement over the strongest baseline. The approach further exhibits strong instantaneous online accuracy and robustness across response-chunk sizes and arrival orders, underscoring its practical utility for real-world crowdsourcing workflows.
- 20 January 2026
- Published by Springer Nature in Journal of King Saud University Computer and Information Sciences
- Vol. 38 (2)
Open AccessOnline label aggregation with incomplete crowd responses
Crowdsourcing delivers responses that are asynchronous and incomplete, making offline aggregators that assume complete response sets impractical. Prior online methods often either require per-step completeness or repeatedly reload historical responses, which is storage- and privacy-unfriendly and susceptible to forgetting. We present OLA-Incomplete, an online label-aggregation framework designed for incomplete response streams. It integrates a variational-inference aggregator with a generative replay module that preserves historical information without reloading prior responses and explicitly models unknown worker reliability. At each update step, the generator replays cumulative responses and side information for previously observed instances to mitigate catastrophic forgetting, while the aggregator infers current truths by maximizing the evidence lower bound over a mixture of replayed and newly received labels. Across three public datasets—Duck, RTE, and PostSent—OLA-Incomplete attains final accuracies of 90.74%, 92.50%, and 95.99%, respectively, delivering at least 7.79% relative improvement over the strongest baseline. The approach further exhibits strong instantaneous online accuracy and robustness across response-chunk sizes and arrival orders, underscoring its practical utility for real-world crowdsourcing workflows.
- Open Access
- 3 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 3366-3377
Empowering Economic Simulation for Massively Multiplayer Online Games through Generative Agent-Based Modeling
- 3 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 3366-3377
Open AccessEmpowering Economic Simulation for Massively Multiplayer Online Games through Generative Agent-Based Modeling
- Top Cited Papers
- 31 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 26 (5) , 5915-5930
Intention-Aware Denoising Diffusion Model for Trajectory Prediction
Trajectory prediction is an essential component in autonomous driving, particularly for collision avoidance systems. Considering the inherent uncertainty of the task, numerous studies have utilized generative models to produce multiple plausible future trajectories for each agent. However, most of them suffer from limited representation ability or unstable training issues. To overcome these limitations, we propose utilizing the diffusion model to generate the distribution of future trajectories. Two cruxes are to be settled to realize such an idea. First, the diversity of intention is intertwined with the uncertain surroundings, making the true distribution hard to parameterize. Second, the diffusion process is time-consuming during the inference phase, rendering it unrealistic to implement in a real-time driving system. We propose an Intention-aware denoising Diffusion Model (IDM), which addresses the above two problems. We decouple the original uncertainty into intention uncertainty and action uncertainty and model them with two dependent diffusion processes. To decrease the inference time, we reduce the variable dimensions in the intention-aware diffusion process and restrict the initial distribution of the action-aware diffusion process, which leads to fewer diffusion steps. To validate our approach, we conduct experiments on the Stanford Drone Dataset (SDD) and the ETH/UCY dataset. Our methods achieve state-of-the-art results, with a minFDE of 13.83 pixels on the SDD dataset and 0.36 meters on ETH/UCY datasets. Compared with the original diffusion model, IDM reduces inference time by two-thirds. Interestingly, our experiments further reveal that introducing intention information is beneficial in modeling the diffusion process of fewer steps.
- 31 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 26 (5) , 5915-5930
Top Cited PapersIntention-Aware Denoising Diffusion Model for Trajectory Prediction
Trajectory prediction is an essential component in autonomous driving, particularly for collision avoidance systems. Considering the inherent uncertainty of the task, numerous studies have utilized generative models to produce multiple plausible future trajectories for each agent. However, most of them suffer from limited representation ability or unstable training issues. To overcome these limitations, we propose utilizing the diffusion model to generate the distribution of future trajectories. Two cruxes are to be settled to realize such an idea. First, the diversity of intention is intertwined with the uncertain surroundings, making the true distribution hard to parameterize. Second, the diffusion process is time-consuming during the inference phase, rendering it unrealistic to implement in a real-time driving system. We propose an Intention-aware denoising Diffusion Model (IDM), which addresses the above two problems. We decouple the original uncertainty into intention uncertainty and action uncertainty and model them with two dependent diffusion processes. To decrease the inference time, we reduce the variable dimensions in the intention-aware diffusion process and restrict the initial distribution of the action-aware diffusion process, which leads to fewer diffusion steps. To validate our approach, we conduct experiments on the Stanford Drone Dataset (SDD) and the ETH/UCY dataset. Our methods achieve state-of-the-art results, with a minFDE of 13.83 pixels on the SDD dataset and 0.36 meters on ETH/UCY datasets. Compared with the original diffusion model, IDM reduces inference time by two-thirds. Interestingly, our experiments further reveal that introducing intention information is beneficial in modeling the diffusion process of fewer steps.
- Open Access
Digital Player: Evaluating Large Language Models based Human-like Agent in Games
With the rapid advancement of Large Language Models (LLMs), LLM-based autonomous agents have shown the potential to function as digital employees, such as digital analysts, teachers, and programmers. In this paper, we develop an application-level testbed based on the open-source strategy game "Unciv", which has millions of active players, to enable researchers to build a "data flywheel" for studying human-like agents in the "digital players" task. This "Civilization"-like game features expansive decision-making spaces along with rich linguistic interactions such as diplomatic negotiations and acts of deception, posing significant challenges for LLM-based agents in terms of numerical reasoning and long-term planning. Another challenge for "digital players" is to generate human-like responses for social interaction, collaboration, and negotiation with human players. The open-source project can be found at https:/github.com/fuxiAIlab/CivAgent.
Open AccessDigital Player: Evaluating Large Language Models based Human-like Agent in Games
With the rapid advancement of Large Language Models (LLMs), LLM-based autonomous agents have shown the potential to function as digital employees, such as digital analysts, teachers, and programmers. In this paper, we develop an application-level testbed based on the open-source strategy game "Unciv", which has millions of active players, to enable researchers to build a "data flywheel" for studying human-like agents in the "digital players" task. This "Civilization"-like game features expansive decision-making spaces along with rich linguistic interactions such as diplomatic negotiations and acts of deception, posing significant challenges for LLM-based agents in terms of numerical reasoning and long-term planning. Another challenge for "digital players" is to generate human-like responses for social interaction, collaboration, and negotiation with human players. The open-source project can be found at https:/github.com/fuxiAIlab/CivAgent.
- Open Access
Rank Aggregation in Crowdsourcing for Listwise Annotations
Rank aggregation through crowdsourcing has recently gained significant attention, particularly in the context of listwise ranking annotations. However, existing methods primarily focus on a single problem and partial ranks, while the aggregation of listwise full ranks across numerous problems remains largely unexplored. This scenario finds relevance in various applications, such as model quality assessment and reinforcement learning with human feedback. In light of practical needs, we propose LAC, a Listwise rank Aggregation method in Crowdsourcing, where the global position information is carefully measured and included. In our design, an especially proposed annotation quality indicator is employed to measure the discrepancy between the annotated rank and the true rank. We also take the difficulty of the ranking problem itself into consideration, as it directly impacts the performance of annotators and consequently influences the final results. To our knowledge, LAC is the first work to directly deal with the full rank aggregation problem in listwise crowdsourcing, and simultaneously infer the difficulty of problems, the ability of annotators, and the ground-truth ranks in an unsupervised way. To evaluate our method, we collect a real-world business-oriented dataset for paragraph ranking. Experimental results on both synthetic and real-world benchmark datasets demonstrate the effectiveness of our proposed LAC method.
Open AccessRank Aggregation in Crowdsourcing for Listwise Annotations
Rank aggregation through crowdsourcing has recently gained significant attention, particularly in the context of listwise ranking annotations. However, existing methods primarily focus on a single problem and partial ranks, while the aggregation of listwise full ranks across numerous problems remains largely unexplored. This scenario finds relevance in various applications, such as model quality assessment and reinforcement learning with human feedback. In light of practical needs, we propose LAC, a Listwise rank Aggregation method in Crowdsourcing, where the global position information is carefully measured and included. In our design, an especially proposed annotation quality indicator is employed to measure the discrepancy between the annotated rank and the true rank. We also take the difficulty of the ranking problem itself into consideration, as it directly impacts the performance of annotators and consequently influences the final results. To our knowledge, LAC is the first work to directly deal with the full rank aggregation problem in listwise crowdsourcing, and simultaneously infer the difficulty of problems, the ability of annotators, and the ground-truth ranks in an unsupervised way. To evaluate our method, we collect a real-world business-oriented dataset for paragraph ranking. Experimental results on both synthetic and real-world benchmark datasets demonstrate the effectiveness of our proposed LAC method.
- Open Access
- 24 August 2024
- Published by Association for Computing Machinery (ACM)
- p. 6073-6082
XRL-Bench: A Benchmark for Evaluating and Comparing Explainable Reinforcement Learning Techniques
Reinforcement Learning (RL) has demonstrated substantial potential across diverse fields, yet understanding its decision-making process, especially in real-world scenarios where rationality and safety are paramount, is an ongoing challenge. This paper delves in to Explainable RL (XRL), a subfield of Explainable AI (XAI) aimed at unravelling the complexities of RL models. Our focus rests on state-explaining techniques, a crucial subset within XRL methods, as they reveal the underlying factors influencing an agent's actions at any given time. Despite their significant role, the lack of a unified evaluation framework hinders assessment of their accuracy and effectiveness. To address this, we introduce XRL-Bench, a unified standardized benchmark tailored for the evaluation and comparison of XRL methods, encompassing three main modules: standard RL environments, explainers based on state importance, and standard evaluators. XRL-Bench supports both tabular and image data for state explanation. We also propose TabularSHAP, an innovative and competitive XRL method. We demonstrate the practical utility of TabularSHAP in real-world online gaming services and offer an open-source benchmark platform for the straightforward implementation and evaluation of XRL methods. Our contributions facilitate the continued progression of XRL technology.
- 24 August 2024
- Published by Association for Computing Machinery (ACM)
- p. 6073-6082
Open AccessXRL-Bench: A Benchmark for Evaluating and Comparing Explainable Reinforcement Learning Techniques
Reinforcement Learning (RL) has demonstrated substantial potential across diverse fields, yet understanding its decision-making process, especially in real-world scenarios where rationality and safety are paramount, is an ongoing challenge. This paper delves in to Explainable RL (XRL), a subfield of Explainable AI (XAI) aimed at unravelling the complexities of RL models. Our focus rests on state-explaining techniques, a crucial subset within XRL methods, as they reveal the underlying factors influencing an agent's actions at any given time. Despite their significant role, the lack of a unified evaluation framework hinders assessment of their accuracy and effectiveness. To address this, we introduce XRL-Bench, a unified standardized benchmark tailored for the evaluation and comparison of XRL methods, encompassing three main modules: standard RL environments, explainers based on state importance, and standard evaluators. XRL-Bench supports both tabular and image data for state explanation. We also propose TabularSHAP, an innovative and competitive XRL method. We demonstrate the practical utility of TabularSHAP in real-world online gaming services and offer an open-source benchmark platform for the straightforward implementation and evaluation of XRL methods. Our contributions facilitate the continued progression of XRL technology.
- Open Access
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 443-452
The MMO Economist: AI Empowers Robust, Healthy, and Sustainable P2W MMO Economies
Massively Multiplayer Online Games (MMOs) feature intricate virtual economies that permeate various in-game activities. However, the balancing act between profitability and equality in MMO economic design proves to be a persistent conundrum, especially in nascent business models like Pay-to-Win (P2W). Conventional efforts are curtailed by two primary constraints: the inability to verify and the provision of suboptimal solutions. In light of these predicaments, this paper delves into MMO economies and explores the promising potential of integrating emerging AI methodologies into economic design. Specifically, we introduce a novel hierarchical Reinforcement Learning (RL) solution for achieving Pareto optimality between profitability and equality in P2W economies. Leveraging our substantial industrial acumen and expertise, we establish an economic simulation environment that facilitates authentic and realistic assessments of MMO economic evolution. Building upon this foundation, we reconceptualize the P2W economic design process within the paradigm of a Markov Decision Process (MDP) and tackle it as a standard RL problem. Comprehensive evaluations corroborate that our solution demonstrates consistent personality specialization in economic simulations akin to real-world MMOs and significantly outperforms other baselines in economic design. Further discussions highlight its superiority in both frontier research and practical applications within the game industry.
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 443-452
Open AccessThe MMO Economist: AI Empowers Robust, Healthy, and Sustainable P2W MMO Economies
Massively Multiplayer Online Games (MMOs) feature intricate virtual economies that permeate various in-game activities. However, the balancing act between profitability and equality in MMO economic design proves to be a persistent conundrum, especially in nascent business models like Pay-to-Win (P2W). Conventional efforts are curtailed by two primary constraints: the inability to verify and the provision of suboptimal solutions. In light of these predicaments, this paper delves into MMO economies and explores the promising potential of integrating emerging AI methodologies into economic design. Specifically, we introduce a novel hierarchical Reinforcement Learning (RL) solution for achieving Pareto optimality between profitability and equality in P2W economies. Leveraging our substantial industrial acumen and expertise, we establish an economic simulation environment that facilitates authentic and realistic assessments of MMO economic evolution. Building upon this foundation, we reconceptualize the P2W economic design process within the paradigm of a Markov Decision Process (MDP) and tackle it as a standard RL problem. Comprehensive evaluations corroborate that our solution demonstrates consistent personality specialization in economic simulations akin to real-world MMOs and significantly outperforms other baselines in economic design. Further discussions highlight its superiority in both frontier research and practical applications within the game industry.
- Open Access
- 24 March 2024
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 38 (8) , 9098-9106
EnMatch: Matchmaking for Better Player Engagement via Neural Combinatorial Optimization
Matchmaking is a core task in e-sports and online games, as it contributes to player engagement and further influences the game's lifecycle. Previous methods focus on creating fair games at all times. They divide players into different tiers based on skill levels and only select players from the same tier for each game. Though this strategy can ensure fair matchmaking, it is not always good for player engagement. In this paper, we propose a novel Engagement-oriented Matchmaking (EnMatch) framework to ensure fair games and simultaneously enhance player engagement. Two main issues need to be addressed. First, it is unclear how to measure the impact of different team compositions and confrontations on player engagement during the game considering the variety of player characteristics. Second, such a detailed consideration on every single player during matchmaking will result in an NP-hard combinatorial optimization problem with non-linear objectives. In light of these challenges, we turn to real-world data analysis to reveal engagement-related factors. The resulting insights guide the development of engagement modeling, enabling the estimation of quantified engagement before a match is completed. To handle the combinatorial optimization problem, we formulate the problem into a reinforcement learning framework, in which a neural combinatorial optimization problem is built and solved. The performance of EnMatch is finally demonstrated through the comparison with other state-of-the-art methods based on several real-world datasets and online deployments on two games.
- 24 March 2024
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 38 (8) , 9098-9106
Open AccessEnMatch: Matchmaking for Better Player Engagement via Neural Combinatorial Optimization
Matchmaking is a core task in e-sports and online games, as it contributes to player engagement and further influences the game's lifecycle. Previous methods focus on creating fair games at all times. They divide players into different tiers based on skill levels and only select players from the same tier for each game. Though this strategy can ensure fair matchmaking, it is not always good for player engagement. In this paper, we propose a novel Engagement-oriented Matchmaking (EnMatch) framework to ensure fair games and simultaneously enhance player engagement. Two main issues need to be addressed. First, it is unclear how to measure the impact of different team compositions and confrontations on player engagement during the game considering the variety of player characteristics. Second, such a detailed consideration on every single player during matchmaking will result in an NP-hard combinatorial optimization problem with non-linear objectives. In light of these challenges, we turn to real-world data analysis to reveal engagement-related factors. The resulting insights guide the development of engagement modeling, enabling the estimation of quantified engagement before a match is completed. To handle the combinatorial optimization problem, we formulate the problem into a reinforcement learning framework, in which a neural combinatorial optimization problem is built and solved. The performance of EnMatch is finally demonstrated through the comparison with other state-of-the-art methods based on several real-world datasets and online deployments on two games.
- Open Access
Intention-aware Denoising Diffusion Model for Trajectory Prediction
Trajectory prediction is an essential component in autonomous driving, particularly for collision avoidance systems. Considering the inherent uncertainty of the task, numerous studies have utilized generative models to produce multiple plausible future trajectories for each agent. However, most of them suffer from restricted representation ability or unstable training issues. To overcome these limitations, we propose utilizing the diffusion model to generate the distribution of future trajectories. Two cruxes are to be settled to realize such an idea. First, the diversity of intention is intertwined with the uncertain surroundings, making the true distribution hard to parameterize. Second, the diffusion process is time-consuming during the inference phase, rendering it unrealistic to implement in a real-time driving system. We propose an Intention-aware denoising Diffusion Model (IDM), which tackles the above two problems. We decouple the original uncertainty into intention uncertainty and action uncertainty and model them with two dependent diffusion processes. To decrease the inference time, we reduce the variable dimensions in the intention-aware diffusion process and restrict the initial distribution of the action-aware diffusion process, which leads to fewer diffusion steps. To validate our approach, we conduct experiments on the Stanford Drone Dataset (SDD) and ETH/UCY dataset. Our methods achieve state-of-the-art results, with an FDE of 13.83 pixels on the SDD dataset and 0.36 meters on the ETH/UCY dataset. Compared with the original diffusion model, IDM reduces inference time by two-thirds. Interestingly, our experiments further reveal that introducing intention information is beneficial in modeling the diffusion process of fewer steps.
Open AccessIntention-aware Denoising Diffusion Model for Trajectory Prediction
Trajectory prediction is an essential component in autonomous driving, particularly for collision avoidance systems. Considering the inherent uncertainty of the task, numerous studies have utilized generative models to produce multiple plausible future trajectories for each agent. However, most of them suffer from restricted representation ability or unstable training issues. To overcome these limitations, we propose utilizing the diffusion model to generate the distribution of future trajectories. Two cruxes are to be settled to realize such an idea. First, the diversity of intention is intertwined with the uncertain surroundings, making the true distribution hard to parameterize. Second, the diffusion process is time-consuming during the inference phase, rendering it unrealistic to implement in a real-time driving system. We propose an Intention-aware denoising Diffusion Model (IDM), which tackles the above two problems. We decouple the original uncertainty into intention uncertainty and action uncertainty and model them with two dependent diffusion processes. To decrease the inference time, we reduce the variable dimensions in the intention-aware diffusion process and restrict the initial distribution of the action-aware diffusion process, which leads to fewer diffusion steps. To validate our approach, we conduct experiments on the Stanford Drone Dataset (SDD) and ETH/UCY dataset. Our methods achieve state-of-the-art results, with an FDE of 13.83 pixels on the SDD dataset and 0.36 meters on the ETH/UCY dataset. Compared with the original diffusion model, IDM reduces inference time by two-thirds. Interestingly, our experiments further reveal that introducing intention information is beneficial in modeling the diffusion process of fewer steps.
- Open Access
A Dataset for the Validation of Truth Inference Algorithms Suitable for Online Deployment
For the purpose of efficient and cost-effective large-scale data labeling, crowdsourcing is increasingly being utilized. To guarantee the quality of data labeling, multiple annotations need to be collected for each data sample, and truth inference algorithms have been developed to accurately infer the true labels. Despite previous studies having released public datasets to evaluate the efficacy of truth inference algorithms, these have typically focused on a single type of crowdsourcing task and neglected the temporal information associated with workers' annotation activities. These limitations significantly restrict the practical applicability of these algorithms, particularly in the context of long-term and online truth inference. In this paper, we introduce a substantial crowdsourcing annotation dataset collected from a real-world crowdsourcing platform. This dataset comprises approximately two thousand workers, one million tasks, and six million annotations. The data was gathered over a period of approximately six months from various types of tasks, and the timestamps of each annotation were preserved. We analyze the characteristics of the dataset from multiple perspectives and evaluate the effectiveness of several representative truth inference algorithms on this dataset. We anticipate that this dataset will stimulate future research on tracking workers' abilities over time in relation to different types of tasks, as well as enhancing online truth inference.
Open AccessA Dataset for the Validation of Truth Inference Algorithms Suitable for Online Deployment
For the purpose of efficient and cost-effective large-scale data labeling, crowdsourcing is increasingly being utilized. To guarantee the quality of data labeling, multiple annotations need to be collected for each data sample, and truth inference algorithms have been developed to accurately infer the true labels. Despite previous studies having released public datasets to evaluate the efficacy of truth inference algorithms, these have typically focused on a single type of crowdsourcing task and neglected the temporal information associated with workers' annotation activities. These limitations significantly restrict the practical applicability of these algorithms, particularly in the context of long-term and online truth inference. In this paper, we introduce a substantial crowdsourcing annotation dataset collected from a real-world crowdsourcing platform. This dataset comprises approximately two thousand workers, one million tasks, and six million annotations. The data was gathered over a period of approximately six months from various types of tasks, and the timestamps of each annotation were preserved. We analyze the characteristics of the dataset from multiple perspectives and evaluate the effectiveness of several representative truth inference algorithms on this dataset. We anticipate that this dataset will stimulate future research on tracking workers' abilities over time in relation to different types of tasks, as well as enhancing online truth inference.