Xiaohan Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Xiaohan Wang has a total of 45 co-authors. Here are the top co-authors by number of shared publications:
- Lin Zhang
Hangzhou International Innovation Institute of Beihang University, Hangzhou, China
7 shared publications - Yuanjun Laili
Beihang University
5 shared publications - Guojun Yin
Meituan, Beijing, China
3 shared publications - Jiajun Chai
Meituan, Beijing, China
3 shared publications - Zhen Chen
School of Automation Science and Electrical Engineering, Beihang University, Beijing, China
3 shared publications - Chengqi Dong
University of Science and Technology of China, Hefei, China and Meituan, Beijing, China
2 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1615-1624
IMAgent: Training Multi-Image Vision Agents via End2End Reinforcement Learning
Recent VLM-based agents aim to replicate OpenAI O3's "thinking with images" via tool use, yet most open-source methods lack tool-calling capability for multi-image inputs, limiting their applicability to real-world multi-image QA tasks. To address this gap, we propose IMAgent, an open-source visual agent trained with end-to-end reinforcement learning for fine-grained single/multi-image reasoning. During inference, VLMs tend to gradually neglect visual inputs; to mitigate this issue, we design two dedicated tools for visual reflection and confirmation, enabling the model to actively refocus attention on image content. Beyond that, we, for the first time, reveal how tool usage enhances agent performance from an attention perspective. Equipped with a carefully designed two-layer motion trajectory masking strategy and tool-use reward gain, IMAgent acquires an effective tool-use paradigm through pure reinforcement learning, eliminating the need for costly supervised fine-tuning data. To further unleash the inherent tool-usage potential of the base VLM and fill data gaps, we construct a challenging, visually enriched multi-image QA dataset via multi-agent system. Extensive experiments validate that IMAgent achieves SOTA performance across mainstream single and multi-image benchmarks, and our in-depth analysis offers actionable insights for the community. Code is available at: https://github.com/cq-dong/IMAgent.
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1615-1624
Open AccessIMAgent: Training Multi-Image Vision Agents via End2End Reinforcement Learning
Recent VLM-based agents aim to replicate OpenAI O3's "thinking with images" via tool use, yet most open-source methods lack tool-calling capability for multi-image inputs, limiting their applicability to real-world multi-image QA tasks. To address this gap, we propose IMAgent, an open-source visual agent trained with end-to-end reinforcement learning for fine-grained single/multi-image reasoning. During inference, VLMs tend to gradually neglect visual inputs; to mitigate this issue, we design two dedicated tools for visual reflection and confirmation, enabling the model to actively refocus attention on image content. Beyond that, we, for the first time, reveal how tool usage enhances agent performance from an attention perspective. Equipped with a carefully designed two-layer motion trajectory masking strategy and tool-use reward gain, IMAgent acquires an effective tool-use paradigm through pure reinforcement learning, eliminating the need for costly supervised fine-tuning data. To further unleash the inherent tool-usage potential of the base VLM and fill data gaps, we construct a challenging, visually enriched multi-image QA dataset via multi-agent system. Extensive experiments validate that IMAgent achieves SOTA performance across mainstream single and multi-image benchmarks, and our in-depth analysis offers actionable insights for the community. Code is available at: https://github.com/cq-dong/IMAgent.
- Open Access
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 3152-3163
CDRRM: Contrast-Driven Rubric Generation for Reliable and Interpretable Reward Modeling
Reward modeling is essential for aligning Large Language Models(LLMs) with human preferences, yet conventional reward models suffer from poor interpretability and heavy reliance on costly expert annotations. While recent rubric-based approaches enhance evaluation transparency, they lack systematic quality control, yielding noisy and redundant criteria, failing to mitigate persistent biases (e.g., verbosity, position) in LLM evaluators, and creating a scalability-reliability trade-off. To address these limitations, we propose CDRRM (Contrast-Driven Rubric Reward Model), a framework built on a novel Contrast-then-Synthesis paradigm for high-quality rubric generation and guided preference judgment. CDRRM first conducts multi-dimensional contrastive profiling on preference pairs to identify causal discriminative factors, then synthesizes these insights into compact, context-aware rubrics to guide preference judgments. Extensive experiments on three authoritative benchmarks (RewardBench, RMBench, RMB) demonstrate that CDRRM achieves state-of-the-art performance across diverse domains and effectively mitigates aforementioned evaluation biases. Notably, our approach delivers exceptional data efficiency: training the rubric generator on only 3k high-quality samples empowers a frozen pre-trained judge model to outperform fully fine-tuned baselines. This work offers a scalable, interpretable, and data-efficient path for reward modeling.
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 3152-3163
Open AccessCDRRM: Contrast-Driven Rubric Generation for Reliable and Interpretable Reward Modeling
Reward modeling is essential for aligning Large Language Models(LLMs) with human preferences, yet conventional reward models suffer from poor interpretability and heavy reliance on costly expert annotations. While recent rubric-based approaches enhance evaluation transparency, they lack systematic quality control, yielding noisy and redundant criteria, failing to mitigate persistent biases (e.g., verbosity, position) in LLM evaluators, and creating a scalability-reliability trade-off. To address these limitations, we propose CDRRM (Contrast-Driven Rubric Reward Model), a framework built on a novel Contrast-then-Synthesis paradigm for high-quality rubric generation and guided preference judgment. CDRRM first conducts multi-dimensional contrastive profiling on preference pairs to identify causal discriminative factors, then synthesizes these insights into compact, context-aware rubrics to guide preference judgments. Extensive experiments on three authoritative benchmarks (RewardBench, RMBench, RMB) demonstrate that CDRRM achieves state-of-the-art performance across diverse domains and effectively mitigates aforementioned evaluation biases. Notably, our approach delivers exceptional data efficiency: training the rubric generator on only 3k high-quality samples empowers a frozen pre-trained judge model to outperform fully fine-tuned baselines. This work offers a scalable, interpretable, and data-efficient path for reward modeling.
- Open Access
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 8989-9000
LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services
- 8 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 8989-9000
Open AccessLocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services
- 22 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 12 (19) , 41135-41147
Dynamic Scheduling of Simulation Workflows in Product Design With Meta-Reinforcement Learning
Product design involves the conceptualization and creation of products, where simulation workflows often require dynamic scheduling. Although deep reinforcement learning has shown competitive performance in addressing dynamic scheduling problems, they encounter challenges when dealing with multiple types of dynamic events in workflow scheduling. To this end, this article introduces a novel meta-reinforcement learning-based dynamic scheduling algorithm (MDSA) to tackle multiple dynamic events. Four types of dynamic events are considered and modeled, including random insertion, loop execution, structural adjustments, and resource uncertainties. The Markov decision process is formulated for the workflow scheduling problem, defining the state, action, reward, and state transition. To capture time-sequential information, a context-aware discretized policy is introduced, and a meta-training algorithm is employed to extract common knowledge across various scheduling scenarios. The effectiveness of the proposed method is demonstrated through a case study in the semiconductor display industry. The simulation environments are constructed using industry-inspired synthetic data, with reference to deployment practices from BOE and task patterns derived from Alibaba cluster traces, to ensure practical relevance while maintaining experimental control. Experimental results indicate that the approach demonstrates strong generalization capabilities, especially in terms of its consistent performance across varying task data distribution and its robustness in adapting to multiple dynamic conditions, thereby enhancing its adaptability in practical applications.
- 22 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 12 (19) , 41135-41147
Dynamic Scheduling of Simulation Workflows in Product Design With Meta-Reinforcement Learning
Product design involves the conceptualization and creation of products, where simulation workflows often require dynamic scheduling. Although deep reinforcement learning has shown competitive performance in addressing dynamic scheduling problems, they encounter challenges when dealing with multiple types of dynamic events in workflow scheduling. To this end, this article introduces a novel meta-reinforcement learning-based dynamic scheduling algorithm (MDSA) to tackle multiple dynamic events. Four types of dynamic events are considered and modeled, including random insertion, loop execution, structural adjustments, and resource uncertainties. The Markov decision process is formulated for the workflow scheduling problem, defining the state, action, reward, and state transition. To capture time-sequential information, a context-aware discretized policy is introduced, and a meta-training algorithm is employed to extract common knowledge across various scheduling scenarios. The effectiveness of the proposed method is demonstrated through a case study in the semiconductor display industry. The simulation environments are constructed using industry-inspired synthetic data, with reference to deployment practices from BOE and task patterns derived from Alibaba cluster traces, to ensure practical relevance while maintaining experimental control. Experimental results indicate that the approach demonstrates strong generalization capabilities, especially in terms of its consistent performance across varying task data distribution and its robustness in adapting to multiple dynamic conditions, thereby enhancing its adaptability in practical applications.
- 4 November 2024
- Published by Springer Nature in Journal of Intelligent Manufacturing
- Vol. 36 (8) , 5779-5800
Online simulation task scheduling in cloud manufacturing with cross attention and deep reinforcement learning
- 4 November 2024
- Published by Springer Nature in Journal of Intelligent Manufacturing
- Vol. 36 (8) , 5779-5800
Online simulation task scheduling in cloud manufacturing with cross attention and deep reinforcement learning
- 4 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 1878-1892
Hybrid Task Scheduling in Cloud Manufacturing With Sparse-Reward Deep Reinforcement Learning
Cloud manufacturing (CMfg) converts the traditional manufacturing system into an Internet-of-things-enabled (IoT-enabled) manufacturing system, where both manufacturing and computational tasks must be scheduled among distributed and heterogeneous resources. Deep reinforcement learning (DRL) has recently become a promising idea for task scheduling in CMfg. However, existing DRL-based methods depend heavily on problem-specific reward engineering and struggle to represent hybrid decision variables. To this end, this paper proposed the sparse-reward deep reinforcement learning (SDRL) method to solve the hybrid task scheduling problem in CMfg. First, the hybrid task scheduling model in CMfg is constructed to minimize the makespan. We reformulate the studied problem as a partially observable Markov decision process (POMDP). Then, the objective hindsight experience replay (objective HER) mechanism is proposed to alleviate the sparse reward issue, through which the scheduling policy can be effectively trained without problem-specific reward engineering. The continuous action space is defined to represent hybrid decision variables, and the implicit action-selection mapping is utilized to alleviate the boundary effect. Numerical experiments validated the effectiveness and superiority of our method compared to eleven popular scheduling algorithms including evolutionary algorithms and DRL. Compared to mainstream DRL scheduling methods, the proposed SDRL outperforms the second-best one at most by $23.6\%$ regarding generalization, and a scheduling solution can be generated in $0.5$ seconds. Note to Practitioners —With the intelligentization of the CMfg platform, hybrid tasks, including manufacturing and computational tasks, need to be scheduled simultaneously. However, this hybrid task scheduling problem is rarely considered by existing works. DRL exhibits many benefits in addressing scheduling problems, but the strong dependency on problem-specific reward engineering limits its application. Additionally, most DRL-based scheduling algorithms are discrete-action DRL, restricting their capacity to effectively represent hybrid decision variables. The studied problem originates from the CMfg platform, but the proposed method holds potential for broader application. The scheduling framework and the POMDP modeling can be applied to similar problems, including hybrid, manufacturing, or computational task scheduling problems. The proposed objective HER serves as a general approach to addressing challenges associated with sparse rewards, which can be extended to diverse combinatorial optimization problems aimed at optimizing an objective. We will open-source our codes to help others to apply the method to other fields.
- 4 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 1878-1892
Hybrid Task Scheduling in Cloud Manufacturing With Sparse-Reward Deep Reinforcement Learning
Cloud manufacturing (CMfg) converts the traditional manufacturing system into an Internet-of-things-enabled (IoT-enabled) manufacturing system, where both manufacturing and computational tasks must be scheduled among distributed and heterogeneous resources. Deep reinforcement learning (DRL) has recently become a promising idea for task scheduling in CMfg. However, existing DRL-based methods depend heavily on problem-specific reward engineering and struggle to represent hybrid decision variables. To this end, this paper proposed the sparse-reward deep reinforcement learning (SDRL) method to solve the hybrid task scheduling problem in CMfg. First, the hybrid task scheduling model in CMfg is constructed to minimize the makespan. We reformulate the studied problem as a partially observable Markov decision process (POMDP). Then, the objective hindsight experience replay (objective HER) mechanism is proposed to alleviate the sparse reward issue, through which the scheduling policy can be effectively trained without problem-specific reward engineering. The continuous action space is defined to represent hybrid decision variables, and the implicit action-selection mapping is utilized to alleviate the boundary effect. Numerical experiments validated the effectiveness and superiority of our method compared to eleven popular scheduling algorithms including evolutionary algorithms and DRL. Compared to mainstream DRL scheduling methods, the proposed SDRL outperforms the second-best one at most by $23.6\%$ regarding generalization, and a scheduling solution can be generated in $0.5$ seconds. Note to Practitioners —With the intelligentization of the CMfg platform, hybrid tasks, including manufacturing and computational tasks, need to be scheduled simultaneously. However, this hybrid task scheduling problem is rarely considered by existing works. DRL exhibits many benefits in addressing scheduling problems, but the strong dependency on problem-specific reward engineering limits its application. Additionally, most DRL-based scheduling algorithms are discrete-action DRL, restricting their capacity to effectively represent hybrid decision variables. The studied problem originates from the CMfg platform, but the proposed method holds potential for broader application. The scheduling framework and the POMDP modeling can be applied to similar problems, including hybrid, manufacturing, or computational task scheduling problems. The proposed objective HER serves as a general approach to addressing challenges associated with sparse rewards, which can be extended to diverse combinatorial optimization problems aimed at optimizing an objective. We will open-source our codes to help others to apply the method to other fields.
- 23 November 2023
- Published by Springer Nature in The International Journal of Advanced Manufacturing Technology
- Vol. 130 (1-2) , 203-221
Large-scale hybrid task scheduling in cloud-edge collaborative manufacturing systems with FCRN-assisted random differential evolution
- 23 November 2023
- Published by Springer Nature in The International Journal of Advanced Manufacturing Technology
- Vol. 130 (1-2) , 203-221
Large-scale hybrid task scheduling in cloud-edge collaborative manufacturing systems with FCRN-assisted random differential evolution
- 31 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 20 (2) , 1753-1763
DSAC-Configured Differential Evolution for Cloud–Edge–Device Collaborative Task Scheduling
Industrial Internet of Things enables various manufacturing processes executed in distributed production lines and flexible workshops. With different cloud–edge–device collaboration ways, interconnected manufacturing tasks and computational tasks are cooperatively completed in manufacturing cells, cloud resources, and edge resources. Large-scale decision variables and complex precedence constraints make the scheduling problem intractable. To this end, this article proposed a discretized soft actor–critic configured differential evolution algorithm to find a stable solution for the cloud–edge–device collaborative task-scheduling problem. A mathematical model is established to describe the relationship between different tasks, the variables, the main constraints in collaboration, and the scheduling targets. A decentralized partially observable Markov decision process is modeled with five neural networks and three discretized loss functions to formulate the discretized soft actor–critic policy efficiently and enable it to find the best differential evolution configurations for different scheduling cases. Experimental analysis of four cloud–edge–device scheduling instances indicates that the proposed method trained in one case is adaptable to the other three cases. In the four cases, the proposed method reduces the total objective by 30.82% and 44.35% at most compared to five deep-reinforcement-learning-based differential evolution algorithms and seven typical evolutionary algorithms, respectively.
- 31 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 20 (2) , 1753-1763
DSAC-Configured Differential Evolution for Cloud–Edge–Device Collaborative Task Scheduling
Industrial Internet of Things enables various manufacturing processes executed in distributed production lines and flexible workshops. With different cloud–edge–device collaboration ways, interconnected manufacturing tasks and computational tasks are cooperatively completed in manufacturing cells, cloud resources, and edge resources. Large-scale decision variables and complex precedence constraints make the scheduling problem intractable. To this end, this article proposed a discretized soft actor–critic configured differential evolution algorithm to find a stable solution for the cloud–edge–device collaborative task-scheduling problem. A mathematical model is established to describe the relationship between different tasks, the variables, the main constraints in collaboration, and the scheduling targets. A decentralized partially observable Markov decision process is modeled with five neural networks and three discretized loss functions to formulate the discretized soft actor–critic policy efficiently and enable it to find the best differential evolution configurations for different scheduling cases. Experimental analysis of four cloud–edge–device scheduling instances indicates that the proposed method trained in one case is adaptable to the other three cases. In the four cases, the proposed method reduces the total objective by 30.82% and 44.35% at most compared to five deep-reinforcement-learning-based differential evolution algorithms and seven typical evolutionary algorithms, respectively.
- 12 January 2023
- Published by Elsevier in Computer Methods and Programs in Biomedicine
- Vol. 230, 107348
MMViT-Seg: A lightweight transformer and CNN fusion network for COVID-19 segmentation
- 12 January 2023
- Published by Elsevier in Computer Methods and Programs in Biomedicine
- Vol. 230, 107348
MMViT-Seg: A lightweight transformer and CNN fusion network for COVID-19 segmentation
- 26 July 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6436-6441
Multi-Batch Production Scheduling for Distributed Serial Lines with Unreliable Machines and Finite Buffers
In the modern manufacturing environment, the production systems develop towards a flexible direction due to the market demands of multiple varieties and small batch-based customized products. To make better use of the existing resources and to improve production efficiency, distributed manufacturing mode begins to appear. This paper concentrates on a multi-type production scheduling problem for distributed serial lines. Specifically, unreliable machines and finite buffers are considered, which are more realistic in the practical production. The mathematical model for the production system with two Bernoulli machines is formulated first. Then, under the hypothetical model mentioned above, the main work of this article is to propose the exact and approximate analyses to calculate the performance indicators. Based on the analysis of the model, the genetic algorithm is applied to optimize the maximum completion time on the distributed serial lines for multi-batch-based production tasks. Finally, the feasibility and effectiveness of the mathematical model and the proposed algorithm are verified through numerical experiments.
- 26 July 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6436-6441
Multi-Batch Production Scheduling for Distributed Serial Lines with Unreliable Machines and Finite Buffers
In the modern manufacturing environment, the production systems develop towards a flexible direction due to the market demands of multiple varieties and small batch-based customized products. To make better use of the existing resources and to improve production efficiency, distributed manufacturing mode begins to appear. This paper concentrates on a multi-type production scheduling problem for distributed serial lines. Specifically, unreliable machines and finite buffers are considered, which are more realistic in the practical production. The mathematical model for the production system with two Bernoulli machines is formulated first. Then, under the hypothetical model mentioned above, the main work of this article is to propose the exact and approximate analyses to calculate the performance indicators. Based on the analysis of the model, the genetic algorithm is applied to optimize the maximum completion time on the distributed serial lines for multi-batch-based production tasks. Finally, the feasibility and effectiveness of the mathematical model and the proposed algorithm are verified through numerical experiments.