Xiaojie Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Xiaojie Wang has a total of 17 co-authors. Here are the top co-authors by number of shared publications:
- Fangxiang Feng
Beijing University of Posts and Telecommunications, Beijing, China
2 shared publications - Caixia Yuan
Beijing University of Posts and Telecommunications, Beijing, China
1 shared publication - Fangkun Zhao
Beijing University of Posts and Telecommunications, Beijing, China
1 shared publication - Guang Liu
Beijing Academy of Artificial Intelligence, Beijing, China
1 shared publication - Hanzi Wang
Beijing University of Posts and Telecommunications, Beijing, Beijing, CHINA
1 shared publication - Haochen Sun
Beijing University of Posts and Telecommunications, Beijing, China
1 shared publication
Latest Publications
- Open Access
- 12 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 546-553
RM2: Retrieve-More-Then-Read-More for Multi-Hop Question Answering
- 12 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 546-553
Open AccessRM2: Retrieve-More-Then-Read-More for Multi-Hop Question Answering
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 10592-10601
DiffHarmony++: Enhancing Image Harmonization with Harmony-VAE and Inverse Harmonization Model
Latent diffusion model has demonstrated impressive efficacy in image generation and editing tasks. Recently, it has also promoted the advancement of image harmonization. However, methods involving latent diffusion model all face a common challenge: the severe image distortion introduced by the VAE component, while image harmonization is a low-level image processing task that relies on pixel-level evaluation metrics. In this paper, we propose Harmony-VAE, leveraging the input of the harmonization task itself to enhance the quality of decoded images. The input involving composite image contains the precise pixel level information, which can complement the correct foreground appearance and color information contained in denoised latents. Meanwhile, the inherent generative nature of diffusion models makes it naturally adapt to inverse image harmonization, i.e. generating synthetic composite images based on real images and foreground masks. We train an inverse harmonization diffusion model to perform data augmentation on two subsets of iHarmony4 and construct a new human harmonization dataset with prominent foreground objects. Extensive experiments demonstrate the effectiveness of our proposed Harmony-VAE and inverse harmonization model. Code and pretrained models are available at https://github.com/nicecv/DiffHarmony.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 10592-10601
Open AccessDiffHarmony++: Enhancing Image Harmonization with Harmony-VAE and Inverse Harmonization Model
Latent diffusion model has demonstrated impressive efficacy in image generation and editing tasks. Recently, it has also promoted the advancement of image harmonization. However, methods involving latent diffusion model all face a common challenge: the severe image distortion introduced by the VAE component, while image harmonization is a low-level image processing task that relies on pixel-level evaluation metrics. In this paper, we propose Harmony-VAE, leveraging the input of the harmonization task itself to enhance the quality of decoded images. The input involving composite image contains the precise pixel level information, which can complement the correct foreground appearance and color information contained in denoised latents. Meanwhile, the inherent generative nature of diffusion models makes it naturally adapt to inverse image harmonization, i.e. generating synthetic composite images based on real images and foreground masks. We train an inverse harmonization diffusion model to perform data augmentation on two subsets of iHarmony4 and construct a new human harmonization dataset with prominent foreground objects. Extensive experiments demonstrate the effectiveness of our proposed Harmony-VAE and inverse harmonization model. Code and pretrained models are available at https://github.com/nicecv/DiffHarmony.
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 817-825
Q-MoE: Connector for MLLMs with Text-Driven Routing
Multimodal Large Language Models (MLLMs) have showcased remarkable advances in handling various vision-language tasks. These models typically consist of a Large Language Model (LLM), a vision encoder and a connector structure, which is used to bridge the modality gap between vision and language. It is challenging for the connector to filter the right visual information for LLM according to the task in hand. Most of previous connectors, such as light-weight projection and Q-former, treat visual information for diverse tasks uniformly, therefore lacking task-specific visual information extraction capabilities. To address the issue, this paper proposes Q-MoE, a query-based connector with Mixture-of-Experts (MoE) to extract task-specific information with text-driven routing. Furthermore, an optimal path based training strategy is proposed to find an optimal expert combination. Extensive experiments on two popular open-source LLMs and several different visual-language tasks demonstrate the effectiveness of the Q-MoE connecter.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 817-825
Open AccessQ-MoE: Connector for MLLMs with Text-Driven Routing
Multimodal Large Language Models (MLLMs) have showcased remarkable advances in handling various vision-language tasks. These models typically consist of a Large Language Model (LLM), a vision encoder and a connector structure, which is used to bridge the modality gap between vision and language. It is challenging for the connector to filter the right visual information for LLM according to the task in hand. Most of previous connectors, such as light-weight projection and Q-former, treat visual information for diverse tasks uniformly, therefore lacking task-specific visual information extraction capabilities. To address the issue, this paper proposes Q-MoE, a query-based connector with Mixture-of-Experts (MoE) to extract task-specific information with text-driven routing. Furthermore, an optimal path based training strategy is proposed to find an optimal expert combination. Extensive experiments on two popular open-source LLMs and several different visual-language tasks demonstrate the effectiveness of the Q-MoE connecter.