Hao Sun
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Hao Sun has a total of 51 co-authors. Here are the top co-authors by number of shared publications:
- Zhongjiang He
China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd, Beijing, China
7 shared publications - Han Fang
China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd., Beijing, China
6 shared publications - Chao Ban
China Telecom Corporation Ltd. Data&AI Technology Company, Beijing, China
3 shared publications - Xianghao Zang
China Telecom Corporation Ltd. Data&AI Technology Company, Beijing, China
3 shared publications - Hao Li
SHANGHAI JIAOTONG UNIVERSITY, Shanghai, China
2 shared publications - Hongbo Sun
China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd., Beijing, China
2 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 2295-2304
Mixture-of-Thought-Tokens: Unifying Perception and Reasoning for Free-form Multimodal Grounding
Multimodal Large Language Models have made great progress in grounding tasks, yet existing methods still struggle to unify precise localization and complex reasoning. For one thing, text-based methods rely on coordinates or index prediction, severely limiting the perceptual capabilities of the model for dense visual objects. Meanwhile, latent token-based methods employ special tokens without inherent spatial references and use a decoding mechanism that lacks thinking steps, weakening high-level reasoning capabilities. Consequently, developing a unified framework that excels in both perception and reasoning remains challenging. To address this, we propose Mixture-of-Thought-Tokens (Motto), a new free-form multimodal grounding method that bridges the perception-reasoning gap, enabling MLLMs to empower diverse, arbitrary grounding queries. Specifically, we introduce Spatially-Grounded Thought Tokenization to explicitly align special tokens with spatial locations for clear spatial correspondence and visual interpretability. We further design a Context-Adaptive Chain-of-Tokens that dynamically switch grounding modes within an interleaved reasoning chain, achieving robust grounding across tasks of varying complexity. In addition, we construct PR-Bench, a new referring expression comprehension benchmark to evaluate the perception-reasoning gap. Extensive experiments demonstrate that Motto achieves state-of-the-art performance across diverse free-form grounding tasks.
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 2295-2304
Open AccessMixture-of-Thought-Tokens: Unifying Perception and Reasoning for Free-form Multimodal Grounding
Multimodal Large Language Models have made great progress in grounding tasks, yet existing methods still struggle to unify precise localization and complex reasoning. For one thing, text-based methods rely on coordinates or index prediction, severely limiting the perceptual capabilities of the model for dense visual objects. Meanwhile, latent token-based methods employ special tokens without inherent spatial references and use a decoding mechanism that lacks thinking steps, weakening high-level reasoning capabilities. Consequently, developing a unified framework that excels in both perception and reasoning remains challenging. To address this, we propose Mixture-of-Thought-Tokens (Motto), a new free-form multimodal grounding method that bridges the perception-reasoning gap, enabling MLLMs to empower diverse, arbitrary grounding queries. Specifically, we introduce Spatially-Grounded Thought Tokenization to explicitly align special tokens with spatial locations for clear spatial correspondence and visual interpretability. We further design a Context-Adaptive Chain-of-Tokens that dynamically switch grounding modes within an interleaved reasoning chain, achieving robust grounding across tasks of varying complexity. In addition, we construct PR-Bench, a new referring expression comprehension benchmark to evaluate the perception-reasoning gap. Extensive experiments demonstrate that Motto achieves state-of-the-art performance across diverse free-form grounding tasks.
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 4603-4612
GeoAnchor: Collaborative Reasoning via Latent Decomposition for 3D Spatial Understanding
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 4603-4612
Open AccessGeoAnchor: Collaborative Reasoning via Latent Decomposition for 3D Spatial Understanding
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 17829-17835
Forensic Prior Injection for Reliable Multimodal Deepfake Understanding
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 17829-17835
Open AccessForensic Prior Injection for Reliable Multimodal Deepfake Understanding
- Open Access
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 494-504
Unifying Granularity and Reliability: A Robust and Efficient Framework for Text-based Person Retrieval
- 19 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 494-504
Open AccessUnifying Granularity and Reliability: A Robust and Efficient Framework for Text-based Person Retrieval
- Open Access
- 15 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 2768-2772
VAV-R1: Difficulty-Aware Multimodal Reasoning for Video Anomaly Validation
- 15 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 2768-2772
Open AccessVAV-R1: Difficulty-Aware Multimodal Reasoning for Video Anomaly Validation
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 9608-9617
SVGen: Interpretable Vector Graphics Generation with Large Language Models
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 9608-9617
Open AccessSVGen: Interpretable Vector Graphics Generation with Large Language Models
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 11414-11423
Secure Tug-of-War (SecTOW): Iterative Defense-Attack Training with Reinforcement Learning for Multimodal Model Security
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 11414-11423
Open AccessSecure Tug-of-War (SecTOW): Iterative Defense-Attack Training with Reinforcement Learning for Multimodal Model Security
- 16 September 2025
- Published by Springer Nature
- p. 398-413
SelectVision: Adaptive Vision Resolution Selection for Visual Document Understanding
- 16 September 2025
- Published by Springer Nature
- p. 398-413
SelectVision: Adaptive Vision Resolution Selection for Visual Document Understanding
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 7055-7064
GOAL: Grounded text-to-image Synthesis with Joint Layout Alignment Tuning
Recent text-to-image (T2I) synthesis models have demonstrated intriguing abilities to produce high-quality images based on text prompts. However, current models still face Text-Image Misalignment problem (e.g., attribute errors and relation mistakes) for compositional generation. Existing models attempted to condition T2I models on grounding inputs to improve controllability while ignoring the explicit supervision from the layout conditions. To tackle this issue, we propose Grounded jOint lAyout aLignment (GOAL), an effective framework for T2I synthesis. Two novel modules, discriminative semantic alignment (DSAlign) and masked attention alignment (MAAlign), are proposed and incorporated in this framework to improve the text-image alignment. DSAlign leverages discriminative tasks at the region-wise level to ensure low-level semantic alignment. MAAlign provides high-level attention alignment by guiding the model to focus on the target object. We also build a dataset GOAL2K for model fine-tuning, which composes 2000 semantically accurate image-text pairs and their layout annotations. Comprehensive evaluations on T2I-Compbench, NSR-1K, and Drawbench demonstrate the superior generation performance of our method. Especially, there are improvements of 19%, 13%, and 12% in color, shape, and texture metrics for T2I-Compbench. Additionally, Q-Align metrics demonstrate that our method can generate images of higher quality.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 7055-7064
Open AccessGOAL: Grounded text-to-image Synthesis with Joint Layout Alignment Tuning
Recent text-to-image (T2I) synthesis models have demonstrated intriguing abilities to produce high-quality images based on text prompts. However, current models still face Text-Image Misalignment problem (e.g., attribute errors and relation mistakes) for compositional generation. Existing models attempted to condition T2I models on grounding inputs to improve controllability while ignoring the explicit supervision from the layout conditions. To tackle this issue, we propose Grounded jOint lAyout aLignment (GOAL), an effective framework for T2I synthesis. Two novel modules, discriminative semantic alignment (DSAlign) and masked attention alignment (MAAlign), are proposed and incorporated in this framework to improve the text-image alignment. DSAlign leverages discriminative tasks at the region-wise level to ensure low-level semantic alignment. MAAlign provides high-level attention alignment by guiding the model to focus on the target object. We also build a dataset GOAL2K for model fine-tuning, which composes 2000 semantically accurate image-text pairs and their layout annotations. Comprehensive evaluations on T2I-Compbench, NSR-1K, and Drawbench demonstrate the superior generation performance of our method. Especially, there are improvements of 19%, 13%, and 12% in color, shape, and texture metrics for T2I-Compbench. Additionally, Q-Align metrics demonstrate that our method can generate images of higher quality.
- Open Access
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 3847-3856
Mask to Reconstruct: Cooperative Semantics Completion for Video-text Retrieval
Recently, masked video modeling has been widely explored and improved the model's understanding ability of visual regions at a local level. However, existing methods usually adopt random masking and follow the same reconstruction paradigm to complete the masked regions, which do not leverage the correlations between cross-modal content. In this paper, we present MAsk for Semantics COmpleTion (MASCOT) based on semantic-based masked modeling. Specifically, after applying attention-based video masking to generate high-informed and low-informed masks, we propose Informed Semantics Completion to recover masked semantics information. The recovery mechanism is achieved by aligning the masked content with the unmasked visual regions and corresponding textual context, which makes the model capture more text-related details at a patch level. Additionally, we shift the emphasis of reconstruction from irrelevant backgrounds to discriminative parts to ignore regions with low-informed masks. Furthermore, we design co-learning to incorporate video cues under different masks and learn more aligned representation. Our MASCOT performs state-of-the-art performance on four text-video retrieval benchmarks, including MSR-VTT, LSMDC, ActivityNet, and DiDeMo.
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 3847-3856
Open AccessMask to Reconstruct: Cooperative Semantics Completion for Video-text Retrieval
Recently, masked video modeling has been widely explored and improved the model's understanding ability of visual regions at a local level. However, existing methods usually adopt random masking and follow the same reconstruction paradigm to complete the masked regions, which do not leverage the correlations between cross-modal content. In this paper, we present MAsk for Semantics COmpleTion (MASCOT) based on semantic-based masked modeling. Specifically, after applying attention-based video masking to generate high-informed and low-informed masks, we propose Informed Semantics Completion to recover masked semantics information. The recovery mechanism is achieved by aligning the masked content with the unmasked visual regions and corresponding textual context, which makes the model capture more text-related details at a patch level. Additionally, we shift the emphasis of reconstruction from irrelevant backgrounds to discriminative parts to ignore regions with low-informed masks. Furthermore, we design co-learning to incorporate video cues under different masks and learn more aligned representation. Our MASCOT performs state-of-the-art performance on four text-video retrieval benchmarks, including MSR-VTT, LSMDC, ActivityNet, and DiDeMo.