HHHaihong Hao

Haihong Hao

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Haihong Hao has a total of 16 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • CoNav: Collaborative Cross-Modal Reasoning for Embodied Navigation

    by Haihong Hao, Mingfei Han, Changlin Li, Zhihui Li, Xiaojun Chang

    Embodied navigation is a core task in embodied AI. It requires comprehensive scene understanding and precise spatial reasoning. Recent vision-language models (VLMs) with strong generalization capabilities and rich commonsense knowledge have shown remarkable performance when applied to embodied navigation tasks. However, these models still encounter insufficient understanding of 3D geometry and spatial semantics when applied to real-world 3D navigation. To address this, we propose CoNav, a collaborative cross-modal reasoning framework. First, we pretrain a 3D-language model with the curriculum learning schedule and prepare a pretrained vision-language navigation agent. Next, with lightweight fine-tuning on a small 2D-3D-text corpus, the vision-language navigation agent learns to combine visual evidence with knowledge from the 3D-language model. Finally, the pretrained 3D-language model communicates with the vision-language navigation agent, enabling collaborative cross-modal reasoning and resolving ambiguities during navigation. This yields more reliable and efficient image-3D fusion for embodied navigation. CoNav introduces a new collaborative framework between a 3D-language model and a vision-language navigation agent for embodied navigation. Notably, CoNav requires only a small 2D-3D-text corpus to align 3D and 2D data. CoNav achieves clear improvements on four standard embodied navigation benchmarks (R2R, CVDN, REVERIE, SOON) and two spatial reasoning benchmarks (ScanQA, SQA3D). Moreover, Under similar success rates, it also finds shorter paths than prior methods, as measured by SPL. The results demonstrate the value of collaborative 2D-3D reasoning for embodied navigation.


  • LatentPilot: Scene-Aware Vision-and-Language Navigation by Dreaming Ahead with Latent Visual Reasoning

    by Haihong Hao, Lei Chen, Mingfei Han, Changlin Li, Dong An, Yuqiang Yang, Zhihui Li, Xiaojun Chang


  • SELongVLM: Empowering Long Video Language Models With Self-Corrective Clip Selection

    by Kecheng Zhang, Zongxin Yang, Mingfei Han, Yunzhi Zhuge, Haihong Hao, Changlin Li, Zhihui Li, Xiaojun Chang

    Recent advances in multimodal large language models (MLLMs) have enabled impressive progress in visual-language reasoning, yet long-video understanding remains a formidable challenge due to the need for coherent reasoning over ultra-long spatiotemporal dependencies. Existing methods struggle with the vast candidate space for relevant information in long videos, often failing to distinguish meaningful events from redundant content. We identify two critical and previously under-explored issues: absolute redundancy, where static visual content inflates token counts without adding narrative value, and relative redundancy, where task-irrelevant segments introduce noise that impairs reasoning. Compounding these issues is the weak spatiotemporal modeling in current MLLMs, which limits their ability to capture complex event dynamics. To address these multifaceted challenges, we introduce SELongVLM, a dynamically lenient-to-stringent selection long video language model. SELongVLM integrates two coordinated branches: a Residual Token Pruner (RTP) that removes repetitive background tokens via inter-frame residual modeling thus mitigating absolute redundancy while preserving motion cues, and a Semantic-aware Self-Correction Selector (SCSelector) that progressively refines query-relevant clip selection without frame-level annotations to reduce relative redundancy, guided by a stringent-to-lenient self-correcting mechanism during optimization. To ensure causal continuity and bolster spatiotemporal reasoning across disjoint clips, the framework further incorporates an action-aware operation for intra-clip dynamics and a temporal memory for cross-clip context, enabling robust spatiotemporal inference on long videos. Extensive experiments across eight benchmarks demonstrate that SELongVLM markedly outperforms existing models on both general and specialized long-video tasks. Specifically, it achieves 65.5% on VideoMME and 69.8% on MLVU for general benchmarks, and delivers strong performance on four specialized benchmarks - for example, 39.2% on TOMATO for fine-grained temporal reasoning and 69.2% on EventBench for event-level understanding.


  • Open Access

    Hierarchical Space-Time Attention for Micro-Expression Recognition

    by Haihong Hao, Shuo Wang, Huixia Ben, Yanbin Hao, Yansong Wang, Weiwei Wang

    Micro-expression recognition (MER) aims to recognize the short and subtle facial movements from the Micro-expression (ME) video clips, which reveal real emotions. Recent MER methods mostly only utilize special frames from ME video clips or extract optical flow from these special frames. However, they neglect the relationship between movements and space-time, while facial cues are hidden within these relationships. To solve this issue, we propose the Hierarchical Space-Time Attention (HSTA). Specifically, we first process ME video frames and special frames or data parallelly by our cascaded Unimodal Space-Time Attention (USTA) to establish connections between subtle facial movements and specific facial areas. Then, we design Crossmodal Space-Time Attention (CSTA) to achieve a higher-quality fusion for crossmodal data. Finally, we hierarchically integrate USTA and CSTA to grasp the deeper facial cues. Our model emphasizes temporal modeling without neglecting the processing of special data, and it fuses the contents in different modalities while maintaining their respective uniqueness. Extensive experiments on the four benchmarks show the effectiveness of our proposed HSTA. Specifically, compared with the latest method on the CASME3 dataset, it achieves about 3% score improvement in seven-category classification.