Meng Liu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Meng Liu has a total of 145 co-authors. Here are the top co-authors by number of shared publications:
- Liqiang Nie
Harbin Institute of Technology (Shenzhen), Shenzhen, China
30 shared publications - Xuemeng Song
Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China
12 shared publications - Zan Gao
School of Computer Science and Engineering, Tianjin University of Technology, Tianjin, China and Shandong Artificial Intelligence Institute, Qilu University of Technology (Shandong Academy of Sciences), Jinan, Shandong Province, China
10 shared publications - Yupeng Hu
Shandong University, Jinan, Shandong, China
8 shared publications - Weili Guan
Harbin Institute of Technology (Shenzhen), Shenzhen, China
6 shared publications - Haoyu Zhang
Harbin Institute of Technology (Shenzhen), Shenzhen, China and Pengcheng Laboratory, Shenzhen, China
4 shared publications
Latest Publications
- Open Access
- 15 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 681-689
Egocentric Action Recognition with Retrieval-Augmented Learning
- 15 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 681-689
Open AccessEgocentric Action Recognition with Retrieval-Augmented Learning
- 11 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. PP (01628828) , 1-15
SpaceEra++: A Unified Framework Towards 3D Spatial Reasoning in Video
Visual-spatial understanding, defined as the ability to infer object relationships and scene layouts from visual inputs, is fundamental to downstream tasks such as robotic navigation and embodied interaction. However, pre-trained vision-language models (VLMs) remain constrained by spatial uncertainty stemming from inherently 2D observations and by the scarcity of data for 3D spatial understanding. To address these limitations, we proposed a novel framework, SpaceEra, in the NeurIPS 2025 Spotlight paper. Although it achieved significant performance gains, we further observed that its effectiveness is hindered by insufficient input from scanning videos and weak reasoning constraints. To tackle these newly emerged challenges, we extend the original framework into a comprehensive system, termed SpaceEra++, which spans data construction, model design, training optimization, and prompting inference. Specifically, to alleviate input insufficiency, we introduce ScenePick, a frame sampling strategy that balances spatial coverage with object semantics to produce compact yet comprehensive scene representations. In addition, to enhance spatial reasoning, we develop SpaceAlign, which enforces pairwise object constraints by jointly exploiting absolute coordinates and relative spatial relations, thereby aligning optimization with spatial accuracy. Extensive experiments across multiple benchmarks demonstrate consistent improvements over strong baselines, while ablation studies validate both the individual and joint contributions of each component, and further analyses provide guidance for future research.
- 11 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. PP (01628828) , 1-15
SpaceEra++: A Unified Framework Towards 3D Spatial Reasoning in Video
Visual-spatial understanding, defined as the ability to infer object relationships and scene layouts from visual inputs, is fundamental to downstream tasks such as robotic navigation and embodied interaction. However, pre-trained vision-language models (VLMs) remain constrained by spatial uncertainty stemming from inherently 2D observations and by the scarcity of data for 3D spatial understanding. To address these limitations, we proposed a novel framework, SpaceEra, in the NeurIPS 2025 Spotlight paper. Although it achieved significant performance gains, we further observed that its effectiveness is hindered by insufficient input from scanning videos and weak reasoning constraints. To tackle these newly emerged challenges, we extend the original framework into a comprehensive system, termed SpaceEra++, which spans data construction, model design, training optimization, and prompting inference. Specifically, to alleviate input insufficiency, we introduce ScenePick, a frame sampling strategy that balances spatial coverage with object semantics to produce compact yet comprehensive scene representations. In addition, to enhance spatial reasoning, we develop SpaceAlign, which enforces pairwise object constraints by jointly exploiting absolute coordinates and relative spatial relations, thereby aligning optimization with spatial accuracy. Extensive experiments across multiple benchmarks demonstrate consistent improvements over strong baselines, while ablation studies validate both the individual and joint contributions of each component, and further analyses provide guidance for future research.
- Open Access
- 11 May 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
- Vol. 44 (4) , 1-26
Uncovering Hidden Connections: Iterative Search and Reasoning for Video-grounded Dialog
Unlike conventional visual question answering, video-grounded dialog requires a deep understanding of both the dialog history and the video content to generate accurate responses. Although existing methods have achieved promising results, they still struggle with progressively comprehending complex dialog history and effectively integrating video information. To address these challenges, we propose an iterative search and reasoning framework composed of a textual encoder, a visual encoder, and a generator. Specifically, the textual encoder adopts a path search and aggregation strategy to identify key cues in the dialog history that are essential for understanding the current question. Meanwhile, the visual encoder employs an iterative reasoning network to extract and highlight critical visual evidence from the video, thereby enabling more comprehensive visual understanding. Finally, we use a pretrained GPT-2 model as the answer generator to transform the discovered latent cues into coherent and contextually appropriate responses. Extensive experiments on three public datasets demonstrate the effectiveness and generalizability of the proposed framework.
- 11 May 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Information Systems
- Vol. 44 (4) , 1-26
Open AccessUncovering Hidden Connections: Iterative Search and Reasoning for Video-grounded Dialog
Unlike conventional visual question answering, video-grounded dialog requires a deep understanding of both the dialog history and the video content to generate accurate responses. Although existing methods have achieved promising results, they still struggle with progressively comprehending complex dialog history and effectively integrating video information. To address these challenges, we propose an iterative search and reasoning framework composed of a textual encoder, a visual encoder, and a generator. Specifically, the textual encoder adopts a path search and aggregation strategy to identify key cues in the dialog history that are essential for understanding the current question. Meanwhile, the visual encoder employs an iterative reasoning network to extract and highlight critical visual evidence from the video, thereby enabling more comprehensive visual understanding. Finally, we use a pretrained GPT-2 model as the answer generator to transform the discovered latent cues into coherent and contextually appropriate responses. Extensive experiments on three public datasets demonstrate the effectiveness and generalizability of the proposed framework.
- 27 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 75 (00189456) , 1-15
Salient Object Detection via Cross-Model Interaction and Wavelet-Guided Edge Refinement
Salient Object Detection (SOD) aims to locate salient regions in complex visual scenes and segment the most salient objects, playing a crucial role in visual measurement systems. However, it has long faced challenges such as irregular topological structures of salient objects and inaccurate boundary prediction caused by cluttered background interference. Existing methods are limited by the inherent constraints of single-model architectures or suffer from inefficient cross-model fusion during encoding. To address these issues, this paper proposes a novel Cross-Model Interaction and Wavelet-guided Edge Refinement Network (CMIWERNet). Specifically, we design a Cross-Model Interaction Module (CMIM) that achieves deep feature collaboration and information exchange during the encoding phase, allowing heterogeneous features to effectively capture both local and global contexts. This enables our model to comprehend the complete topology of salient objects. Meanwhile, a Wavelet-guided Edge Refinement Module (WERM) is introduced to extract edge cues through spatial-frequency analysis and progressively refine the decoding process, thereby suppressing interference from cluttered backgrounds and achieving accurate edge prediction. Experimental evaluations indicate that our method (Ours-V) achieves eighteen best performances and six top-3 performances (excluding Ours-R) among all the twenty-four metrics in six datasets, surpassing existing solutions and offering a more excellent SOD solution for visual measurement applications. The source code will be available at: https://github.com/miaopingping/CMIWERNet.
- 27 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 75 (00189456) , 1-15
Salient Object Detection via Cross-Model Interaction and Wavelet-Guided Edge Refinement
Salient Object Detection (SOD) aims to locate salient regions in complex visual scenes and segment the most salient objects, playing a crucial role in visual measurement systems. However, it has long faced challenges such as irregular topological structures of salient objects and inaccurate boundary prediction caused by cluttered background interference. Existing methods are limited by the inherent constraints of single-model architectures or suffer from inefficient cross-model fusion during encoding. To address these issues, this paper proposes a novel Cross-Model Interaction and Wavelet-guided Edge Refinement Network (CMIWERNet). Specifically, we design a Cross-Model Interaction Module (CMIM) that achieves deep feature collaboration and information exchange during the encoding phase, allowing heterogeneous features to effectively capture both local and global contexts. This enables our model to comprehend the complete topology of salient objects. Meanwhile, a Wavelet-guided Edge Refinement Module (WERM) is introduced to extract edge cues through spatial-frequency analysis and progressively refine the decoding process, thereby suppressing interference from cluttered backgrounds and achieving accurate edge prediction. Experimental evaluations indicate that our method (Ours-V) achieves eighteen best performances and six top-3 performances (excluding Ours-R) among all the twenty-four metrics in six datasets, surpassing existing solutions and offering a more excellent SOD solution for visual measurement applications. The source code will be available at: https://github.com/miaopingping/CMIWERNet.
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28, 11806-11816
Mistake Detection in Egocentric Procedural Videos via CoT-based Action Anticipation
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28, 11806-11816
Mistake Detection in Egocentric Procedural Videos via CoT-based Action Anticipation
- 12 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 36 (5) , 5852-5864
Adaptive Attention-Based Unsupervised Domain Adaptation for Egocentric Action Recognition
In egocentric videos, collecting and annotating supervised data is more complicated and time-consuming than in exocentric videos, limiting research in this area. As a remedy, Unsupervised Domain Adaptation (UDA) enhances model performance on unlabeled target domains by bridging the distribution gap between source and target domains. However, UDA for egocentric action recognition is under-explored, facing unique challenges such as simultaneous learning of verb and noun representations, focusing on human-object interactions, and managing excessive verb-noun combinations. To tackle these issues, we propose a novel Unsupervised Domain Adaptation for Egocentric Action Recognition (UDA-EAR) approach that adaptively models egocentric actions and facilitates cross-domain knowledge transfer, improving recognition performance in unlabeled target domains. Specifically, our UDA-EAR employs adaptive spatio-temporal and spatio-channel attention in a dual-branch pipeline to focus on motion intervals and interaction regions, respectively, allowing specialized learning of discriminative representations while avoiding negative combination dependencies from domain gaps. Additionally, an adversarial domain alignment mechanism aligns the data distributions between source and target domains, effectively transferring fine-grained verb-noun knowledge of egocentric videos. Extensive experiments demonstrate that our UDA-EAR outperforms state-of-the-art baselines on widely used egocentric datasets, significantly improving egocentric action recognition accuracy. Our source codes and datasets are available at https://github.com/zou-y23/UDA-EAR.
- 12 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 36 (5) , 5852-5864
Adaptive Attention-Based Unsupervised Domain Adaptation for Egocentric Action Recognition
In egocentric videos, collecting and annotating supervised data is more complicated and time-consuming than in exocentric videos, limiting research in this area. As a remedy, Unsupervised Domain Adaptation (UDA) enhances model performance on unlabeled target domains by bridging the distribution gap between source and target domains. However, UDA for egocentric action recognition is under-explored, facing unique challenges such as simultaneous learning of verb and noun representations, focusing on human-object interactions, and managing excessive verb-noun combinations. To tackle these issues, we propose a novel Unsupervised Domain Adaptation for Egocentric Action Recognition (UDA-EAR) approach that adaptively models egocentric actions and facilitates cross-domain knowledge transfer, improving recognition performance in unlabeled target domains. Specifically, our UDA-EAR employs adaptive spatio-temporal and spatio-channel attention in a dual-branch pipeline to focus on motion intervals and interaction regions, respectively, allowing specialized learning of discriminative representations while avoiding negative combination dependencies from domain gaps. Additionally, an adversarial domain alignment mechanism aligns the data distributions between source and target domains, effectively transferring fine-grained verb-noun knowledge of egocentric videos. Extensive experiments demonstrate that our UDA-EAR outperforms state-of-the-art baselines on widely used egocentric datasets, significantly improving egocentric action recognition accuracy. Our source codes and datasets are available at https://github.com/zou-y23/UDA-EAR.
- Open Access
When Eyes and Ears Disagree: Can MLLMs Discern Audio-Visual Confusion?
Can Multimodal Large Language Models (MLLMs) discern confused objects that are visually present but audio-absent? To study this, we introduce a new benchmark, AV-ConfuseBench, which simulates an ``Audio-Visual Confusion'' scene by modifying the corresponding sound of an object in the video, e.g., mute the sounding object and ask MLLMs Is there a/an muted-object sound''. Experimental results reveal that MLLMs, such as Qwen2.5-Omni and Gemini 2.5, struggle to discriminate non-existent audio due to visually dominated reasoning. Motivated by this observation, we introduce RL-CoMM, a Reinforcement Learning-based Collaborative Multi-MLLM that is built upon the Qwen2.5-Omni foundation. RL-CoMM includes two stages: 1) To alleviate visually dominated ambiguities, we introduce an external model, a Large Audio Language Model (LALM), as the reference model to generate audio-only reasoning. Then, we design a Step-wise Reasoning Reward function that enables MLLMs to self-improve audio-visual reasoning with the audio-only reference. 2) To ensure an accurate answer prediction, we introduce Answer-centered Confidence Optimization to reduce the uncertainty of potential heterogeneous reasoning differences. Extensive experiments on audio-visual question answering and audio-visual hallucination show that RL-CoMM improves the accuracy by 10~30\% over the baseline model with limited training data. Follow: https://github.com/rikeilong/AVConfusion.
Open AccessWhen Eyes and Ears Disagree: Can MLLMs Discern Audio-Visual Confusion?
Can Multimodal Large Language Models (MLLMs) discern confused objects that are visually present but audio-absent? To study this, we introduce a new benchmark, AV-ConfuseBench, which simulates an ``Audio-Visual Confusion'' scene by modifying the corresponding sound of an object in the video, e.g., mute the sounding object and ask MLLMs Is there a/an muted-object sound''. Experimental results reveal that MLLMs, such as Qwen2.5-Omni and Gemini 2.5, struggle to discriminate non-existent audio due to visually dominated reasoning. Motivated by this observation, we introduce RL-CoMM, a Reinforcement Learning-based Collaborative Multi-MLLM that is built upon the Qwen2.5-Omni foundation. RL-CoMM includes two stages: 1) To alleviate visually dominated ambiguities, we introduce an external model, a Large Audio Language Model (LALM), as the reference model to generate audio-only reasoning. Then, we design a Step-wise Reasoning Reward function that enables MLLMs to self-improve audio-visual reasoning with the audio-only reference. 2) To ensure an accurate answer prediction, we introduce Answer-centered Confidence Optimization to reduce the uncertainty of potential heterogeneous reasoning differences. Extensive experiments on audio-visual question answering and audio-visual hallucination show that RL-CoMM improves the accuracy by 10~30\% over the baseline model with limited training data. Follow: https://github.com/rikeilong/AVConfusion.
- 3 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 36 (3) , 3874-3884
ViMAEdit: Vision-Guided and Mask-Enhanced Adaptive Editing Algorithm for Prompt-Based Image Editing
Text-to-image diffusion models have demonstrated remarkable progress in synthesizing high-quality images from text prompts, which boosts researches on prompt-based image editing that edits a source image according to a target prompt. Despite their advances, existing methods still encounter three key issues: 1) limited capacity of the text prompt in guiding target image generation, 2) insufficient mining of word-to-patch and patch-to-patch relationships for grounding editing areas, and 3) unified editing strength for all regions during each denoising step. To address these issues, we present a Vision-guided and Mask-enhanced Adaptive Editing (ViMAEdit) method with three key novel designs. First, we propose to leverage image embeddings as explicit guidance to enhance the conventional textual prompt-based denoising process, where a CLIP-based target image embedding estimation strategy is introduced. Second, we devise a self-attention-guided iterative editing area grounding strategy, which iteratively exploits patch-to-patch relationships conveyed by self-attention maps to refine those word-to-patch relationships contained in cross-attention maps. Last, we present a spatially adaptive variance-guided sampling, which highlights sampling variances for critical image regions to promote the editing capability. Experimental results demonstrate the superior editing capacity of ViMAEdit over all existing methods.
- 3 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 36 (3) , 3874-3884
ViMAEdit: Vision-Guided and Mask-Enhanced Adaptive Editing Algorithm for Prompt-Based Image Editing
Text-to-image diffusion models have demonstrated remarkable progress in synthesizing high-quality images from text prompts, which boosts researches on prompt-based image editing that edits a source image according to a target prompt. Despite their advances, existing methods still encounter three key issues: 1) limited capacity of the text prompt in guiding target image generation, 2) insufficient mining of word-to-patch and patch-to-patch relationships for grounding editing areas, and 3) unified editing strength for all regions during each denoising step. To address these issues, we present a Vision-guided and Mask-enhanced Adaptive Editing (ViMAEdit) method with three key novel designs. First, we propose to leverage image embeddings as explicit guidance to enhance the conventional textual prompt-based denoising process, where a CLIP-based target image embedding estimation strategy is introduced. Second, we devise a self-attention-guided iterative editing area grounding strategy, which iteratively exploits patch-to-patch relationships conveyed by self-attention maps to refine those word-to-patch relationships contained in cross-attention maps. Last, we present a spatially adaptive variance-guided sampling, which highlights sampling variances for critical image regions to promote the editing capability. Experimental results demonstrate the superior editing capacity of ViMAEdit over all existing methods.
- 29 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (2) , 1521-1541
A Survey on Video Temporal Grounding With Multimodal Large Language Model
The recent advancement in video temporal grounding (VTG) has significantly enhanced fine-grained video understanding, primarily driven by multimodal large language models (MLLMs). With superior multimodal comprehension and reasoning abilities, VTG approaches based on MLLMs (VTG-MLLMs) are gradually surpassing traditional fine-tuned methods. They not only achieve competitive performance but also excel in generalization across zero-shot, multi-task, and multi-domain settings. Despite extensive surveys on general video-language understanding, comprehensive reviews specifically addressing VTG-MLLMs remain scarce. To fill this gap, this survey systematically examines current research on VTG-MLLMs through a three-dimensional taxonomy: 1) the functional roles of MLLMs, highlighting their architectural significance; 2) training paradigms, analyzing strategies for temporal reasoning and task adaptation; and 3) video feature processing techniques, which determine spatiotemporal representation effectiveness. We further discuss benchmark datasets, evaluation protocols, and summarize empirical findings. Finally, we identify existing limitations and propose promising research directions.
- 29 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (2) , 1521-1541
A Survey on Video Temporal Grounding With Multimodal Large Language Model
The recent advancement in video temporal grounding (VTG) has significantly enhanced fine-grained video understanding, primarily driven by multimodal large language models (MLLMs). With superior multimodal comprehension and reasoning abilities, VTG approaches based on MLLMs (VTG-MLLMs) are gradually surpassing traditional fine-tuned methods. They not only achieve competitive performance but also excel in generalization across zero-shot, multi-task, and multi-domain settings. Despite extensive surveys on general video-language understanding, comprehensive reviews specifically addressing VTG-MLLMs remain scarce. To fill this gap, this survey systematically examines current research on VTG-MLLMs through a three-dimensional taxonomy: 1) the functional roles of MLLMs, highlighting their architectural significance; 2) training paradigms, analyzing strategies for temporal reasoning and task adaptation; and 3) video feature processing techniques, which determine spatiotemporal representation effectiveness. We further discuss benchmark datasets, evaluation protocols, and summarize empirical findings. Finally, we identify existing limitations and propose promising research directions.