Mengshi Qi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Mengshi Qi has a total of 64 co-authors. Here are the top co-authors by number of shared publications:
- Huadong Ma
Beijing University of Posts and Telecommunications
13 shared publications Huadong Ma
6 shared publications- Jiebo Luo
University of Rochester, Rochester, New York, USA
4 shared publications - Xianlin Zhang
School of Digital Media and Design Arts, Beijing University of Posts and Telecommunications, Beijing, China
4 shared publications - Yunhong Wang
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
4 shared publications - Annan Li
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
2 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 10469-10478
VTO: Visual Tool Orchestration for Video Anomaly Detection
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 10469-10478
Open AccessVTO: Visual Tool Orchestration for Video Anomaly Detection
- 25 September 2026
- Published by Springer Nature
- p. 616-633
CritiqueDriveVLM: From Verifier-Guided Reinforcement Learning to Latent Thought Distillation for Autonomous Driving
- 25 September 2026
- Published by Springer Nature
- p. 616-633
CritiqueDriveVLM: From Verifier-Guided Reinforcement Learning to Latent Thought Distillation for Autonomous Driving
- 11 June 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 35 (10577149) , 6371-6384
Explainable Action Form Assessment by Exploiting Multimodal Chain-of-Thoughts Reasoning
Evaluating whether human action is standard or not and providing reasonable feedback to improve action standardization is very crucial but challenging in real-world scenarios. However, current video understanding methods are mainly concerned with what and where the action is, which is unable to meet the requirements. Meanwhile, most of the existing datasets lack the labels indicating the degree of action standardization, and the action quality assessment datasets lack explainability and detailed feedback. Therefore, we define a new Human Action Form Assessment (AFA) task, and introduce a new diverse dataset CoT-AFA, which contains a large scale of fitness and martial arts videos with multi-level annotations for comprehensive video analysis. We enrich the CoT-AFA dataset with a novel Chain-of-Thought explanation paradigm. Instead of offering isolated feedback, our explanations provide a complete reasoning process—from identifying an action step to analyzing its outcome and proposing a concrete solution. Furthermore, we propose a framework named Explainable Fitness Assessor, which can not only judge an action but also explain why and provide a solution. This framework employs two parallel processing streams and a dynamic gating mechanism to fuse visual and semantic information, thereby boosting its analytical capabilities. The experimental results demonstrate that our method has achieved improvements in explanation generation (e.g., +16.0% in CIDEr) ,action classification (+2.7% in accuracy) and quality assessment (+2.1% in accuracy), revealing great potential of CoT-AFA for future studies. Our dataset and source code is available at https://github.com/MICLAB-BUPT/EFA.
- 11 June 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 35 (10577149) , 6371-6384
Explainable Action Form Assessment by Exploiting Multimodal Chain-of-Thoughts Reasoning
Evaluating whether human action is standard or not and providing reasonable feedback to improve action standardization is very crucial but challenging in real-world scenarios. However, current video understanding methods are mainly concerned with what and where the action is, which is unable to meet the requirements. Meanwhile, most of the existing datasets lack the labels indicating the degree of action standardization, and the action quality assessment datasets lack explainability and detailed feedback. Therefore, we define a new Human Action Form Assessment (AFA) task, and introduce a new diverse dataset CoT-AFA, which contains a large scale of fitness and martial arts videos with multi-level annotations for comprehensive video analysis. We enrich the CoT-AFA dataset with a novel Chain-of-Thought explanation paradigm. Instead of offering isolated feedback, our explanations provide a complete reasoning process—from identifying an action step to analyzing its outcome and proposing a concrete solution. Furthermore, we propose a framework named Explainable Fitness Assessor, which can not only judge an action but also explain why and provide a solution. This framework employs two parallel processing streams and a dynamic gating mechanism to fuse visual and semantic information, thereby boosting its analytical capabilities. The experimental results demonstrate that our method has achieved improvements in explanation generation (e.g., +16.0% in CIDEr) ,action classification (+2.7% in accuracy) and quality assessment (+2.1% in accuracy), revealing great potential of CoT-AFA for future studies. Our dataset and source code is available at https://github.com/MICLAB-BUPT/EFA.
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 35, 9028-9041
Chain-of-Evidence Multimodal Reasoning for Few-Shot Temporal Action Localization
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 35, 9028-9041
Chain-of-Evidence Multimodal Reasoning for Few-Shot Temporal Action Localization
- 22 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (4) , 4642-4656
DC-SAM: In-Context Segment Anything in Images and Videos via Dual Consistency
In-context segmentation, also known as one-shot segmentation, aims to segment objects based on a single labeled example. While the Segment Anything Model (SAM) excels in interactive segmentation, it is not inherently designed for in-context tasks. To bridge this gap, we propose a new Dual Consistency SAM (DC-SAM), a prompt-tuning framework that adapts SAM and SAM2 for both image and video in-context segmentation. Instead of relying solely on pre-trained backbones, DC-SAM enhances the prompt encoder by generating high-quality visual prompts through feature fusion. Furthermore, we introduce a novel cycle-consistent cross-attention mechanism to enforce alignment between fused features and visual prompts, complemented by a dual-branch design incorporating discriminative positive and negative prompts. Additionally, we extend DC-SAM to the video domain via a novel mask-tube training strategy. To facilitate research, we curate the first In-Context Video Object Segmentation (IC-VOS) benchmark. Extensive experiments demonstrate that DC-SAM achieves state-of-the-art performance, yielding 55.5 mIoU (+1.4) on COCO-20$^{i}$ , 73.0 (+1.1) mIoU on PASCAL-5$^{i}$ , and a$\mathcal {J} { \& amp;} \mathcal{F}$ score of 71.52 on IC-VOS.- 22 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (4) , 4642-4656
DC-SAM: In-Context Segment Anything in Images and Videos via Dual Consistency
In-context segmentation, also known as one-shot segmentation, aims to segment objects based on a single labeled example. While the Segment Anything Model (SAM) excels in interactive segmentation, it is not inherently designed for in-context tasks. To bridge this gap, we propose a new Dual Consistency SAM (DC-SAM), a prompt-tuning framework that adapts SAM and SAM2 for both image and video in-context segmentation. Instead of relying solely on pre-trained backbones, DC-SAM enhances the prompt encoder by generating high-quality visual prompts through feature fusion. Furthermore, we introduce a novel cycle-consistent cross-attention mechanism to enforce alignment between fused features and visual prompts, complemented by a dual-branch design incorporating discriminative positive and negative prompts. Additionally, we extend DC-SAM to the video domain via a novel mask-tube training strategy. To facilitate research, we curate the first In-Context Video Object Segmentation (IC-VOS) benchmark. Extensive experiments demonstrate that DC-SAM achieves state-of-the-art performance, yielding 55.5 mIoU (+1.4) on COCO-20$^{i}$ , 73.0 (+1.1) mIoU on PASCAL-5$^{i}$ , and a$\mathcal {J} { \& amp;} \mathcal{F}$ score of 71.52 on IC-VOS.- Open Access
Explainable Action Form Assessment by Exploiting Multimodal Chain-of-Thoughts Reasoning
Evaluating whether human action is standard or not and providing reasonable feedback to improve action standardization is very crucial but challenging in real-world scenarios. However, current video understanding methods are mainly concerned with what and where the action is, which is unable to meet the requirements. Meanwhile, most of the existing datasets lack the labels indicating the degree of action standardization, and the action quality assessment datasets lack explainability and detailed feedback. Therefore, we define a new Human Action Form Assessment (AFA) task, and introduce a new diverse dataset CoT-AFA, which contains a large scale of fitness and martial arts videos with multi-level annotations for comprehensive video analysis. We enrich the CoT-AFA dataset with a novel Chain-of-Thought explanation paradigm. Instead of offering isolated feedback, our explanations provide a complete reasoning process--from identifying an action step to analyzing its outcome and proposing a concrete solution. Furthermore, we propose a framework named Explainable Fitness Assessor, which can not only judge an action but also explain why and provide a solution. This framework employs two parallel processing streams and a dynamic gating mechanism to fuse visual and semantic information, thereby boosting its analytical capabilities. The experimental results demonstrate that our method has achieved improvements in explanation generation (e.g., +16.0% in CIDEr), action classification (+2.7% in accuracy) and quality assessment (+2.1% in accuracy), revealing great potential of CoT-AFA for future studies. Our dataset and source code is available at https://github.com/MICLAB-BUPT/EFA.
Open AccessExplainable Action Form Assessment by Exploiting Multimodal Chain-of-Thoughts Reasoning
Evaluating whether human action is standard or not and providing reasonable feedback to improve action standardization is very crucial but challenging in real-world scenarios. However, current video understanding methods are mainly concerned with what and where the action is, which is unable to meet the requirements. Meanwhile, most of the existing datasets lack the labels indicating the degree of action standardization, and the action quality assessment datasets lack explainability and detailed feedback. Therefore, we define a new Human Action Form Assessment (AFA) task, and introduce a new diverse dataset CoT-AFA, which contains a large scale of fitness and martial arts videos with multi-level annotations for comprehensive video analysis. We enrich the CoT-AFA dataset with a novel Chain-of-Thought explanation paradigm. Instead of offering isolated feedback, our explanations provide a complete reasoning process--from identifying an action step to analyzing its outcome and proposing a concrete solution. Furthermore, we propose a framework named Explainable Fitness Assessor, which can not only judge an action but also explain why and provide a solution. This framework employs two parallel processing streams and a dynamic gating mechanism to fuse visual and semantic information, thereby boosting its analytical capabilities. The experimental results demonstrate that our method has achieved improvements in explanation generation (e.g., +16.0% in CIDEr), action classification (+2.7% in accuracy) and quality assessment (+2.1% in accuracy), revealing great potential of CoT-AFA for future studies. Our dataset and source code is available at https://github.com/MICLAB-BUPT/EFA.
- Open Access
Synergistic Tensor and Pipeline Parallelism
In the machine learning system, the hybrid model parallelism combining tensor parallelism (TP) and pipeline parallelism (PP) has become the dominant solution for distributed training of Large Language Models~(LLMs) and Multimodal LLMs (MLLMs). However, TP introduces significant collective communication overheads, while PP suffers from synchronization inefficiencies such as pipeline bubbles. Existing works primarily address these challenges from isolated perspectives, focusing either on overlapping TP communication or on flexible PP scheduling to mitigate pipeline bubbles. In this paper, we propose a new synergistic tensor and pipeline parallelism schedule that simultaneously reduces both types of bubbles. Our proposed schedule decouples the forward and backward passes in PP into fine-grained computation units, which are then braided to form a composite computation sequence. This compositional structure enables near-complete elimination of TP-related bubbles. Building upon this structure, we further design the PP schedule to minimize PP bubbles. Experimental results demonstrate that our approach improves training throughput by up to 12% for LLMs and 16% for MLLMs compared to existing scheduling methods. Our source code is avaiable at https://github.com/MICLAB-BUPT/STP.
Open AccessSynergistic Tensor and Pipeline Parallelism
In the machine learning system, the hybrid model parallelism combining tensor parallelism (TP) and pipeline parallelism (PP) has become the dominant solution for distributed training of Large Language Models~(LLMs) and Multimodal LLMs (MLLMs). However, TP introduces significant collective communication overheads, while PP suffers from synchronization inefficiencies such as pipeline bubbles. Existing works primarily address these challenges from isolated perspectives, focusing either on overlapping TP communication or on flexible PP scheduling to mitigate pipeline bubbles. In this paper, we propose a new synergistic tensor and pipeline parallelism schedule that simultaneously reduces both types of bubbles. Our proposed schedule decouples the forward and backward passes in PP into fine-grained computation units, which are then braided to form a composite computation sequence. This compositional structure enables near-complete elimination of TP-related bubbles. Building upon this structure, we further design the PP schedule to minimize PP bubbles. Experimental results demonstrate that our approach improves training throughput by up to 12% for LLMs and 16% for MLLMs compared to existing scheduling methods. Our source code is avaiable at https://github.com/MICLAB-BUPT/STP.
- 31 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 25 (4) , 4876-4889
PITN: Physics-Informed Temporal Networks for Cuffless Blood Pressure Estimation
Estimating blood pressure (BP) plays a crucial role in healthcare. Traditionally, BP has been measured using a cuff, which is unsuitable for continuous BP monitoring. Recent advancements in cuffless wearable devices, such as graphene electronic tattoos and PPG sensors provide viable alternatives. However, existing BP estimation methods often overlook the importance of temporal modeling, despite the multi-periodicity and temporal dependencies inherent in cuffless BP signals. Moreover, continuous BP monitoring requires personalized modeling, which is further challenged by data scarcity. To address these two challenges, we introduce a novel Physics-Informed Temporal Network (PITN) with adversarial contrastive learning to enable precise BP estimation with very limited data for three different modalities (i.e, bioimpedance, PPG, millimeter wave). Specifically, we first introduce a novel Physics-Informed Temporal Network for investigating BP dynamics’ multi-periodicity for cardiovascular cycle modeling and temporal variation. We then apply adversarial training to generate extra physiological time series data, improving PITN’s robustness in the face of sparse subject-specific training data. Furthermore, we utilize contrastive learning to capture the discriminative variations of cardiovascular physiologic phenomena. This approach aggregates physiological signals with similar blood pressure values in latent space while separating clusters of samples with dissimilar blood pressure values. Experiments on three widely-adopted datasets with different modailties demonstrate the superiority and effectiveness of the proposed methods over previous state-of-the-art approaches.
- 31 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 25 (4) , 4876-4889
PITN: Physics-Informed Temporal Networks for Cuffless Blood Pressure Estimation
Estimating blood pressure (BP) plays a crucial role in healthcare. Traditionally, BP has been measured using a cuff, which is unsuitable for continuous BP monitoring. Recent advancements in cuffless wearable devices, such as graphene electronic tattoos and PPG sensors provide viable alternatives. However, existing BP estimation methods often overlook the importance of temporal modeling, despite the multi-periodicity and temporal dependencies inherent in cuffless BP signals. Moreover, continuous BP monitoring requires personalized modeling, which is further challenged by data scarcity. To address these two challenges, we introduce a novel Physics-Informed Temporal Network (PITN) with adversarial contrastive learning to enable precise BP estimation with very limited data for three different modalities (i.e, bioimpedance, PPG, millimeter wave). Specifically, we first introduce a novel Physics-Informed Temporal Network for investigating BP dynamics’ multi-periodicity for cardiovascular cycle modeling and temporal variation. We then apply adversarial training to generate extra physiological time series data, improving PITN’s robustness in the face of sparse subject-specific training data. Furthermore, we utilize contrastive learning to capture the discriminative variations of cardiovascular physiologic phenomena. This approach aggregates physiological signals with similar blood pressure values in latent space while separating clusters of samples with dissimilar blood pressure values. Experiments on three widely-adopted datasets with different modailties demonstrate the superiority and effectiveness of the proposed methods over previous state-of-the-art approaches.
- 30 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (3) , 2514-2527
Robust Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoning
In this paper, we propose a new Robust Disentangled Counterfactual Learning (RDCL) approach for physical audiovisual commonsense reasoning. The task aims to infer objects’ physical commonsense based on both video and audio input, with the main challenge being how to imitate the reasoning ability of humans, even in the scenario of missing modalities. Most of the current methods fail to take full advantage of different characteristics in multi-modal data, and the lack of causal reasoning ability in models impedes the progress of implicit physical knowledge inference. To address these issues, our proposed RDCL method decouples videos into static (time-invariant) and dynamic (time-varying) factors in the latent space using the disentangled sequential encoder, which adopts a variational autoencoder (VAE) to maximize the mutual information with a contrastive loss function. Furthermore, we introduce a counterfactual learning module to augment the model's reasoning ability by modeling physical knowledge relationships among different objects under counterfactual intervention. To alleviate the incomplete modality data issue, we introduce a robust multimodal learning method to recover the missing data by decomposing the shared features and model-specific features. Our proposed method is a plug-and-play module that can be incorporated into any baseline, including VLMs. In experiments, we show that our proposed method improves the reasoning accuracy and robustness of baseline methods and achieves the state-of-the-art performance.
- 30 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 48 (3) , 2514-2527
Robust Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoning
In this paper, we propose a new Robust Disentangled Counterfactual Learning (RDCL) approach for physical audiovisual commonsense reasoning. The task aims to infer objects’ physical commonsense based on both video and audio input, with the main challenge being how to imitate the reasoning ability of humans, even in the scenario of missing modalities. Most of the current methods fail to take full advantage of different characteristics in multi-modal data, and the lack of causal reasoning ability in models impedes the progress of implicit physical knowledge inference. To address these issues, our proposed RDCL method decouples videos into static (time-invariant) and dynamic (time-varying) factors in the latent space using the disentangled sequential encoder, which adopts a variational autoencoder (VAE) to maximize the mutual information with a contrastive loss function. Furthermore, we introduce a counterfactual learning module to augment the model's reasoning ability by modeling physical knowledge relationships among different objects under counterfactual intervention. To alleviate the incomplete modality data issue, we introduce a robust multimodal learning method to recover the missing data by decomposing the shared features and model-specific features. Our proposed method is a plug-and-play module that can be incorporated into any baseline, including VLMs. In experiments, we show that our proposed method improves the reasoning accuracy and robustness of baseline methods and achieves the state-of-the-art performance.
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 11170-11178
SafeDriveRAG: Towards Safe Autonomous Driving with Knowledge Graph-based Retrieval-Augmented Generation
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 11170-11178
Open AccessSafeDriveRAG: Towards Safe Autonomous Driving with Knowledge Graph-based Retrieval-Augmented Generation