MQMengshi Qi

Mengshi Qi

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Mengshi Qi has a total of 64 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    VTO: Visual Tool Orchestration for Video Anomaly Detection

    by Rui Wang, Yeteng Wu, Xianlin Zhang, Mengshi Qi

  • Explainable Action Form Assessment by Exploiting Multimodal Chain-of-Thoughts Reasoning

    by Mengshi Qi, Yeteng Wu, Wulian Yun, Xianlin Zhang, Huadong Ma

    Evaluating whether human action is standard or not and providing reasonable feedback to improve action standardization is very crucial but challenging in real-world scenarios. However, current video understanding methods are mainly concerned with what and where the action is, which is unable to meet the requirements. Meanwhile, most of the existing datasets lack the labels indicating the degree of action standardization, and the action quality assessment datasets lack explainability and detailed feedback. Therefore, we define a new Human Action Form Assessment (AFA) task, and introduce a new diverse dataset CoT-AFA, which contains a large scale of fitness and martial arts videos with multi-level annotations for comprehensive video analysis. We enrich the CoT-AFA dataset with a novel Chain-of-Thought explanation paradigm. Instead of offering isolated feedback, our explanations provide a complete reasoning process—from identifying an action step to analyzing its outcome and proposing a concrete solution. Furthermore, we propose a framework named Explainable Fitness Assessor, which can not only judge an action but also explain why and provide a solution. This framework employs two parallel processing streams and a dynamic gating mechanism to fuse visual and semantic information, thereby boosting its analytical capabilities. The experimental results demonstrate that our method has achieved improvements in explanation generation (e.g., +16.0% in CIDEr) ,action classification (+2.7% in accuracy) and quality assessment (+2.1% in accuracy), revealing great potential of CoT-AFA for future studies. Our dataset and source code is available at https://github.com/MICLAB-BUPT/EFA.


  • DC-SAM: In-Context Segment Anything in Images and Videos via Dual Consistency

    by Mengshi Qi, Pengfei Zhu, Xiangtai Li, Xiaoyang Bi, Lu Qi, Huadong Ma, Ming-Hsuan Yang

    In-context segmentation, also known as one-shot segmentation, aims to segment objects based on a single labeled example. While the Segment Anything Model (SAM) excels in interactive segmentation, it is not inherently designed for in-context tasks. To bridge this gap, we propose a new Dual Consistency SAM (DC-SAM), a prompt-tuning framework that adapts SAM and SAM2 for both image and video in-context segmentation. Instead of relying solely on pre-trained backbones, DC-SAM enhances the prompt encoder by generating high-quality visual prompts through feature fusion. Furthermore, we introduce a novel cycle-consistent cross-attention mechanism to enforce alignment between fused features and visual prompts, complemented by a dual-branch design incorporating discriminative positive and negative prompts. Additionally, we extend DC-SAM to the video domain via a novel mask-tube training strategy. To facilitate research, we curate the first In-Context Video Object Segmentation (IC-VOS) benchmark. Extensive experiments demonstrate that DC-SAM achieves state-of-the-art performance, yielding 55.5 mIoU (+1.4) on COCO-20$^{i}$, 73.0 (+1.1) mIoU on PASCAL-5$^{i}$, and a $\mathcal {J} { \& amp;} \mathcal{F}$ score of 71.52 on IC-VOS.


  • Open Access

    Explainable Action Form Assessment by Exploiting Multimodal Chain-of-Thoughts Reasoning

    by Mengshi Qi, Yeteng Wu, Xianlin Zhang, Huadong Ma

    Evaluating whether human action is standard or not and providing reasonable feedback to improve action standardization is very crucial but challenging in real-world scenarios. However, current video understanding methods are mainly concerned with what and where the action is, which is unable to meet the requirements. Meanwhile, most of the existing datasets lack the labels indicating the degree of action standardization, and the action quality assessment datasets lack explainability and detailed feedback. Therefore, we define a new Human Action Form Assessment (AFA) task, and introduce a new diverse dataset CoT-AFA, which contains a large scale of fitness and martial arts videos with multi-level annotations for comprehensive video analysis. We enrich the CoT-AFA dataset with a novel Chain-of-Thought explanation paradigm. Instead of offering isolated feedback, our explanations provide a complete reasoning process--from identifying an action step to analyzing its outcome and proposing a concrete solution. Furthermore, we propose a framework named Explainable Fitness Assessor, which can not only judge an action but also explain why and provide a solution. This framework employs two parallel processing streams and a dynamic gating mechanism to fuse visual and semantic information, thereby boosting its analytical capabilities. The experimental results demonstrate that our method has achieved improvements in explanation generation (e.g., +16.0% in CIDEr), action classification (+2.7% in accuracy) and quality assessment (+2.1% in accuracy), revealing great potential of CoT-AFA for future studies. Our dataset and source code is available at https://github.com/MICLAB-BUPT/EFA.


  • Open Access

    Synergistic Tensor and Pipeline Parallelism

    by Mengshi Qi, Jiaxuan Peng, Jie Zhang, Juan Zhu, Yong Li, Huadong Ma

    In the machine learning system, the hybrid model parallelism combining tensor parallelism (TP) and pipeline parallelism (PP) has become the dominant solution for distributed training of Large Language Models~(LLMs) and Multimodal LLMs (MLLMs). However, TP introduces significant collective communication overheads, while PP suffers from synchronization inefficiencies such as pipeline bubbles. Existing works primarily address these challenges from isolated perspectives, focusing either on overlapping TP communication or on flexible PP scheduling to mitigate pipeline bubbles. In this paper, we propose a new synergistic tensor and pipeline parallelism schedule that simultaneously reduces both types of bubbles. Our proposed schedule decouples the forward and backward passes in PP into fine-grained computation units, which are then braided to form a composite computation sequence. This compositional structure enables near-complete elimination of TP-related bubbles. Building upon this structure, we further design the PP schedule to minimize PP bubbles. Experimental results demonstrate that our approach improves training throughput by up to 12% for LLMs and 16% for MLLMs compared to existing scheduling methods. Our source code is avaiable at https://github.com/MICLAB-BUPT/STP.

  • PITN: Physics-Informed Temporal Networks for Cuffless Blood Pressure Estimation

    by Rui Wang, Mengshi Qi, Yingxia Shao, Anfu Zhou, Huadong Ma

    Estimating blood pressure (BP) plays a crucial role in healthcare. Traditionally, BP has been measured using a cuff, which is unsuitable for continuous BP monitoring. Recent advancements in cuffless wearable devices, such as graphene electronic tattoos and PPG sensors provide viable alternatives. However, existing BP estimation methods often overlook the importance of temporal modeling, despite the multi-periodicity and temporal dependencies inherent in cuffless BP signals. Moreover, continuous BP monitoring requires personalized modeling, which is further challenged by data scarcity. To address these two challenges, we introduce a novel Physics-Informed Temporal Network (PITN) with adversarial contrastive learning to enable precise BP estimation with very limited data for three different modalities (i.e, bioimpedance, PPG, millimeter wave). Specifically, we first introduce a novel Physics-Informed Temporal Network for investigating BP dynamics’ multi-periodicity for cardiovascular cycle modeling and temporal variation. We then apply adversarial training to generate extra physiological time series data, improving PITN’s robustness in the face of sparse subject-specific training data. Furthermore, we utilize contrastive learning to capture the discriminative variations of cardiovascular physiologic phenomena. This approach aggregates physiological signals with similar blood pressure values in latent space while separating clusters of samples with dissimilar blood pressure values. Experiments on three widely-adopted datasets with different modailties demonstrate the superiority and effectiveness of the proposed methods over previous state-of-the-art approaches.


  • Robust Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoning

    by Mengshi Qi, Changsheng Lv, Huadong Ma

    In this paper, we propose a new Robust Disentangled Counterfactual Learning (RDCL) approach for physical audiovisual commonsense reasoning. The task aims to infer objects’ physical commonsense based on both video and audio input, with the main challenge being how to imitate the reasoning ability of humans, even in the scenario of missing modalities. Most of the current methods fail to take full advantage of different characteristics in multi-modal data, and the lack of causal reasoning ability in models impedes the progress of implicit physical knowledge inference. To address these issues, our proposed RDCL method decouples videos into static (time-invariant) and dynamic (time-varying) factors in the latent space using the disentangled sequential encoder, which adopts a variational autoencoder (VAE) to maximize the mutual information with a contrastive loss function. Furthermore, we introduce a counterfactual learning module to augment the model's reasoning ability by modeling physical knowledge relationships among different objects under counterfactual intervention. To alleviate the incomplete modality data issue, we introduce a robust multimodal learning method to recover the missing data by decomposing the shared features and model-specific features. Our proposed method is a plug-and-play module that can be incorporated into any baseline, including VLMs. In experiments, we show that our proposed method improves the reasoning accuracy and robustness of baseline methods and achieves the state-of-the-art performance.