Kun Dong
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Kun Dong has a total of 23 co-authors. Here are the top co-authors by number of shared publications:
- Jian Xue
University of Chinese Academy of Sciences, Beijing, China
12 shared publications - Ke Lu
School of Software, Henan University, Kaifeng, China
9 shared publications - Zehai Niu
School of Artificial Intelligence, Shenzhen University, Shenzhen, China
7 shared publications - Qingyuan Liu
University of Chinese Academy of Sciences
6 shared publications - Xing Lan
School of Engineering Science, University of Chinese Academy of Sciences, Beijing, China
6 shared publications - Xiaoyu Qin
Department of Computer Science and Technology, Tsinghua University, Beijing, China
4 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 10-18
TSRNet: Two Stream Attention and Retentive Network for Temporal Action Detection
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 10-18
Open AccessTSRNet: Two Stream Attention and Retentive Network for Temporal Action Detection
- 6 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 2325-2336
MotionFlow: Efficient Motion Generation With Latent Flow Matching
In the field of human centric multimedia, text-driven human motion generation is a significant pursuit with wide-ranging applications across diverse scenarios. Despite substantial advancements, existing methods often suffer from a trade-off between inference latency and high-quality generation. To overcome this gap, we propose the Motion Latent Flow Matching model (MotionFlow), a novel and powerful framework for motion generation. It introduces flow matching algorithm in the latent space, which can achieve superior performance with just one-step inference. In addition to the text-driven task, we further extend our method to controllable motion generation. Specifically, we integrate a control encoder into the latent space and further decode the predicted latent code into motion space to support explicit supervision, ensuring the synthesized motion can tightly align with the input signals. Extensive experiments demonstrate that our MotionFlow not only outperforms current leading approaches for the text-driven task, but also delivers remarkable capabilities in controllable motion generation.
- 6 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 2325-2336
MotionFlow: Efficient Motion Generation With Latent Flow Matching
In the field of human centric multimedia, text-driven human motion generation is a significant pursuit with wide-ranging applications across diverse scenarios. Despite substantial advancements, existing methods often suffer from a trade-off between inference latency and high-quality generation. To overcome this gap, we propose the Motion Latent Flow Matching model (MotionFlow), a novel and powerful framework for motion generation. It introduces flow matching algorithm in the latent space, which can achieve superior performance with just one-step inference. In addition to the text-driven task, we further extend our method to controllable motion generation. Specifically, we integrate a control encoder into the latent space and further decode the predicted latent code into motion space to support explicit supervision, ensuring the synthesized motion can tightly align with the input signals. Extensive experiments demonstrate that our MotionFlow not only outperforms current leading approaches for the text-driven task, but also delivers remarkable capabilities in controllable motion generation.
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 35, 9400-9413
SRGT: Spatial-Resolution-Guided Teacher for Robust Semi-Supervised Aerial Object Detection
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 35, 9400-9413
SRGT: Spatial-Resolution-Guided Teacher for Robust Semi-Supervised Aerial Object Detection
- Open Access
- 30 November 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
Towards Accurate and Smooth Human Motion Reconstruction from Sparse Tracking with State Space Model
Reconstructing accurate full-body poses from sparse tracking is crucial in generative multimedia, yet achieving effective sequence modeling and high-quality generation remains challenging. Recently, the state space models (SSMs), especially Mamba, have showcased considerable promise in sequence modeling, indicating an appealing direction for building motion generation models. However, adapting SSM to this task is quite challenging due to two main aspects. Firstly, the sparsity of input conditions poses significant difficulty in accurately estimating full-body motions. Secondly, SSM lacks a specialized design to capture the temporal dependencies within motion sequences. To address these challenges, we propose a novel conditional diffusion model, Pose Mamba Diffusion (PMDiff), which comprises a bidirectional Pose Mamba Denoiser to achieve effective sequence modeling. Our PMDiff can effectively leverage the robust generation capabilities of diffusion models in conjunction with the sequence modeling proficiency of Mamba to track full-body poses accurately. Additionally, we notice that synthesized human motions often lack smoothness, resulting in unnatural motion sequences. To overcome this issue, we further introduce a learning-free Temporal Motion Filter (TMF), which can remarkably improve the smoothness of the generated human motion. Extensive experiments conducted on the large motion capture database (AMASS) show that our PMDiff outperforms previous state-of-the-art methods in terms of both accuracy and smoothness.
- 30 November 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
Open AccessTowards Accurate and Smooth Human Motion Reconstruction from Sparse Tracking with State Space Model
Reconstructing accurate full-body poses from sparse tracking is crucial in generative multimedia, yet achieving effective sequence modeling and high-quality generation remains challenging. Recently, the state space models (SSMs), especially Mamba, have showcased considerable promise in sequence modeling, indicating an appealing direction for building motion generation models. However, adapting SSM to this task is quite challenging due to two main aspects. Firstly, the sparsity of input conditions poses significant difficulty in accurately estimating full-body motions. Secondly, SSM lacks a specialized design to capture the temporal dependencies within motion sequences. To address these challenges, we propose a novel conditional diffusion model, Pose Mamba Diffusion (PMDiff), which comprises a bidirectional Pose Mamba Denoiser to achieve effective sequence modeling. Our PMDiff can effectively leverage the robust generation capabilities of diffusion models in conjunction with the sequence modeling proficiency of Mamba to track full-body poses accurately. Additionally, we notice that synthesized human motions often lack smoothness, resulting in unnatural motion sequences. To overcome this issue, we further introduce a learning-free Temporal Motion Filter (TMF), which can remarkably improve the smoothness of the generated human motion. Extensive experiments conducted on the large motion capture database (AMASS) show that our PMDiff outperforms previous state-of-the-art methods in terms of both accuracy and smoothness.
- Open Access
- 12 November 2025
- Published by Springer Nature in Research Square
MPRW-HGNN: A Meta-path Random Walk based Heterogeneous Graph Neural Network
In recent years, graph neural networks have been widely used in multimedia information retrieval and other graph data processing tasks due to their powerful feature learning capabilities. However, existing graph neural network models typically handle only homogeneous graph data with simple structures and single semantics effectively. However, when faced with heterogeneous multimedia data characterized by complex interactions and rich semantics, the performance of traditional models degrades significantly. To address this challenge, this paper proposes a heterogeneous graph neural network algorithm based on meta-path random walk (MPRW-HGNN). First, we design a module at the meta-path instance level that generates meta-path instances and their structural representations via random walks. Then, a soft-attention mechanism is employed to fuse information from multiple meta-paths, better capturing the fine-grained semantic structures around nodes. Subsequently, we utilize a self-attention mechanism to explore semantic correlations and differences between multiple paths, enabling adaptive weighted fusion of multiple path information to generate robust node features for heterogeneous graph data. Through extensive experiments on information retrieval tasks using the IMDB movie dataset and DBLP academic paper dataset, we demonstrate the significant advantages of our proposed algorithm in handling multimodal data and improving retrieval accuracy.
- 12 November 2025
- Published by Springer Nature in Research Square
Open AccessMPRW-HGNN: A Meta-path Random Walk based Heterogeneous Graph Neural Network
In recent years, graph neural networks have been widely used in multimedia information retrieval and other graph data processing tasks due to their powerful feature learning capabilities. However, existing graph neural network models typically handle only homogeneous graph data with simple structures and single semantics effectively. However, when faced with heterogeneous multimedia data characterized by complex interactions and rich semantics, the performance of traditional models degrades significantly. To address this challenge, this paper proposes a heterogeneous graph neural network algorithm based on meta-path random walk (MPRW-HGNN). First, we design a module at the meta-path instance level that generates meta-path instances and their structural representations via random walks. Then, a soft-attention mechanism is employed to fuse information from multiple meta-paths, better capturing the fine-grained semantic structures around nodes. Subsequently, we utilize a self-attention mechanism to explore semantic correlations and differences between multiple paths, enabling adaptive weighted fusion of multiple path information to generate robust node features for heterogeneous graph data. Through extensive experiments on information retrieval tasks using the IMDB movie dataset and DBLP academic paper dataset, we demonstrate the significant advantages of our proposed algorithm in handling multimodal data and improving retrieval accuracy.
- 12 May 2025
- Published by Springer Nature
- p. 274-289
FlexControl: Flexible and Efficient Full-Body Controllable Text-to-Motion Generation
- 12 May 2025
- Published by Springer Nature
- p. 274-289
FlexControl: Flexible and Efficient Full-Body Controllable Text-to-Motion Generation
- 12 May 2025
- Published by Springer Nature
- p. 307-323
Motion Reconstruction via Human Anatomy Diffusion from Sparse Tracking
- 12 May 2025
- Published by Springer Nature
- p. 307-323
Motion Reconstruction via Human Anatomy Diffusion from Sparse Tracking
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 4024-4033
Realistic Full-Body Motion Generation from Sparse Tracking with State Space Model
In the domain of generative multimedia and interactive experiences, generating realistic and accurate full-body poses from sparse tracking is crucial for many real-world applications, while achieving sequence modeling and efficient motion generation remains challenging. Recently, state space models (SSMs) with efficient hardware-aware designs (i.e., Mamba) have shown great potential for sequence modeling, particularly in temporal contexts. However, processing motion data is still challenging for SSMs. Specifically, the sparsity of input conditions makes motion generation an ill-posed problem. Moreover, the complex structure of the human body further complicates this task. To address these issues, we present Motion Mamba Diffusion (MMD), a novel conditional diffusion model, which effectively utilizes the sequence modeling capability of SSMs and the robust generation ability of diffusion models to track full-body poses accurately. In particular, we design a bidirectional Temporal Mamba Module (TMM) to model motion sequence. Additionally, a Spatial Mamba Module (SMM) is further proposed for feature enhancement within a single frame. Extensive experiments on the large motion capture dataset (AMASS) demonstrate that our proposed approach outperforms the latest methods in terms of accuracy and smoothness, thus providing a crucial advancement for creating realistic virtual avatars in various applications.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 4024-4033
Open AccessRealistic Full-Body Motion Generation from Sparse Tracking with State Space Model
In the domain of generative multimedia and interactive experiences, generating realistic and accurate full-body poses from sparse tracking is crucial for many real-world applications, while achieving sequence modeling and efficient motion generation remains challenging. Recently, state space models (SSMs) with efficient hardware-aware designs (i.e., Mamba) have shown great potential for sequence modeling, particularly in temporal contexts. However, processing motion data is still challenging for SSMs. Specifically, the sparsity of input conditions makes motion generation an ill-posed problem. Moreover, the complex structure of the human body further complicates this task. To address these issues, we present Motion Mamba Diffusion (MMD), a novel conditional diffusion model, which effectively utilizes the sequence modeling capability of SSMs and the robust generation ability of diffusion models to track full-body poses accurately. In particular, we design a bidirectional Temporal Mamba Module (TMM) to model motion sequence. Additionally, a Spatial Mamba Module (SMM) is further proposed for feature enhancement within a single frame. Extensive experiments on the large motion capture dataset (AMASS) demonstrate that our proposed approach outperforms the latest methods in terms of accuracy and smoothness, thus providing a crucial advancement for creating realistic virtual avatars in various applications.
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 13-22
AdaptControl: Adaptive Human Motion Control and Generation via User Prompt and Spatial Trajectory Guidance
In the field of generative multimedia and interactive experiences, human motion generation guided by natural language and spatial signals has emerged as a critical research direction. However, generating human motions that precisely follow spatial trajectories while maintaining naturalness remains a significant challenge. Existing methods often struggle to achieve a balance between spatial control precision and motion realism. Moreover, the spatial guidance signals and the relative rotations of human limbs reside in different representation spaces, complicating the precise control of human motions along specified paths. To address these issues, we propose AdaptControl, a motion generation model capable of precise spatial trajectory control while ensuring that the generated motion adheres to both text descriptions and spatial guidance. AdaptControl employs a Natural Guidance module to optimize the generated motion noise while closely following the spatial constraints and maintaining realistic dynamics. Simultaneously, the Control Dominance Adjustor adaptively fuses features from the text and spatial signals, allowing the model to consider information from both sources. Finally, the Fusion Diffusion module integrates the guided noise from the Natural Guidance module with the fused features from the Control Dominance Adjustor to generate the final output motion. Extensive experiments demonstrate that AdaptControl outperforms state-of-the-art methods in trajectory adherence, motion realism, and semantic consistency, opening up new possibilities for intuitive and interactive human motion generation and facilitating the development of user-centered design and interactive applications.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 13-22
Open AccessAdaptControl: Adaptive Human Motion Control and Generation via User Prompt and Spatial Trajectory Guidance
In the field of generative multimedia and interactive experiences, human motion generation guided by natural language and spatial signals has emerged as a critical research direction. However, generating human motions that precisely follow spatial trajectories while maintaining naturalness remains a significant challenge. Existing methods often struggle to achieve a balance between spatial control precision and motion realism. Moreover, the spatial guidance signals and the relative rotations of human limbs reside in different representation spaces, complicating the precise control of human motions along specified paths. To address these issues, we propose AdaptControl, a motion generation model capable of precise spatial trajectory control while ensuring that the generated motion adheres to both text descriptions and spatial guidance. AdaptControl employs a Natural Guidance module to optimize the generated motion noise while closely following the spatial constraints and maintaining realistic dynamics. Simultaneously, the Control Dominance Adjustor adaptively fuses features from the text and spatial signals, allowing the model to consider information from both sources. Finally, the Fusion Diffusion module integrates the guided noise from the Natural Guidance module with the fused features from the Control Dominance Adjustor to generate the final output motion. Extensive experiments demonstrate that AdaptControl outperforms state-of-the-art methods in trajectory adherence, motion realism, and semantic consistency, opening up new possibilities for intuitive and interactive human motion generation and facilitating the development of user-centered design and interactive applications.
- 13 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (12) , 13016-13028
Does Pixel Value Represent Facial Landmark Well in Heatmap?
Heatmap-based methods have dominated the face alignment task, yet the maximum response decoding scheme necessitates further reform. While some studies have attempted to compensate for prediction offsets using a post-processing module, the prediction errors induced by the maximum response decoding scheme remain challenging to rectify. In this paper, we assume that using heatmap value to denote the ground-truth probability is not accurate enough. To cure this problem, we propose DISPAL, a novel DIS tribution-based P robability for f A cial L andmarks, which signifies the ground-truth probability by the similarity between the pixel’s neighbouring value distribution and Gaussian distribution. This innovative probability enables us to pinpoint the keypoint location more robustly than previous methods that rely solely on the peak score. It also exhibits remarkable generalization to complex decoding methodologies. Furthermore, we propose supervising this probability as an additional task loss to help the model learn better heatmap representation. Extensive empirical results on WFLW, 300W, and COFW datasets demonstrate that our distribution-based probability mechanism significantly surpasses original value-based probability approaches.
- 13 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (12) , 13016-13028
Does Pixel Value Represent Facial Landmark Well in Heatmap?
Heatmap-based methods have dominated the face alignment task, yet the maximum response decoding scheme necessitates further reform. While some studies have attempted to compensate for prediction offsets using a post-processing module, the prediction errors induced by the maximum response decoding scheme remain challenging to rectify. In this paper, we assume that using heatmap value to denote the ground-truth probability is not accurate enough. To cure this problem, we propose DISPAL, a novel DIS tribution-based P robability for f A cial L andmarks, which signifies the ground-truth probability by the similarity between the pixel’s neighbouring value distribution and Gaussian distribution. This innovative probability enables us to pinpoint the keypoint location more robustly than previous methods that rely solely on the peak score. It also exhibits remarkable generalization to complex decoding methodologies. Furthermore, we propose supervising this probability as an additional task loss to help the model learn better heatmap representation. Extensive empirical results on WFLW, 300W, and COFW datasets demonstrate that our distribution-based probability mechanism significantly surpasses original value-based probability approaches.