Zhenjiang Miao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Zhenjiang Miao has a total of 25 co-authors. Here are the top co-authors by number of shared publications:
- Wanru Xu
Visual Intelligence + X International Cooperation Joint Laboratory of the Ministry of Education, Beijing, China
5 shared publications - Yi Tian
Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University, Beijing, China
4 shared publications - Lili Wan
Institute of Information Science, Beijing Jiaotong University, Beijing, China
2 shared publications - Yigang Cen
Beijing Jiaotong University
2 shared publications - Jiaji Wang
Univ Hong Kong, Dept Civil Engn, Hong Kong, Peoples R China
1 shared publication - Jian Yu
Beijing Jiaotong University, Beijing, China
1 shared publication
Latest Publications
- 6 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 3313-3327
Natural Cognizing Video: A Decoupling and Integration Network for General Event Boundary Captioning
Video captioning is a prominent and challenging research area. Previous studies have focused predominantly on describing entire video segments, often overlooking the more significant status changes within these segments. We propose a novel decoupling and integration network for general event boundary captioning (DIN-GEBC), which is applied to the Kinetics-GEBC dataset, a video dataset with fine-grained status descriptions. DIN-GEBC focuses on generating three types of captions for each video segment boundary: a caption for the dominant subject (subject caption), a caption describing the event prior to the status moment (status-before), and a caption describing the event after the status moment (status-after). To address different descriptive focuses and characteristics, DIN-GEBC is proposed for decoupling and integrating both tasks and features. For task decoupling, DIN-GEBC is designed with a dual branch structure, in which the generation of the subject caption is addressed by the dominant subject branch with a Video Q-former encoder and the generation of the status-before and status-after captions is addressed by the event branch with a Reinventing RNNs for the Transformer Era (RWKV) encoder. For task integration, DIN-GEBC enables the dominant subject branch to guide the event branch in producing detailed information regarding the subject experiencing the change. Feature disentanglement, in which the common features are used by the dominant subject branch to capture the unchanging information, is also performed, and the difference features are applied to the event branch to capture the changing information. The experimental results show that our model outperforms existing models on the Kinetics-GEBC dataset even with fewer parameters. The code is available at https://github.com/Mrliu001219/DIN-GEBC.
- 6 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 3313-3327
Natural Cognizing Video: A Decoupling and Integration Network for General Event Boundary Captioning
Video captioning is a prominent and challenging research area. Previous studies have focused predominantly on describing entire video segments, often overlooking the more significant status changes within these segments. We propose a novel decoupling and integration network for general event boundary captioning (DIN-GEBC), which is applied to the Kinetics-GEBC dataset, a video dataset with fine-grained status descriptions. DIN-GEBC focuses on generating three types of captions for each video segment boundary: a caption for the dominant subject (subject caption), a caption describing the event prior to the status moment (status-before), and a caption describing the event after the status moment (status-after). To address different descriptive focuses and characteristics, DIN-GEBC is proposed for decoupling and integrating both tasks and features. For task decoupling, DIN-GEBC is designed with a dual branch structure, in which the generation of the subject caption is addressed by the dominant subject branch with a Video Q-former encoder and the generation of the status-before and status-after captions is addressed by the event branch with a Reinventing RNNs for the Transformer Era (RWKV) encoder. For task integration, DIN-GEBC enables the dominant subject branch to guide the event branch in producing detailed information regarding the subject experiencing the change. Feature disentanglement, in which the common features are used by the dominant subject branch to capture the unchanging information, is also performed, and the difference features are applied to the event branch to capture the changing information. The experimental results show that our model outperforms existing models on the Kinetics-GEBC dataset even with fewer parameters. The code is available at https://github.com/Mrliu001219/DIN-GEBC.
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 3163-3172
InstructStep: Fine-Grained Localization of Step Content and Relation in Instructional Video
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 3163-3172
Open AccessInstructStep: Fine-Grained Localization of Step Content and Relation in Instructional Video
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 8865-8873
Probabilistic Distillation Transformer: Modelling Uncertainties for Visual Abductive Reasoning
Visual abduction reasoning aims to find the most plausible explanation for incomplete observations, and suffers from inherent uncertainties and ambiguities, which mainly stem from the latent causal relations, incomplete observations, and the reasoning itself. To address this, we propose a probabilistic model named Uncertainty-Guided Probabilistic Distillation Transformer (UPD-Trans) to model uncertainties for Visual Abductive Reasoning. In order to better discover the correct cause-effect chain, we model all the potential causal relations into a unified reasoning framework, thus both the direct relations and latent relations are considered. In order to reduce the effect of the stochasticity and uncertainty for reasoning: 1) we extend the deterministic Transformer to a probabilistic Transformer by considering those uncertain factors as Gaussian random variables and explicitly modeling their distribution; 2) we introduce a distillation mechanism between the posterior branch with complete observations and the prior branch with incomplete observations to transfer posterior knowledge. Evaluation results on the benchmark datasets, consistently demonstrate the commendable performance of our UPD-Trans, with significant improvements after latent relation modeling and uncertainty modeling.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 8865-8873
Open AccessProbabilistic Distillation Transformer: Modelling Uncertainties for Visual Abductive Reasoning
Visual abduction reasoning aims to find the most plausible explanation for incomplete observations, and suffers from inherent uncertainties and ambiguities, which mainly stem from the latent causal relations, incomplete observations, and the reasoning itself. To address this, we propose a probabilistic model named Uncertainty-Guided Probabilistic Distillation Transformer (UPD-Trans) to model uncertainties for Visual Abductive Reasoning. In order to better discover the correct cause-effect chain, we model all the potential causal relations into a unified reasoning framework, thus both the direct relations and latent relations are considered. In order to reduce the effect of the stochasticity and uncertainty for reasoning: 1) we extend the deterministic Transformer to a probabilistic Transformer by considering those uncertain factors as Gaussian random variables and explicitly modeling their distribution; 2) we introduce a distillation mechanism between the posterior branch with complete observations and the prior branch with incomplete observations to transfer posterior knowledge. Evaluation results on the benchmark datasets, consistently demonstrate the commendable performance of our UPD-Trans, with significant improvements after latent relation modeling and uncertainty modeling.
- Open Access
- 11 July 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 22 (9) , 1-22
Boosting Semi-Supervised Video Captioning via Learning Candidates Adjusters
Video captioning is a multimodal task on both CV and NLP, whose goal is to automatically obtain the description of video content with natural language statements. Although there are amounts of video data, their annotations with description sentences are very limited. In this paper, we define the semi-supervised video captioning (SSVC) problem in order to improve performance with limited annotations by leveraging the semantic knowledge from both well-annotated samples and no-annotated samples. To address the problem, we introduce a LCA-boosted model (LCABM) for boosting SSVC, where it is first to explore a learnable candidates adjuster to adjust the caption candidates and then treat these adjusted captions as pesudo labels to train the SSVC model with no-annotated samples in reverse. In particular, the model learning is considered as a bi-level optimization problem and solved by an EM-like multi-stage training algorithm. The experiments show the effectiveness of our proposed LCABM, whose performance is comparable and even better than those state-of-the-art fully-supervised methods even with less annotations.
- 11 July 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 22 (9) , 1-22
Open AccessBoosting Semi-Supervised Video Captioning via Learning Candidates Adjusters
Video captioning is a multimodal task on both CV and NLP, whose goal is to automatically obtain the description of video content with natural language statements. Although there are amounts of video data, their annotations with description sentences are very limited. In this paper, we define the semi-supervised video captioning (SSVC) problem in order to improve performance with limited annotations by leveraging the semantic knowledge from both well-annotated samples and no-annotated samples. To address the problem, we introduce a LCA-boosted model (LCABM) for boosting SSVC, where it is first to explore a learnable candidates adjuster to adjust the caption candidates and then treat these adjusted captions as pesudo labels to train the SSVC model with no-annotated samples in reverse. In particular, the model learning is considered as a bi-level optimization problem and solved by an EM-like multi-stage training algorithm. The experiments show the effectiveness of our proposed LCABM, whose performance is comparable and even better than those state-of-the-art fully-supervised methods even with less annotations.
- Open Access
- 1 May 2019
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 1229 (1) , 012033
A Method of License Plate Recognition Based on BP Neural Network with Median Filtering
With the rapid increase of cars, the research of license plate recognition in intelligent traffic algorithm has attracted much attention. In this paper, pointing at the shortcomings of the existing recognition algorithm, we adopt a license plate recognition method based on BP neural network with a median filtering. The method consists of a median filtering algorithm for denoising and a BP neural network for training and testing. First the character library is processed by median filtering to remove the influence of noise. Then, utilizing the strong learning ability of BP, the character library is passed into the network for classification. Compared with the traditional algorithm, our method not only speeds up the recognition efficiency, but also obtains more robust results. Therefore, the proposed approach can provide a technical support for the practical application of license plate recognizing.
- 1 May 2019
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 1229 (1) , 012033
Open AccessA Method of License Plate Recognition Based on BP Neural Network with Median Filtering
With the rapid increase of cars, the research of license plate recognition in intelligent traffic algorithm has attracted much attention. In this paper, pointing at the shortcomings of the existing recognition algorithm, we adopt a license plate recognition method based on BP neural network with a median filtering. The method consists of a median filtering algorithm for denoising and a BP neural network for training and testing. First the character library is processed by median filtering to remove the influence of noise. Then, utilizing the strong learning ability of BP, the character library is passed into the network for classification. Compared with the traditional algorithm, our method not only speeds up the recognition efficiency, but also obtains more robust results. Therefore, the proposed approach can provide a technical support for the practical application of license plate recognizing.
- Open Access
- 19 March 2015
- Published by Springer Nature
- p. 786-801
Learning Spatio-Temporal Features for Action Recognition with Modified Hidden Conditional Random Field
- 19 March 2015
- Published by Springer Nature
- p. 786-801
Open AccessLearning Spatio-Temporal Features for Action Recognition with Modified Hidden Conditional Random Field
- 29 August 2013
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 24 (3) , 361-373
Multihuman Tracking Based on a Spatial–Temporal Appearance Match
In this paper, we focus on the improvements of appearance representation for multihuman tracking. Many previous methods extracted low-level appearance features, such as color histogram and texture, even combined with spatial information for each frame. These methods ignore the temporal distribution of features. The features of each frame may not be stable due to illumination, human pose variation, and image noise. In order to improve it, we propose a novel appearance representation called the spatial-temporal appearance model based on the statistical distribution of Gaussian mixture model (GMM). It represents the appearance of a tracklet as a whole with dynamic spatial and temporal information. The spatial information is the dynamic subregions. The temporal information is the dynamic duration time of each subregion. Each subregion is modeled as the weighted Gaussian distribution of GMM. The online expectation-maximization (online EM) algorithm is used to estimate the parameters of GMM. Then, we propose a tracklet association method using Bayesian prediction and Jensen-Shannon divergence. The Bayesian prediction is used to predict the locations of targets. The Jensen-Shannon divergence is used to compute the distance of spatial-temporal appearance distribution between two tracklets. Finally, we test our approach on four challenging datasets (TRECVID, CAVIAR, ETH, and EPFL Terrace) and achieve good results.
- 29 August 2013
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 24 (3) , 361-373
Multihuman Tracking Based on a Spatial–Temporal Appearance Match
In this paper, we focus on the improvements of appearance representation for multihuman tracking. Many previous methods extracted low-level appearance features, such as color histogram and texture, even combined with spatial information for each frame. These methods ignore the temporal distribution of features. The features of each frame may not be stable due to illumination, human pose variation, and image noise. In order to improve it, we propose a novel appearance representation called the spatial-temporal appearance model based on the statistical distribution of Gaussian mixture model (GMM). It represents the appearance of a tracklet as a whole with dynamic spatial and temporal information. The spatial information is the dynamic subregions. The temporal information is the dynamic duration time of each subregion. Each subregion is modeled as the weighted Gaussian distribution of GMM. The online expectation-maximization (online EM) algorithm is used to estimate the parameters of GMM. Then, we propose a tracklet association method using Bayesian prediction and Jensen-Shannon divergence. The Bayesian prediction is used to predict the locations of targets. The Jensen-Shannon divergence is used to compute the distance of spatial-temporal appearance distribution between two tracklets. Finally, we test our approach on four challenging datasets (TRECVID, CAVIAR, ETH, and EPFL Terrace) and achieve good results.
- 1 May 2012
- Published by SPIE-Intl Soc Optical Eng
- Vol. 8334, 83344K-83344K-5
An image inpainting method based on image retrieval
- 1 May 2012
- Published by SPIE-Intl Soc Optical Eng
- Vol. 8334, 83344K-83344K-5
An image inpainting method based on image retrieval
- 1 January 2006
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 21593442,p. 1-4
A Pervasive Computing Security System Based on Human Activities Analysis
In recent years, the research on intelligent environment based on pervasive/ubiquitous computing technology has attracted more and more attention, especially in security such as KidsRoom by MIT AI Lab. Considering practical applications of pervasive computing technology, it is necessary to present a pervasive computing system model in the view of engineering. In this paper, we illustrated an intelligent surveillant security system based on a HPC model. This system can detect moving person, recognize his behavior, and send out alerting messages such as SMS when intruder or strange behavior is detected. We design this system based on pervasive computing and propose the implementation methods step by step
- 1 January 2006
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 21593442,p. 1-4
A Pervasive Computing Security System Based on Human Activities Analysis
In recent years, the research on intelligent environment based on pervasive/ubiquitous computing technology has attracted more and more attention, especially in security such as KidsRoom by MIT AI Lab. Considering practical applications of pervasive computing technology, it is necessary to present a pervasive computing system model in the view of engineering. In this paper, we illustrated an intelligent surveillant security system based on a HPC model. This system can detect moving person, recognize his behavior, and send out alerting messages such as SMS when intruder or strange behavior is detected. We design this system based on pervasive computing and propose the implementation methods step by step
- 1 March 1999
- Published by Elsevier in Neural Networks
- Vol. 12 (2) , 259-271
Analysis and optimal design of continuous neural networks with applications to associative memory
- 1 March 1999
- Published by Elsevier in Neural Networks
- Vol. 12 (2) , 259-271
Analysis and optimal design of continuous neural networks with applications to associative memory