Zhimin Gao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Zhimin Gao has a total of 55 co-authors. Here are the top co-authors by number of shared publications:
- Mingliang Xu
Zhengzhou University
11 shared publications - Wanqing Li
University of Wollongong
10 shared publications - Pichao Wang
Amazon
9 shared publications - Yonghong Hou
School of Electrical and Information Engineering, Tianjin University, Tianjin, China
6 shared publications - Xiaoheng Jiang
Zhengzhou University
3 shared publications - Dong Yang
School of Civil Engineering, Chongqing University 1 , No. 83 Shabeijie, Shapingba, Chongqing 400045
2 shared publications
Latest Publications
- Open Access
Human Pose-based Estimation, Tracking and Action Recognition with Deep Learning: A Survey
Human pose analysis has garnered significant attention within both the research community and practical applications, owing to its expanding array of uses, including gaming, video surveillance, sports performance analysis, and human-computer interactions, among others. The advent of deep learning has significantly improved the accuracy of pose capture, making pose-based applications increasingly practical. This paper presents a comprehensive survey of pose-based applications utilizing deep learning, encompassing pose estimation, pose tracking, and action recognition.Pose estimation involves the determination of human joint positions from images or image sequences. Pose tracking is an emerging research direction aimed at generating consistent human pose trajectories over time. Action recognition, on the other hand, targets the identification of action types using pose estimation or tracking data. These three tasks are intricately interconnected, with the latter often reliant on the former. In this survey, we comprehensively review related works, spanning from single-person pose estimation to multi-person pose estimation, from 2D pose estimation to 3D pose estimation, from single image to video, from mining temporal context gradually to pose tracking, and lastly from tracking to pose-based action recognition. As a survey centered on the application of deep learning to pose analysis, we explicitly discuss both the strengths and limitations of existing techniques. Notably, we emphasize methodologies for integrating these three tasks into a unified framework within video sequences. Additionally, we explore the challenges involved and outline potential directions for future research.
Open AccessHuman Pose-based Estimation, Tracking and Action Recognition with Deep Learning: A Survey
Human pose analysis has garnered significant attention within both the research community and practical applications, owing to its expanding array of uses, including gaming, video surveillance, sports performance analysis, and human-computer interactions, among others. The advent of deep learning has significantly improved the accuracy of pose capture, making pose-based applications increasingly practical. This paper presents a comprehensive survey of pose-based applications utilizing deep learning, encompassing pose estimation, pose tracking, and action recognition.Pose estimation involves the determination of human joint positions from images or image sequences. Pose tracking is an emerging research direction aimed at generating consistent human pose trajectories over time. Action recognition, on the other hand, targets the identification of action types using pose estimation or tracking data. These three tasks are intricately interconnected, with the latter often reliant on the former. In this survey, we comprehensively review related works, spanning from single-person pose estimation to multi-person pose estimation, from 2D pose estimation to 3D pose estimation, from single image to video, from mining temporal context gradually to pose tracking, and lastly from tracking to pose-based action recognition. As a survey centered on the application of deep learning to pose analysis, we explicitly discuss both the strengths and limitations of existing techniques. Notably, we emphasize methodologies for integrating these three tasks into a unified framework within video sequences. Additionally, we explore the challenges involved and outline potential directions for future research.
- 2 March 2023
- Published by Springer Nature
- p. 155-171
Focal and Global Spatial-Temporal Transformer for Skeleton-Based Action Recognition
- 2 March 2023
- Published by Springer Nature
- p. 155-171
Focal and Global Spatial-Temporal Transformer for Skeleton-Based Action Recognition
- 7 October 2022
- Published by Springer Nature in Neural Computing and Applications
- Vol. 35  (2) , 2007-2024
FT-HID: a large-scale RGB-D dataset for first- and third-person human interaction analysis
- 7 October 2022
- Published by Springer Nature in Neural Computing and Applications
- Vol. 35  (2) , 2007-2024
FT-HID: a large-scale RGB-D dataset for first- and third-person human interaction analysis
- Open Access
Focal and Global Spatial-Temporal Transformer for Skeleton-based Action Recognition
Despite great progress achieved by transformer in various vision tasks, it is still underexplored for skeleton-based action recognition with only a few attempts. Besides, these methods directly calculate the pair-wise global self-attention equally for all the joints in both the spatial and temporal dimensions, undervaluing the effect of discriminative local joints and the short-range temporal dynamics. In this work, we propose a novel Focal and Global Spatial-Temporal Transformer network (FG-STFormer), that is equipped with two key components: (1) FG-SFormer: focal joints and global parts coupling spatial transformer. It forces the network to focus on modelling correlations for both the learned discriminative spatial joints and human body parts respectively. The selective focal joints eliminate the negative effect of non-informative ones during accumulating the correlations. Meanwhile, the interactions between the focal joints and body parts are incorporated to enhance the spatial dependencies via mutual cross-attention. (2) FG-TFormer: focal and global temporal transformer. Dilated temporal convolution is integrated into the global self-attention mechanism to explicitly capture the local temporal motion patterns of joints or body parts, which is found to be vital important to make temporal transformer work. Extensive experimental results on three benchmarks, namely NTU-60, NTU-120 and NW-UCLA, show our FG-STFormer surpasses all existing transformer-based methods, and compares favourably with state-of-the art GCN-based methods.
Open AccessFocal and Global Spatial-Temporal Transformer for Skeleton-based Action Recognition
Despite great progress achieved by transformer in various vision tasks, it is still underexplored for skeleton-based action recognition with only a few attempts. Besides, these methods directly calculate the pair-wise global self-attention equally for all the joints in both the spatial and temporal dimensions, undervaluing the effect of discriminative local joints and the short-range temporal dynamics. In this work, we propose a novel Focal and Global Spatial-Temporal Transformer network (FG-STFormer), that is equipped with two key components: (1) FG-SFormer: focal joints and global parts coupling spatial transformer. It forces the network to focus on modelling correlations for both the learned discriminative spatial joints and human body parts respectively. The selective focal joints eliminate the negative effect of non-informative ones during accumulating the correlations. Meanwhile, the interactions between the focal joints and body parts are incorporated to enhance the spatial dependencies via mutual cross-attention. (2) FG-TFormer: focal and global temporal transformer. Dilated temporal convolution is integrated into the global self-attention mechanism to explicitly capture the local temporal motion patterns of joints or body parts, which is found to be vital important to make temporal transformer work. Extensive experimental results on three benchmarks, namely NTU-60, NTU-120 and NW-UCLA, show our FG-STFormer surpasses all existing transformer-based methods, and compares favourably with state-of-the art GCN-based methods.
- Open Access
FT-HID: A Large Scale RGB-D Dataset for First and Third Person Human Interaction Analysis
Analysis of human interaction is one important research topic of human motion analysis. It has been studied either using first person vision (FPV) or third person vision (TPV). However, the joint learning of both types of vision has so far attracted little attention. One of the reasons is the lack of suitable datasets that cover both FPV and TPV. In addition, existing benchmark datasets of either FPV or TPV have several limitations, including the limited number of samples, participant subjects, interaction categories, and modalities. In this work, we contribute a large-scale human interaction dataset, namely, FT-HID dataset. FT-HID contains pair-aligned samples of first person and third person visions. The dataset was collected from 109 distinct subjects and has more than 90K samples for three modalities. The dataset has been validated by using several existing action recognition methods. In addition, we introduce a novel multi-view interaction mechanism for skeleton sequences, and a joint learning multi-stream framework for first person and third person visions. Both methods yield promising results on the FT-HID dataset. It is expected that the introduction of this vision-aligned large-scale dataset will promote the development of both FPV and TPV, and their joint learning techniques for human action analysis. The dataset and code are available at \href{https://github.com/ENDLICHERE/FT-HID}{here}.
Open AccessFT-HID: A Large Scale RGB-D Dataset for First and Third Person Human Interaction Analysis
Analysis of human interaction is one important research topic of human motion analysis. It has been studied either using first person vision (FPV) or third person vision (TPV). However, the joint learning of both types of vision has so far attracted little attention. One of the reasons is the lack of suitable datasets that cover both FPV and TPV. In addition, existing benchmark datasets of either FPV or TPV have several limitations, including the limited number of samples, participant subjects, interaction categories, and modalities. In this work, we contribute a large-scale human interaction dataset, namely, FT-HID dataset. FT-HID contains pair-aligned samples of first person and third person visions. The dataset was collected from 109 distinct subjects and has more than 90K samples for three modalities. The dataset has been validated by using several existing action recognition methods. In addition, we introduce a novel multi-view interaction mechanism for skeleton sequences, and a joint learning multi-stream framework for first person and third person visions. Both methods yield promising results on the FT-HID dataset. It is expected that the introduction of this vision-aligned large-scale dataset will promote the development of both FPV and TPV, and their joint learning techniques for human action analysis. The dataset and code are available at \href{https://github.com/ENDLICHERE/FT-HID}{here}.
- 1 November 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 32  (7) , 4893-4899
A Central Difference Graph Convolutional Operator for Skeleton-Based Action Recognition
This paper proposes a new graph convolutional operator called central difference graph convolution (CDGC) for skeleton based action recognition. It is not only able to aggregate node information like a vanilla graph convolutional operation but also gradient information. Without introducing any additional parameters, CDGC can replace vanilla graph convolution in any existing Graph Convolutional Networks (GCNs). In addition, an accelerated version of the CDGC is developed which greatly improves the speed of training. Experiments on two popular large-scale datasets NTU RGB+D 60 & 120 have demonstrated the efficacy of the proposed CDGC. Code is available at https://github.com/iesymiao/CD-GCN .
- 1 November 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 32  (7) , 4893-4899
A Central Difference Graph Convolutional Operator for Skeleton-Based Action Recognition
This paper proposes a new graph convolutional operator called central difference graph convolution (CDGC) for skeleton based action recognition. It is not only able to aggregate node information like a vanilla graph convolutional operation but also gradient information. Without introducing any additional parameters, CDGC can replace vanilla graph convolution in any existing Graph Convolutional Networks (GCNs). In addition, an accelerated version of the CDGC is developed which greatly improves the speed of training. Experiments on two popular large-scale datasets NTU RGB+D 60 & 120 have demonstrated the efficacy of the proposed CDGC. Code is available at https://github.com/iesymiao/CD-GCN .
- Open Access
- 1 August 2021
- Published by IOP Publishing in IOP Conference Series: Earth and Environmental Science
- Vol. 838  (1) , 012010
Frequency Control Strategy of DFIGs based on Improved Virtual Inertia Method
The DFIGs can participate in frequency control by virtual inertia method. After exiting the frequency control, secondary frequency drop will occur inevitably. The current study of the problem mostly focused on the rotor speed recovery strategy and PD control parameter optimization problems, which cannot eliminate the phenomenon of secondary frequency drop in essence. This paper proposes a frequency control strategy based on the improved virtual inertia control method. When the DFIG exits the frequency modulation process, the supplementary control component based on the turbine speed is added to avoid the sudden drop of output power, so as to suppress the secondary frequency drop. Finally, a simulation model is built in Matlab/Simulink, and the feasibility of the proposed strategy is verified in two case studies of power system frequency reduction and load fluctuation.
- 1 August 2021
- Published by IOP Publishing in IOP Conference Series: Earth and Environmental Science
- Vol. 838  (1) , 012010
Open AccessFrequency Control Strategy of DFIGs based on Improved Virtual Inertia Method
The DFIGs can participate in frequency control by virtual inertia method. After exiting the frequency control, secondary frequency drop will occur inevitably. The current study of the problem mostly focused on the rotor speed recovery strategy and PD control parameter optimization problems, which cannot eliminate the phenomenon of secondary frequency drop in essence. This paper proposes a frequency control strategy based on the improved virtual inertia control method. When the DFIG exits the frequency modulation process, the supplementary control component based on the turbine speed is added to avoid the sudden drop of output power, so as to suppress the secondary frequency drop. Finally, a simulation model is built in Matlab/Simulink, and the feasibility of the proposed strategy is verified in two case studies of power system frequency reduction and load fluctuation.
- Open Access
- 1 August 2021
- Published by IOP Publishing in IOP Conference Series: Earth and Environmental Science
- Vol. 838  (1) , 012006
Event-Driven Fast Frequency Response for Energy Storage System Considering State of Charge
The increasing penetration of renewable energy sources reduces the equivalent inertia and spinning reserve. Power systems are becoming more and more vulnerable when subjected to active power deficits. With the trait of quick response and flexible ramp, the energy storage system (ESS) offers a promising capability to counter frequency decline after large disturbances. This paper proposes an event-driven fast frequency response (EFFR) approach for the ESS to sustain frequency stability. A centralized EFFR control framework is established, in which the wide area information is collected, and the disturbance power is broadcasted to ESSs under contingencies. Moreover, a fuzzy logic controller (FLC) is employed to determine the power set-point of the ESS based on the disturbance power and state of charge (SOC). A case study is presented to validate the feasibility and effectiveness of the proposed approach.
- 1 August 2021
- Published by IOP Publishing in IOP Conference Series: Earth and Environmental Science
- Vol. 838  (1) , 012006
Open AccessEvent-Driven Fast Frequency Response for Energy Storage System Considering State of Charge
The increasing penetration of renewable energy sources reduces the equivalent inertia and spinning reserve. Power systems are becoming more and more vulnerable when subjected to active power deficits. With the trait of quick response and flexible ramp, the energy storage system (ESS) offers a promising capability to counter frequency decline after large disturbances. This paper proposes an event-driven fast frequency response (EFFR) approach for the ESS to sustain frequency stability. A centralized EFFR control framework is established, in which the wide area information is collected, and the disturbance power is broadcasted to ESSs under contingencies. Moreover, a fuzzy logic controller (FLC) is employed to determine the power set-point of the ESS based on the disturbance power and state of charge (SOC). A case study is presented to validate the feasibility and effectiveness of the proposed approach.
- Open Access
Trear: Transformer-based RGB-D Egocentric Action Recognition
In this paper, we propose a \textbf{Tr}ansformer-based RGB-D \textbf{e}gocentric \textbf{a}ction \textbf{r}ecognition framework, called Trear. It consists of two modules, inter-frame attention encoder and mutual-attentional fusion block. Instead of using optical flow or recurrent units, we adopt self-attention mechanism to model the temporal structure of the data from different modalities. Input frames are cropped randomly to mitigate the effect of the data redundancy. Features from each modality are interacted through the proposed fusion block and combined through a simple yet effective fusion operation to produce a joint RGB-D representation. Empirical experiments on two large egocentric RGB-D datasets, THU-READ and FPHA, and one small dataset, WCVS, have shown that the proposed method outperforms the state-of-the-art results by a large margin.
Open AccessTrear: Transformer-based RGB-D Egocentric Action Recognition
In this paper, we propose a \textbf{Tr}ansformer-based RGB-D \textbf{e}gocentric \textbf{a}ction \textbf{r}ecognition framework, called Trear. It consists of two modules, inter-frame attention encoder and mutual-attentional fusion block. Instead of using optical flow or recurrent units, we adopt self-attention mechanism to model the temporal structure of the data from different modalities. Input frames are cropped randomly to mitigate the effect of the data redundancy. Features from each modality are interacted through the proposed fusion block and combined through a simple yet effective fusion operation to produce a joint RGB-D representation. Empirical experiments on two large egocentric RGB-D datasets, THU-READ and FPHA, and one small dataset, WCVS, have shown that the proposed method outperforms the state-of-the-art results by a large margin.
- 2 January 2021
- Published by Springer Nature in Neural Computing and Applications
- Vol. 33  (13) , 8031-8042
Transformer guided geometry model for flow-based unsupervised visual odometry
- 2 January 2021
- Published by Springer Nature in Neural Computing and Applications
- Vol. 33  (13) , 8031-8042
Transformer guided geometry model for flow-based unsupervised visual odometry