Youtian Du
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Youtian Du has a total of 62 co-authors. Here are the top co-authors by number of shared publications:
Xiaohong Guan
Faculty of Electronics and Information Engineering MOE KLINNS Lab Xian Jiaotong University 710049 Xian China
10 shared publicationsFeng Chen
Department of Automation, Tsinghua University, Beijing, China
7 shared publicationsWenli Xu
Department of Automation, Tsinghua University, Beijing, China
6 shared publicationsZhongmin Cai
Xi An Jiao Tong Univ, Minist Educ, Key Lab Intelligent Networks & Network Secur, Xian 710049, Peoples R China
4 shared publications- Hang Wang
Xian Jiaotong University, Xian, China
3 shared publications - Lei Zhang
Northwestern Polytechnical University, Xi'an, China
3 shared publications
Latest Publications
- 25 August 2026
- Published by Elsevier in Expert Systems with Applications
- Vol. 333
Learning to optimize via positive momentum natural evolution strategies
- 25 August 2026
- Published by Elsevier in Expert Systems with Applications
- Vol. 333
Learning to optimize via positive momentum natural evolution strategies
- 2 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 8325-8339
IVAC-$\mathbf {P^{2}L}$: Leveraging Irregular Repetition Priors for Improving Video Action Counting
The quantification of repetitive actions in videos, a task commonly referred to as Video Action Counting (VAC), is a critical challenge in understanding and analyzing content in sports, fitness, and daily activities. Traditional approaches to VAC have largely overlooked the nuanced irregularities inherent in action repetitions, such as interruptions and variable lengths between cycles. Addressing this gap, our study introduces a novel perspective on VAC, focusing on Irregular Video Action Counting (IVAC), which emphasizes the importance of modeling the irregular repetition priors present in video content. We conceptualize these priors through two key aspects: Inter-cycle Consistency and Cycle-interval Inconsistency. Inter-cycle Consistency ensures that the spatiotemporal representations across all cycle segments in a video remain homogeneous, thereby reflecting the uniformity of actions between different cycle segments. In contrast, Cycle-interval Inconsistency mandates a clear semantic distinction between the representations of cycle segments and intervals, acknowledging the inherent dissimilarities in content. To effectively encapsulate these priors, we introduce a novel methodology consisting of consistency and inconsistency modules, underpinned by a tailored pull-push loss ($\mathrm {P^{2}~L}$) mechanism. This approach employs a pull loss to enhance the cohesion among cycle segment features and a push loss to distinctly differentiate between cycle and interval segment features. Empirical evaluations on the RepCount dataset illustrate that our IVAC-$\mathrm {P^{2}~L}$ model sets a new benchmark in state-of-the-art performance for the VAC task. Moreover, our model demonstrates adaptability and generalization across diverse video content, achieving superior performance on two additional datasets, UCFRep and Countix, without necessitating dataset-specific fine-tuning. These findings not only validate the effectiveness of our approach in addressing the complexities of irregular repetitions in videos but also open new avenues for future research in video understanding and analysis.
- 2 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 8325-8339
IVAC-$\mathbf {P^{2}L}$: Leveraging Irregular Repetition Priors for Improving Video Action Counting
The quantification of repetitive actions in videos, a task commonly referred to as Video Action Counting (VAC), is a critical challenge in understanding and analyzing content in sports, fitness, and daily activities. Traditional approaches to VAC have largely overlooked the nuanced irregularities inherent in action repetitions, such as interruptions and variable lengths between cycles. Addressing this gap, our study introduces a novel perspective on VAC, focusing on Irregular Video Action Counting (IVAC), which emphasizes the importance of modeling the irregular repetition priors present in video content. We conceptualize these priors through two key aspects: Inter-cycle Consistency and Cycle-interval Inconsistency. Inter-cycle Consistency ensures that the spatiotemporal representations across all cycle segments in a video remain homogeneous, thereby reflecting the uniformity of actions between different cycle segments. In contrast, Cycle-interval Inconsistency mandates a clear semantic distinction between the representations of cycle segments and intervals, acknowledging the inherent dissimilarities in content. To effectively encapsulate these priors, we introduce a novel methodology consisting of consistency and inconsistency modules, underpinned by a tailored pull-push loss ($\mathrm {P^{2}~L}$) mechanism. This approach employs a pull loss to enhance the cohesion among cycle segment features and a push loss to distinctly differentiate between cycle and interval segment features. Empirical evaluations on the RepCount dataset illustrate that our IVAC-$\mathrm {P^{2}~L}$ model sets a new benchmark in state-of-the-art performance for the VAC task. Moreover, our model demonstrates adaptability and generalization across diverse video content, achieving superior performance on two additional datasets, UCFRep and Countix, without necessitating dataset-specific fine-tuning. These findings not only validate the effectiveness of our approach in addressing the complexities of irregular repetitions in videos but also open new avenues for future research in video understanding and analysis.
- 7 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 33, 613-626
PiVoD: Pitch, Volume and Duration Invariant Representation for Music Tone Quality Evaluation
Tone quality is of pivotal importance in the auditory perception of musical performance. Depending on the performer and the instrument, tone quality evaluation is subjective and time-consuming, with inherent difficulties stemming from the absence of precise measurement methods. In this study, we develop a novel method for tone quality evaluation utilizing an adversarial domain-invariant learning strategy to construct a representation invariant to changes in pitch, volume, and duration. The wide-band Mel frequency cepstral coefficients are employed for pitch-invariant feature extraction and instance normalization for volume invariance. An adversarial-trained time-delay neural network encoder is developed for enhancing pitch and duration invariance via random pitch shift and temporal segmentation. Experiments conducted on our curated dataset and the Good-sound dataset show that significant improvements from the new method are achieved in evaluating tone quality ascribed to performers and instruments, yielding a 15.3% and 9.5% increase in classification accuracy, respectively, compared to classical feature-based techniques. Remarkably, the class-wise outcomes exhibit enhancements in F-scores of 33.6% and 9.8% for each respective dataset. Ablation studies on pitch, volume, and duration invariance further underscore the efficacy of our approach. This substantial enhancement over existing methods presents a novel perspective on tone quality representation and offers a practical resource for music performance analysis.
- 7 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 33, 613-626
PiVoD: Pitch, Volume and Duration Invariant Representation for Music Tone Quality Evaluation
Tone quality is of pivotal importance in the auditory perception of musical performance. Depending on the performer and the instrument, tone quality evaluation is subjective and time-consuming, with inherent difficulties stemming from the absence of precise measurement methods. In this study, we develop a novel method for tone quality evaluation utilizing an adversarial domain-invariant learning strategy to construct a representation invariant to changes in pitch, volume, and duration. The wide-band Mel frequency cepstral coefficients are employed for pitch-invariant feature extraction and instance normalization for volume invariance. An adversarial-trained time-delay neural network encoder is developed for enhancing pitch and duration invariance via random pitch shift and temporal segmentation. Experiments conducted on our curated dataset and the Good-sound dataset show that significant improvements from the new method are achieved in evaluating tone quality ascribed to performers and instruments, yielding a 15.3% and 9.5% increase in classification accuracy, respectively, compared to classical feature-based techniques. Remarkably, the class-wise outcomes exhibit enhancements in F-scores of 33.6% and 9.8% for each respective dataset. Ablation studies on pitch, volume, and duration invariance further underscore the efficacy of our approach. This substantial enhancement over existing methods presents a novel perspective on tone quality representation and offers a practical resource for music performance analysis.
- Open Access
IVAC-P2L: Leveraging Irregular Repetition Priors for Improving Video Action Counting
Video Action Counting (VAC) is crucial in analyzing sports, fitness, and everyday activities by quantifying repetitive actions in videos. However, traditional VAC methods have overlooked the complexity of action repetitions, such as interruptions and the variability in cycle duration. Our research addresses the shortfall by introducing a novel approach to VAC, called Irregular Video Action Counting (IVAC). IVAC prioritizes modeling irregular repetition patterns in videos, which we define through two primary aspects: Inter-cycle Consistency and Cycle-interval Inconsistency. Inter-cycle Consistency ensures homogeneity in the spatial-temporal representations of cycle segments, signifying action uniformity within cycles. Cycle-interval inconsistency highlights the importance of distinguishing between cycle segments and intervals based on their inherent content differences. To encapsulate these principles, we propose a new methodology that includes consistency and inconsistency modules, supported by a unique pull-push loss (P2L) mechanism. The IVAC-P2L model applies a pull loss to promote coherence among cycle segment features and a push loss to clearly distinguish features of cycle segments from interval segments. Empirical evaluations conducted on the RepCount dataset demonstrate that the IVAC-P2L model sets a new benchmark in VAC task performance. Furthermore, the model demonstrates exceptional adaptability and generalization across various video contents, outperforming existing models on two additional datasets, UCFRep and Countix, without the need for dataset-specific optimization. These results confirm the efficacy of our approach in addressing irregular repetitions in videos and pave the way for further advancements in video analysis and understanding.
Open AccessIVAC-P2L: Leveraging Irregular Repetition Priors for Improving Video Action Counting
Video Action Counting (VAC) is crucial in analyzing sports, fitness, and everyday activities by quantifying repetitive actions in videos. However, traditional VAC methods have overlooked the complexity of action repetitions, such as interruptions and the variability in cycle duration. Our research addresses the shortfall by introducing a novel approach to VAC, called Irregular Video Action Counting (IVAC). IVAC prioritizes modeling irregular repetition patterns in videos, which we define through two primary aspects: Inter-cycle Consistency and Cycle-interval Inconsistency. Inter-cycle Consistency ensures homogeneity in the spatial-temporal representations of cycle segments, signifying action uniformity within cycles. Cycle-interval inconsistency highlights the importance of distinguishing between cycle segments and intervals based on their inherent content differences. To encapsulate these principles, we propose a new methodology that includes consistency and inconsistency modules, supported by a unique pull-push loss (P2L) mechanism. The IVAC-P2L model applies a pull loss to promote coherence among cycle segment features and a push loss to clearly distinguish features of cycle segments from interval segments. Empirical evaluations conducted on the RepCount dataset demonstrate that the IVAC-P2L model sets a new benchmark in VAC task performance. Furthermore, the model demonstrates exceptional adaptability and generalization across various video contents, outperforming existing models on two additional datasets, UCFRep and Countix, without the need for dataset-specific optimization. These results confirm the efficacy of our approach in addressing irregular repetitions in videos and pave the way for further advancements in video analysis and understanding.
- 21 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 20 (4) , 5087-5099
Human–Machine Collaborative Reinforcement Learning for Power Line Flow Regulation
The complexity and uncertainty in power systems leads to a great challenge for controlling the power grid using traditional manual adjustment methods. Reinforcement learning is a promising data-driven paradigm to address control issues in power grids. This article presents a novel human–machine collaborative (HMC) framework for line flow control. We formulate the collaboration between humans and machines as an extended Markov decision process (MDP) and introduce a human–machine collaborative reinforcement learning (HMC-RL) approach, which comprises a routing module, a machine dispatching module and an HMC dispatching module. The routing module determines whether the power system should be operated by the machine or through human–machine collaboration. The machine dispatching module predicts a machine dispatching action for regulating line flow, while the HMC module predicts an HMC dispatching action with human assistance. Experimental results conducted on the IEEE 39-bus and IEEE 118-bus systems demonstrate that our HMC-RL approach can significantly improve the performance of regulation compared to the machine dispatching policy. Specifically, HMC-RL achieves a 40.03% performance improvement on the IEEE 118-bus system, with 25.8% of human participation.
- 21 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 20 (4) , 5087-5099
Human–Machine Collaborative Reinforcement Learning for Power Line Flow Regulation
The complexity and uncertainty in power systems leads to a great challenge for controlling the power grid using traditional manual adjustment methods. Reinforcement learning is a promising data-driven paradigm to address control issues in power grids. This article presents a novel human–machine collaborative (HMC) framework for line flow control. We formulate the collaboration between humans and machines as an extended Markov decision process (MDP) and introduce a human–machine collaborative reinforcement learning (HMC-RL) approach, which comprises a routing module, a machine dispatching module and an HMC dispatching module. The routing module determines whether the power system should be operated by the machine or through human–machine collaboration. The machine dispatching module predicts a machine dispatching action for regulating line flow, while the HMC module predicts an HMC dispatching action with human assistance. Experimental results conducted on the IEEE 39-bus and IEEE 118-bus systems demonstrate that our HMC-RL approach can significantly improve the performance of regulation compared to the machine dispatching policy. Specifically, HMC-RL achieves a 40.03% performance improvement on the IEEE 118-bus system, with 25.8% of human participation.
- 8 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 32 (10577149) , 190-202
One-Stage Visual Relationship Referring With Transformers and Adaptive Message Passing
There exist a variety of visual relationships among entities in an image. Given a relationship query $\langle subject, predicate, object \rangle $ , the task of visual relationship referring (VRR) aims to disambiguate instances of the same entity category and simultaneously localize the subject and object entities in an image. Previous works of VRR can be generally categorized into one-stage and multi-stage methods. The former ones directly localize a pair of entities from the image but they suffer from low prediction accuracy, while the latter ones perform better but they are indirect to localize only a couple of entities by pre-generating a rich amount of candidate proposals. In this paper, we formulate the task of VRR as an end-to-end bounding box regression problem and propose a novel one-stage approach, called VRR-TAMP, by effectively integrating Transformers and an adaptive message passing mechanism. First, visual relationship queries and images are respectively encoded to generate the basic modality-specific embeddings, which are then fed into a cross-modal Transformer encoder to produce the joint representation. Second, to obtain the specific representation of each entity, we introduce an adaptive message passing mechanism and design an entity-specific information distiller SR-GMP, which refers to a gated message passing (GMP) module that works on the joint representation learned from a single learnable token. The GMP module adaptively distills the final representation of an entity by incorporating the contextual cues regarding the predicate and the other entity. Experiments on VRD and Visual Genome datasets demonstrate that our approach significantly outperforms its one-stage competitors and achieves competitive results with the state-of-the-art multi-stage methods.
- 8 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Image Processing
- Vol. 32 (10577149) , 190-202
One-Stage Visual Relationship Referring With Transformers and Adaptive Message Passing
There exist a variety of visual relationships among entities in an image. Given a relationship query $\langle subject, predicate, object \rangle $ , the task of visual relationship referring (VRR) aims to disambiguate instances of the same entity category and simultaneously localize the subject and object entities in an image. Previous works of VRR can be generally categorized into one-stage and multi-stage methods. The former ones directly localize a pair of entities from the image but they suffer from low prediction accuracy, while the latter ones perform better but they are indirect to localize only a couple of entities by pre-generating a rich amount of candidate proposals. In this paper, we formulate the task of VRR as an end-to-end bounding box regression problem and propose a novel one-stage approach, called VRR-TAMP, by effectively integrating Transformers and an adaptive message passing mechanism. First, visual relationship queries and images are respectively encoded to generate the basic modality-specific embeddings, which are then fed into a cross-modal Transformer encoder to produce the joint representation. Second, to obtain the specific representation of each entity, we introduce an adaptive message passing mechanism and design an entity-specific information distiller SR-GMP, which refers to a gated message passing (GMP) module that works on the joint representation learned from a single learnable token. The GMP module adaptively distills the final representation of an entity by incorporating the contextual cues regarding the predicate and the other entity. Experiments on VRD and Visual Genome datasets demonstrate that our approach significantly outperforms its one-stage competitors and achieves competitive results with the state-of-the-art multi-stage methods.
- 2 November 2022
- Published by Springer Nature in Applied Intelligence
- Vol. 53 (11) , 14690-14702
Improving weakly supervised phrase grounding via visual representation contextualization with contrastive learning
- 2 November 2022
- Published by Springer Nature in Applied Intelligence
- Vol. 53 (11) , 14690-14702
Improving weakly supervised phrase grounding via visual representation contextualization with contrastive learning
- 6 August 2022
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 82 (5) , 6557-6579
Fine-grained label learning in object detection with weak supervision of captions
- 6 August 2022
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 82 (5) , 6557-6579
Fine-grained label learning in object detection with weak supervision of captions
- 1 November 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 121-122
Learning More Complex Actions with Deep Reinforcement Learning
How to learn more complex actions based on the predefined atomic actions is a significant problem for improving the intelligence of machines. In this paper, we present a novel approach called action space expanding Q-learning (ASE-Q) to the learning of more complex actions. We first introduce two types of formation of the complex actions; then, we design an action combination network to dynamically learn the complex actions a Q-network to evaluate action-value of the complex actions. Learning more complex actions can speed up the process that an agent accomplishes a task and improve its capacity in learning new actions.
- 1 November 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 121-122
Learning More Complex Actions with Deep Reinforcement Learning
How to learn more complex actions based on the predefined atomic actions is a significant problem for improving the intelligence of machines. In this paper, we present a novel approach called action space expanding Q-learning (ASE-Q) to the learning of more complex actions. We first introduce two types of formation of the complex actions; then, we design an action combination network to dynamically learn the complex actions a Q-network to evaluate action-value of the complex actions. Learning more complex actions can speed up the process that an agent accomplishes a task and improve its capacity in learning new actions.
- Open Access
- 12 June 2021
- Published by Institution of Engineering and Technology (IET) in IET Image Processing
- Vol. 15 (12) , 2937-2947
Leveraging attention‐based visual clue extraction for image classification
Deep learning‐based approaches have made considerable progress in image classification tasks, but most of the approaches lack interpretability, especially in revealing the decisive information causing the categorization of images. This paper seeks to answer the question of what clues encode the discriminative visual information between image categories and can help improve the classification performance. To this end, an attention‐based clue extraction network (ACENet) is introduced to mine the decisive local visual information for image classification. ACENet constructs a clue‐attention mechanism, that is global‐local attention, between the image and visual clue proposals extracted from it and then introduces a contrastive loss defined over the achieved discrete attention distribution to increase the discriminability of clue proposals. The loss encourages considerable attention to be devoted to discriminative clue proposals, that is those similar within the same category and dissimilar across categories. The experimental results for the Negative Web Image (NWI) dataset and the public ImageNet2012 dataset demonstrate that ACENet can extract true clues to improve the image classification performance and outperforms the baselines and the state‐of‐the‐art methods.
- 12 June 2021
- Published by Institution of Engineering and Technology (IET) in IET Image Processing
- Vol. 15 (12) , 2937-2947
Open AccessLeveraging attention‐based visual clue extraction for image classification
Deep learning‐based approaches have made considerable progress in image classification tasks, but most of the approaches lack interpretability, especially in revealing the decisive information causing the categorization of images. This paper seeks to answer the question of what clues encode the discriminative visual information between image categories and can help improve the classification performance. To this end, an attention‐based clue extraction network (ACENet) is introduced to mine the decisive local visual information for image classification. ACENet constructs a clue‐attention mechanism, that is global‐local attention, between the image and visual clue proposals extracted from it and then introduces a contrastive loss defined over the achieved discrete attention distribution to increase the discriminability of clue proposals. The loss encourages considerable attention to be devoted to discriminative clue proposals, that is those similar within the same category and dissimilar across categories. The experimental results for the Negative Web Image (NWI) dataset and the public ImageNet2012 dataset demonstrate that ACENet can extract true clues to improve the image classification performance and outperforms the baselines and the state‐of‐the‐art methods.