Junrong Ban
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Junrong Ban has a total of 8 co-authors. Here are the top co-authors by number of shared publications:
- Fuyang Chen
College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China
2 shared publications - Xiao Han
Nanjing University of Aeronautics and Astronautics
2 shared publications - Daoqiang Zhang
Nanjing University of Aeronautics and Astronautics
1 shared publication - Kehan Li
College of Arts, Nanjing University of Aeronautics and Astronautics, Nanjing 211100, China
1 shared publication - Lijun Sun
College of Arts, Nanjing University of Aeronautics and Astronautics, Nanjing 211100, China
1 shared publication - Liying Zhang
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing 211100, China
1 shared publication
Latest Publications
- Open Access
Neural Evidence for Implicit Processing of Rhythmic Syntax at Musical Cadences
Rhythmic syntax forms the temporal structural framework essential for musical meaning and emotion. While crucial for music perception, how rhythmic syntax is processed without direct attentional focus remains unknown. Using EEG, we examined this process by manipulating rhythmic syntactic regularity in cadence, comparing sequences ending on metrically strong positions (regular) and those with syncopated endings (irregular). Participants performed a pitch-detection task irrelevant to syntactic processing. The results showed that syntactic irregular endings failed to elicit a significant early right anterior negativity (ERAN), demonstrating the absence of automatic violation detection. In contrast, these violations robustly evoked a late P600, reflecting conscious structural processing of rhythmic syntax. Moreover, we used a deep learning method to confirm the EEG results and further validate the differential processing of regular and irregular rhythmic syntax in the brain. Our study provides direct neurophysiological evidence for the implicit processing of rhythmic syntactic violations in early and late stages.
Open AccessNeural Evidence for Implicit Processing of Rhythmic Syntax at Musical Cadences
Rhythmic syntax forms the temporal structural framework essential for musical meaning and emotion. While crucial for music perception, how rhythmic syntax is processed without direct attentional focus remains unknown. Using EEG, we examined this process by manipulating rhythmic syntactic regularity in cadence, comparing sequences ending on metrically strong positions (regular) and those with syncopated endings (irregular). Participants performed a pitch-detection task irrelevant to syntactic processing. The results showed that syntactic irregular endings failed to elicit a significant early right anterior negativity (ERAN), demonstrating the absence of automatic violation detection. In contrast, these violations robustly evoked a late P600, reflecting conscious structural processing of rhythmic syntax. Moreover, we used a deep learning method to confirm the EEG results and further validate the differential processing of regular and irregular rhythmic syntax in the brain. Our study provides direct neurophysiological evidence for the implicit processing of rhythmic syntactic violations in early and late stages.
- 7 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Fuzzy Systems
- Vol. 33 (1) , 168-179
FMFN: A Fuzzy Multimodal Fusion Network for Emotion Recognition in Ensemble Conducting
Conducting and interacting with an orchestra is a multimodal process that integrates channels such as music, visual cues, posture, and gestures to convey artistic intent accurately. For robots, discerning human emotions from these channels can enhance human-machine interactions. Currently, gesture recognition systems in orchestras focus more on rhythm, speed, and dynamics, while studying emotional factors in orchestra conducting music requires more profound research. We introduced the Facial Expression and Orchestra Gesture Emotion (FEGE) dataset, consisting of eight different emotions for recognition. This paper introduces a Fuzzy Multimodal Fusion Network (FMFN) based on fuzzy logic, which operates in multi-feature spaces and is designed for emotion recognition in bimodal tasks involving facial expressions and orchestra-conducting gestures. The network maps facial expressions and gestures into a multi-feature space through bimodal processing, learns unique and shared representations, and decodes them using classifiers optimized by FMFN parameters. Finally, it processes data uncertainty and fuzziness using a fuzzy logic system, improving the classification decision process to enhance the robustness and adaptability of emotion recognition tasks in bimodal visual modalities. Experimental results on the FEGE dataset confirmed the effectiveness of our network. The proposed bimodal fusion network achieved an accuracy of 89.16% in bimodal emotion recognition, which is approximately a 21% improvement over single-modal recognition results. This approach can also be better applied to human-machine interaction systems, particularly in orchestra conducting training, aiming to enhance the most critical emotional factors conveyed during the conducting process, thus elevating the depth of artistic intent.
- 7 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Fuzzy Systems
- Vol. 33 (1) , 168-179
FMFN: A Fuzzy Multimodal Fusion Network for Emotion Recognition in Ensemble Conducting
Conducting and interacting with an orchestra is a multimodal process that integrates channels such as music, visual cues, posture, and gestures to convey artistic intent accurately. For robots, discerning human emotions from these channels can enhance human-machine interactions. Currently, gesture recognition systems in orchestras focus more on rhythm, speed, and dynamics, while studying emotional factors in orchestra conducting music requires more profound research. We introduced the Facial Expression and Orchestra Gesture Emotion (FEGE) dataset, consisting of eight different emotions for recognition. This paper introduces a Fuzzy Multimodal Fusion Network (FMFN) based on fuzzy logic, which operates in multi-feature spaces and is designed for emotion recognition in bimodal tasks involving facial expressions and orchestra-conducting gestures. The network maps facial expressions and gestures into a multi-feature space through bimodal processing, learns unique and shared representations, and decodes them using classifiers optimized by FMFN parameters. Finally, it processes data uncertainty and fuzziness using a fuzzy logic system, improving the classification decision process to enhance the robustness and adaptability of emotion recognition tasks in bimodal visual modalities. Experimental results on the FEGE dataset confirmed the effectiveness of our network. The proposed bimodal fusion network achieved an accuracy of 89.16% in bimodal emotion recognition, which is approximately a 21% improvement over single-modal recognition results. This approach can also be better applied to human-machine interaction systems, particularly in orchestra conducting training, aiming to enhance the most critical emotional factors conveyed during the conducting process, thus elevating the depth of artistic intent.
- Top Cited PapersOpen Access
- 15 February 2023
- Published by MDPI AG in Electronics
- Vol. 12 (4) , 978
Music Emotion Recognition Based on a Neural Network with an Inception-GRU Residual Structure
As a key field in music information retrieval, music emotion recognition is indeed a challenging task. To enhance the accuracy of music emotion classification and recognition, this paper uses the idea of inception structure to use different receptive fields to extract features of different dimensions and perform compression, expansion, and recompression operations to mine more effective features and connect the timing signals in the residual network to the GRU module to extract timing features. A one-dimensional (1D) residual Convolutional Neural Network (CNN) with an improved Inception module and Gate Recurrent Unit (GRU) was presented and tested on the Soundtrack dataset. Fast Fourier Transform (FFT) was used to process the samples experimentally and determine their spectral characteristics. Compared with the shallow learning methods such as support vector machine and random forest and the deep learning method based on Visual Geometry Group (VGG) CNN proposed by Sarkar et al., the proposed deep learning method of the 1D CNN with the Inception-GRU residual structure demonstrated better performance in music emotion recognition and classification tasks, achieving an accuracy of 84%.
- 15 February 2023
- Published by MDPI AG in Electronics
- Vol. 12 (4) , 978
Top Cited PapersOpen AccessMusic Emotion Recognition Based on a Neural Network with an Inception-GRU Residual Structure
As a key field in music information retrieval, music emotion recognition is indeed a challenging task. To enhance the accuracy of music emotion classification and recognition, this paper uses the idea of inception structure to use different receptive fields to extract features of different dimensions and perform compression, expansion, and recompression operations to mine more effective features and connect the timing signals in the residual network to the GRU module to extract timing features. A one-dimensional (1D) residual Convolutional Neural Network (CNN) with an improved Inception module and Gate Recurrent Unit (GRU) was presented and tested on the Soundtrack dataset. Fast Fourier Transform (FFT) was used to process the samples experimentally and determine their spectral characteristics. Compared with the shallow learning methods such as support vector machine and random forest and the deep learning method based on Visual Geometry Group (VGG) CNN proposed by Sarkar et al., the proposed deep learning method of the 1D CNN with the Inception-GRU residual structure demonstrated better performance in music emotion recognition and classification tasks, achieving an accuracy of 84%.