Peiyang Yu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Peiyang Yu has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Feng Yu
Central Conservatory of Music, Beijing, China
2 shared publications - Song-Chun Zhu
Peking University
2 shared publications - Bo Zheng
Alibaba Group, Hangzhou, China
1 shared publication - Duo Xu
Tianjin Conservatory of Music, Tianjin, China
1 shared publication - Hui Qv
Intelligent Science and Technology Academy of CASIC, Beijing, China
1 shared publication - Jishang Chen
Music Information Technology and Musical Artificial Intelligence, Central Conservatory of Music, Beijing, China
1 shared publication
Latest Publications
- 27 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Social Systems
- Vol. 12 (2) , 873-889
MusicAOG: An Energy-Based Model for Learning and Sampling a Hierarchical Representation of Symbolic Music
In addressing the challenge of interpretability and generalizability of artificial music intelligence, this article introduces a novel symbolic representation that amalgamates both explicit and implicit musical information across diverse traditions and granularities. Utilizing a hierarchical and-or graph representation, the model employs nodes and edges to encapsulate a broad spectrum of musical elements, including structures, textures, rhythms, and harmonies. This hierarchical approach expands the representability across various scales of music. This representation serves as the foundation for an energy-based model, uniquely tailored to learn musical concepts through a flexible algorithm framework relying on the minimax entropy principle. Utilizing an adapted Metropolis–Hastings sampling technique, the model enables fine-grained control over music generation. Through a comprehensive empirical evaluation, this novel approach demonstrates significant improvements in interpretability and controllability compared to existing methodologies. This study marks a substantial contribution to the fields of music analysis, composition, and computational musicology.
- 27 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Social Systems
- Vol. 12 (2) , 873-889
MusicAOG: An Energy-Based Model for Learning and Sampling a Hierarchical Representation of Symbolic Music
In addressing the challenge of interpretability and generalizability of artificial music intelligence, this article introduces a novel symbolic representation that amalgamates both explicit and implicit musical information across diverse traditions and granularities. Utilizing a hierarchical and-or graph representation, the model employs nodes and edges to encapsulate a broad spectrum of musical elements, including structures, textures, rhythms, and harmonies. This hierarchical approach expands the representability across various scales of music. This representation serves as the foundation for an energy-based model, uniquely tailored to learn musical concepts through a flexible algorithm framework relying on the minimax entropy principle. Utilizing an adapted Metropolis–Hastings sampling technique, the model enables fine-grained control over music generation. Through a comprehensive empirical evaluation, this novel approach demonstrates significant improvements in interpretability and controllability compared to existing methodologies. This study marks a substantial contribution to the fields of music analysis, composition, and computational musicology.
- 1 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Social Systems
- Vol. 12 (2) , 905-917
Video Echoed in Harmony: Learning and Sampling Video-Integrated Chord Progression Sequences for Controllable Video Background Music Generation
Automatically generating video background music mitigates the inefficiency and time-consuming drawbacks of current manual video editing. Two key challenges hinder the expansion of the inception of video-to-music tasks. 1) Limited availability of high-quality video–music datasets and annotations. 2) Absence of music generation methods that consider actual musicality, which are controlled by interpretable factors based on music theory. In the article, we propose video echoed in harmony (VEH), a method for learning and sampling video-integrated chord progression sequences. Our approach adopts harmony, represented by chord progressions that are aligned with various music formats [musical instrument digital interface (MIDI), audio, and score], imitating chord precedence in human music composition. Visual-language models link visual features to chord progressions through genre labels and descriptive words in generated textualized videos. The two aforementioned features collectively obviate the necessity of extensive video–music paired data. Besides, an energy-based chord progression learning and sampling algorithm quantifies abstract harmony impressions to statistical features, serving as interpretable factors for the controllable music generation based on music theory. Experimental results demonstrate that the proposed method outperforms the state-of-the-art, producing a superior music alignment for the given video.
- 1 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Social Systems
- Vol. 12 (2) , 905-917
Video Echoed in Harmony: Learning and Sampling Video-Integrated Chord Progression Sequences for Controllable Video Background Music Generation
Automatically generating video background music mitigates the inefficiency and time-consuming drawbacks of current manual video editing. Two key challenges hinder the expansion of the inception of video-to-music tasks. 1) Limited availability of high-quality video–music datasets and annotations. 2) Absence of music generation methods that consider actual musicality, which are controlled by interpretable factors based on music theory. In the article, we propose video echoed in harmony (VEH), a method for learning and sampling video-integrated chord progression sequences. Our approach adopts harmony, represented by chord progressions that are aligned with various music formats [musical instrument digital interface (MIDI), audio, and score], imitating chord precedence in human music composition. Visual-language models link visual features to chord progressions through genre labels and descriptive words in generated textualized videos. The two aforementioned features collectively obviate the necessity of extensive video–music paired data. Besides, an energy-based chord progression learning and sampling algorithm quantifies abstract harmony impressions to statistical features, serving as interpretable factors for the controllable music generation based on music theory. Experimental results demonstrate that the proposed method outperforms the state-of-the-art, producing a superior music alignment for the given video.