Yuguang Yang
Yuguang Yang has a total of 39 co-authors. Here are the top co-authors by number of shared publications:
- Yue Wu
Key Laboratory of All Optical Network and Advanced Telecommunication Network of EMC, Institute of Lightwave Technology, Beijing Jiaotong University, Beijing, China
4 shared publications - Hongbin Zhou
Ximalaya Inc China
3 shared publications - Jianhao Ye
Ximalaya Inc China
3 shared publications - Wenxing Jin
Key Laboratory of All Optical Network and Advanced Telecommunication Network of EMC, Institute of Lightwave Technology, Beijing Jiaotong University, Beijing, China
3 shared publications - Ya Shen
Key Laboratory of All Optical Network and Advanced Telecommunication Network of EMC, Institute of Lightwave Technology, Beijing Jiaotong University, Beijing, China
3 shared publications - Guobin Ren
Beijing Jiaotong University
2 shared publications
Latest Publications
- Top Cited PapersOpen Access
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (24) , 25669-25677
StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matching
Zero-shot voice conversion (VC) aims to transfer the timbre from the source speaker to an arbitrary unseen speaker while preserving the original linguistic content. Despite recent advancements in zero-shot VC using language model-based or diffusion-based approaches, several challenges remain: 1) current approaches primarily focus on adapting timbre from unseen speakers and are unable to transfer style and timbre to different unseen speakers independently; 2) these approaches often suffer from slower inference speeds due to the autoregressive modeling methods or the need for numerous sampling steps; 3) the quality and similarity of the converted samples are still not fully satisfactory. To address these challenges, we propose a Style controllable zero-shot VC approach named StableVC, which aims to transfer timbre and style from source speech to different unseen target speakers. Specifically, we decompose speech into linguistic content, timbre, and style, and then employ a conditional flow matching module to reconstruct the high-quality mel-spectrogram based on these decomposed features. To effectively capture timbre and style in a zero-shot manner, we introduce a novel dual attention mechanism with an adaptive gate, rather than using conventional feature concatenation. With this non-autoregressive design, StableVC can efficiently capture the intricate timbre and style from different unseen speakers and generate high-quality speech significantly faster than real-time. Experiments demonstrate that our proposed StableVC outperforms state-of-the-art baseline systems in zero-shot VC and achieves flexible control over timbre and style from different unseen speakers. Moreover, StableVC offers approximately 25x and 1.65x faster sampling compared to autoregressive and diffusion-based baselines.
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (24) , 25669-25677
Top Cited PapersOpen AccessStableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matching
Zero-shot voice conversion (VC) aims to transfer the timbre from the source speaker to an arbitrary unseen speaker while preserving the original linguistic content. Despite recent advancements in zero-shot VC using language model-based or diffusion-based approaches, several challenges remain: 1) current approaches primarily focus on adapting timbre from unseen speakers and are unable to transfer style and timbre to different unseen speakers independently; 2) these approaches often suffer from slower inference speeds due to the autoregressive modeling methods or the need for numerous sampling steps; 3) the quality and similarity of the converted samples are still not fully satisfactory. To address these challenges, we propose a Style controllable zero-shot VC approach named StableVC, which aims to transfer timbre and style from source speech to different unseen target speakers. Specifically, we decompose speech into linguistic content, timbre, and style, and then employ a conditional flow matching module to reconstruct the high-quality mel-spectrogram based on these decomposed features. To effectively capture timbre and style in a zero-shot manner, we introduce a novel dual attention mechanism with an adaptive gate, rather than using conventional feature concatenation. With this non-autoregressive design, StableVC can efficiently capture the intricate timbre and style from different unseen speakers and generate high-quality speech significantly faster than real-time. Experiments demonstrate that our proposed StableVC outperforms state-of-the-art baseline systems in zero-shot VC and achieves flexible control over timbre and style from different unseen speakers. Moreover, StableVC offers approximately 25x and 1.65x faster sampling compared to autoregressive and diffusion-based baselines.
- Open Access
StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matching
Zero-shot voice conversion (VC) aims to transfer the timbre from the source speaker to an arbitrary unseen speaker while preserving the original linguistic content. Despite recent advancements in zero-shot VC using language model-based or diffusion-based approaches, several challenges remain: 1) current approaches primarily focus on adapting timbre from unseen speakers and are unable to transfer style and timbre to different unseen speakers independently; 2) these approaches often suffer from slower inference speeds due to the autoregressive modeling methods or the need for numerous sampling steps; 3) the quality and similarity of the converted samples are still not fully satisfactory. To address these challenges, we propose a style controllable zero-shot VC approach named StableVC, which aims to transfer timbre and style from source speech to different unseen target speakers. Specifically, we decompose speech into linguistic content, timbre, and style, and then employ a conditional flow matching module to reconstruct the high-quality mel-spectrogram based on these decomposed features. To effectively capture timbre and style in a zero-shot manner, we introduce a novel dual attention mechanism with an adaptive gate, rather than using conventional feature concatenation. With this non-autoregressive design, StableVC can efficiently capture the intricate timbre and style from different unseen speakers and generate high-quality speech significantly faster than real-time. Experiments demonstrate that our proposed StableVC outperforms state-of-the-art baseline systems in zero-shot VC and achieves flexible control over timbre and style from different unseen speakers. Moreover, StableVC offers approximately 25x and 1.65x faster sampling compared to autoregressive and diffusion-based baselines.
Open AccessStableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matching
Zero-shot voice conversion (VC) aims to transfer the timbre from the source speaker to an arbitrary unseen speaker while preserving the original linguistic content. Despite recent advancements in zero-shot VC using language model-based or diffusion-based approaches, several challenges remain: 1) current approaches primarily focus on adapting timbre from unseen speakers and are unable to transfer style and timbre to different unseen speakers independently; 2) these approaches often suffer from slower inference speeds due to the autoregressive modeling methods or the need for numerous sampling steps; 3) the quality and similarity of the converted samples are still not fully satisfactory. To address these challenges, we propose a style controllable zero-shot VC approach named StableVC, which aims to transfer timbre and style from source speech to different unseen target speakers. Specifically, we decompose speech into linguistic content, timbre, and style, and then employ a conditional flow matching module to reconstruct the high-quality mel-spectrogram based on these decomposed features. To effectively capture timbre and style in a zero-shot manner, we introduce a novel dual attention mechanism with an adaptive gate, rather than using conventional feature concatenation. With this non-autoregressive design, StableVC can efficiently capture the intricate timbre and style from different unseen speakers and generate high-quality speech significantly faster than real-time. Experiments demonstrate that our proposed StableVC outperforms state-of-the-art baseline systems in zero-shot VC and achieves flexible control over timbre and style from different unseen speakers. Moreover, StableVC offers approximately 25x and 1.65x faster sampling compared to autoregressive and diffusion-based baselines.
- Open Access
Takin-ADA: Emotion Controllable Audio-Driven Animation with Canonical and Landmark Loss Optimization
Existing audio-driven facial animation methods face critical challenges, including expression leakage, ineffective subtle expression transfer, and imprecise audio-driven synchronization. We discovered that these issues stem from limitations in motion representation and the lack of fine-grained control over facial expressions. To address these problems, we present Takin-ADA, a novel two-stage approach for real-time audio-driven portrait animation. In the first stage, we introduce a specialized loss function that enhances subtle expression transfer while reducing unwanted expression leakage. The second stage utilizes an advanced audio processing technique to improve lip-sync accuracy. Our method not only generates precise lip movements but also allows flexible control over facial expressions and head motions. Takin-ADA achieves high-resolution (512x512) facial animations at up to 42 FPS on an RTX 4090 GPU, outperforming existing commercial solutions. Extensive experiments demonstrate that our model significantly surpasses previous methods in video quality, facial dynamics realism, and natural head movements, setting a new benchmark in the field of audio-driven facial animation.
Open AccessTakin-ADA: Emotion Controllable Audio-Driven Animation with Canonical and Landmark Loss Optimization
Existing audio-driven facial animation methods face critical challenges, including expression leakage, ineffective subtle expression transfer, and imprecise audio-driven synchronization. We discovered that these issues stem from limitations in motion representation and the lack of fine-grained control over facial expressions. To address these problems, we present Takin-ADA, a novel two-stage approach for real-time audio-driven portrait animation. In the first stage, we introduce a specialized loss function that enhances subtle expression transfer while reducing unwanted expression leakage. The second stage utilizes an advanced audio processing technique to improve lip-sync accuracy. Our method not only generates precise lip movements but also allows flexible control over facial expressions and head motions. Takin-ADA achieves high-resolution (512x512) facial animations at up to 42 FPS on an RTX 4090 GPU, outperforming existing commercial solutions. Extensive experiments demonstrate that our model significantly surpasses previous methods in video quality, facial dynamics realism, and natural head movements, setting a new benchmark in the field of audio-driven facial animation.
- Open Access
GEmo-CLAP: Gender-Attribute-Enhanced Contrastive Language-Audio Pretraining for Accurate Speech Emotion Recognition
Contrastive cross-modality pretraining has recently exhibited impressive success in diverse fields, whereas there is limited research on their merits in speech emotion recognition (SER). In this paper, we propose GEmo-CLAP, a kind of gender-attribute-enhanced contrastive language-audio pretraining (CLAP) method for SER. Specifically, we first construct an effective emotion CLAP (Emo-CLAP) for SER, using pre-trained text and audio encoders. Second, given the significance of gender information in SER, two novel multi-task learning based GEmo-CLAP (ML-GEmo-CLAP) and soft label based GEmo-CLAP (SL-GEmo-CLAP) models are further proposed to incorporate gender information of speech signals, forming more reasonable objectives. Experiments on IEMOCAP indicate that our proposed two GEmo-CLAPs consistently outperform Emo-CLAP with different pre-trained models. Remarkably, the proposed WavLM-based SL-GEmo-CLAP obtains the best WAR of 83.16\%, which performs better than state-of-the-art SER methods.
Open AccessGEmo-CLAP: Gender-Attribute-Enhanced Contrastive Language-Audio Pretraining for Accurate Speech Emotion Recognition
Contrastive cross-modality pretraining has recently exhibited impressive success in diverse fields, whereas there is limited research on their merits in speech emotion recognition (SER). In this paper, we propose GEmo-CLAP, a kind of gender-attribute-enhanced contrastive language-audio pretraining (CLAP) method for SER. Specifically, we first construct an effective emotion CLAP (Emo-CLAP) for SER, using pre-trained text and audio encoders. Second, given the significance of gender information in SER, two novel multi-task learning based GEmo-CLAP (ML-GEmo-CLAP) and soft label based GEmo-CLAP (SL-GEmo-CLAP) models are further proposed to incorporate gender information of speech signals, forming more reasonable objectives. Experiments on IEMOCAP indicate that our proposed two GEmo-CLAPs consistently outperform Emo-CLAP with different pre-trained models. Remarkably, the proposed WavLM-based SL-GEmo-CLAP obtains the best WAR of 83.16\%, which performs better than state-of-the-art SER methods.
- Open Access
HYBRIDFORMER: improving SqueezeFormer with hybrid attention and NSR mechanism
SqueezeFormer has recently shown impressive performance in automatic speech recognition (ASR). However, its inference speed suffers from the quadratic complexity of softmax-attention (SA). In addition, limited by the large convolution kernel size, the local modeling ability of SqueezeFormer is insufficient. In this paper, we propose a novel method HybridFormer to improve SqueezeFormer in a fast and efficient way. Specifically, we first incorporate linear attention (LA) and propose a hybrid LASA paradigm to increase the model's inference speed. Second, a hybrid neural architecture search (NAS) guided structural re-parameterization (SRep) mechanism, termed NSR, is proposed to enhance the ability of the model to extract local interactions. Extensive experiments conducted on the LibriSpeech dataset demonstrate that our proposed HybridFormer can achieve a 9.1% relative word error rate (WER) reduction over SqueezeFormer on the test-other dataset. Furthermore, when input speech is 30s, the HybridFormer can improve the model's inference speed up to 18%. Our source code is available online.
Open AccessHYBRIDFORMER: improving SqueezeFormer with hybrid attention and NSR mechanism
SqueezeFormer has recently shown impressive performance in automatic speech recognition (ASR). However, its inference speed suffers from the quadratic complexity of softmax-attention (SA). In addition, limited by the large convolution kernel size, the local modeling ability of SqueezeFormer is insufficient. In this paper, we propose a novel method HybridFormer to improve SqueezeFormer in a fast and efficient way. Specifically, we first incorporate linear attention (LA) and propose a hybrid LASA paradigm to increase the model's inference speed. Second, a hybrid neural architecture search (NAS) guided structural re-parameterization (SRep) mechanism, termed NSR, is proposed to enhance the ability of the model to extract local interactions. Extensive experiments conducted on the LibriSpeech dataset demonstrate that our proposed HybridFormer can achieve a 9.1% relative word error rate (WER) reduction over SqueezeFormer on the test-other dataset. Furthermore, when input speech is 30s, the HybridFormer can improve the model's inference speed up to 18%. Our source code is available online.
- Open Access
- 28 November 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Photonics Journal
- Vol. 10 (1) , 1-9
Refractive Index and Temperature Sensing Based on an Optoelectronic Oscillator Incorporating a Fabry–Perot Fiber Bragg Grating
A novel high-speed and high-sensitivity refractive index (RI) and temperature sensing based on an optoelectronic oscillator (OEO) is proposed and demonstrated. A novel Fabry-Perot fiber Bragg grating (FP-FBG), incorporated in the OEO to implement a microwave photonic filter, has a narrow notch in the reflection spectrum and combines a gap with two FBGs in a silica V-sharp slot. A microwave signal is generated by the OEO whose oscillating frequency is determined by the center frequency of the FP-FBG notch. The RI or the temperature change can be measured by monitoring the frequency shift of the microwave signal using an electrical spectrum analyzer or a digital signal processor. An experiment is performed to verify the proposal. The sensing sensitivities of the RI and temperature as high as 413.8 MHz/0.001RIU and 2516 MHz/°C are demonstrated respectively.
- 28 November 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Photonics Journal
- Vol. 10 (1) , 1-9
Open AccessRefractive Index and Temperature Sensing Based on an Optoelectronic Oscillator Incorporating a Fabry–Perot Fiber Bragg Grating
A novel high-speed and high-sensitivity refractive index (RI) and temperature sensing based on an optoelectronic oscillator (OEO) is proposed and demonstrated. A novel Fabry-Perot fiber Bragg grating (FP-FBG), incorporated in the OEO to implement a microwave photonic filter, has a narrow notch in the reflection spectrum and combines a gap with two FBGs in a silica V-sharp slot. A microwave signal is generated by the OEO whose oscillating frequency is determined by the center frequency of the FP-FBG notch. The RI or the temperature change can be measured by monitoring the frequency shift of the microwave signal using an electrical spectrum analyzer or a digital signal processor. An experiment is performed to verify the proposal. The sensing sensitivities of the RI and temperature as high as 413.8 MHz/0.001RIU and 2516 MHz/°C are demonstrated respectively.
- 2 October 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Photonics Technology Letters
- Vol. 29 (23) , 2008-2011
Switchable Optoelectronic Oscillator Using an FM-PS-FBG for Strain and Temperature Sensing
By incorporating a phase-shifted fiber Bragg grating (PS-FBG) inscribed in few-mode fiber (FMF) named FM-PS-FBG for the first time, a novel frequency-switchable optoelectronic oscillator (OEO) to perform simultaneous strain and temperature measurement is proposed and experimentally demonstrated. In the OEO loop, two switchable oscillation frequencies are determined by the reflected bands of FM-PS-FBG based on different mode couplings and corresponding notches. Due to the characteristics of modes, two OEO-generated microwave signals exhibit different strain and temperature sensitivities. Therefore, through monitoring the frequency shifts of microwave signals, the simultaneous strain and temperature sensing can be realized by the proposed OEO sensor with high-speed and high-resolution interrogation.
- 2 October 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Photonics Technology Letters
- Vol. 29 (23) , 2008-2011
Switchable Optoelectronic Oscillator Using an FM-PS-FBG for Strain and Temperature Sensing
By incorporating a phase-shifted fiber Bragg grating (PS-FBG) inscribed in few-mode fiber (FMF) named FM-PS-FBG for the first time, a novel frequency-switchable optoelectronic oscillator (OEO) to perform simultaneous strain and temperature measurement is proposed and experimentally demonstrated. In the OEO loop, two switchable oscillation frequencies are determined by the reflected bands of FM-PS-FBG based on different mode couplings and corresponding notches. Due to the characteristics of modes, two OEO-generated microwave signals exhibit different strain and temperature sensitivities. Therefore, through monitoring the frequency shifts of microwave signals, the simultaneous strain and temperature sensing can be realized by the proposed OEO sensor with high-speed and high-resolution interrogation.
- 21 August 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Photonics Technology Letters
- Vol. 29 (19) , 1659-1662
Generation of the Tunable Second-Order Optical Vortex Beams in Narrow Linewidth Fiber Laser
We experimentally demonstrate a narrow linewidth single-longitudinal-mode erbium-doped fiber ring laser with an output of tunable second-order optical vortex beams (OVB). The laser is based on a mode selective all-fiber fused coupler, which is composed of a single-mode fiber (SMF) and a few-mode fiber (FMF). The fused SMF-FMF coupler inserted in the cavity of the laser acts as a mode converter, which converts LP01 mode to LP21 mode group. The tunable OVB is produced by combing two even LP21 modes with orthogonal polarization directions. The tunability of second order OVB in the fiber laser is experimentally realized by adjusting a polarizer at the output end of the fiber. The 20 dB linewidth of the fiber laser is approximate 7.2 kHz.
- 21 August 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Photonics Technology Letters
- Vol. 29 (19) , 1659-1662
Generation of the Tunable Second-Order Optical Vortex Beams in Narrow Linewidth Fiber Laser
We experimentally demonstrate a narrow linewidth single-longitudinal-mode erbium-doped fiber ring laser with an output of tunable second-order optical vortex beams (OVB). The laser is based on a mode selective all-fiber fused coupler, which is composed of a single-mode fiber (SMF) and a few-mode fiber (FMF). The fused SMF-FMF coupler inserted in the cavity of the laser acts as a mode converter, which converts LP01 mode to LP21 mode group. The tunable OVB is produced by combing two even LP21 modes with orthogonal polarization directions. The tunability of second order OVB in the fiber laser is experimentally realized by adjusting a polarizer at the output end of the fiber. The 20 dB linewidth of the fiber laser is approximate 7.2 kHz.
- 1 May 2016
- Published by Elsevier in Optics Communications
- Vol. 366, 308-313
Strict dual-mode large-mode-area fiber with multicore structure
- 1 May 2016
- Published by Elsevier in Optics Communications
- Vol. 366, 308-313
Strict dual-mode large-mode-area fiber with multicore structure