You Zuo
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
You Zuo has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
- Chengzhen Meng
University of Science and Technology of China, Hefei, China
2 shared publications - Chenming He
University of Science and Technology of China, Hefei, China
2 shared publications - Dequan Wang
University of Science and Technology of China, Hefei, China
2 shared publications - Feiyu Han
Nanjing University of Information Science and Technology
2 shared publications - Lingyu Wang
University of Science and Technology of China, Hefei, China
2 shared publications - Panlong Yang
Nanjing University of Information Science and Technology
2 shared publications
Latest Publications
- Open Access
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-24
FeelWave: Enabling Emotion-Aware Voice Interaction through Noise-Robust mmWave Emotion Sensing
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-24
Open AccessFeelWave: Enabling Emotion-Aware Voice Interaction through Noise-Robust mmWave Emotion Sensing
- Open Access
- 2 December 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 9 (4) , 1-33
mmMUSE: An mmWave-based Motion-resilient Universal Speech Enhancement System
Speech enhancement improves the user interaction experience in voice-based smart systems. While microphone-based speech perception is limited by airborne noise, mmWave is immune to such interference. However, user-device motion hinders mmWave-based vocal extraction, dispersing vocal signals and introducing distortions. In this paper, we propose mmMUSE , an mmWave-based motion-resilient universal speech enhancement system that integrates mmWave and audio. To mitigate motion interference, we propose a two-stage method for robust vocal vibration extraction. Moreover, by proposing the Vocal-Noise-Ratio metric to assess the prominence of the vocal vibration, we enable real-time voice activity detection. We also design a complex-valued network that includes an attention-based fusion network for cross-modal complementing and a time-frequency masking network for correcting amplitude and phase of speech to isolate noises. Using datasets from 46 participants, mmMUSE outperforms the state-of-the-art speech enhancement models by 26% in SISDR and 34% in STOI on average. It also achieves SISDR improvements of 16.72 dB, 17.93 dB, 14.93 dB, and 18.95 dB in controlled environments involving intense noise, extensive motion, multiple speakers, and various obstructive materials, respectively. Finally, in real-world scenarios, including running, public spaces, and driving, mmMUSE achieves WER below 10%.
- 2 December 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 9 (4) , 1-33
Open AccessmmMUSE: An mmWave-based Motion-resilient Universal Speech Enhancement System
Speech enhancement improves the user interaction experience in voice-based smart systems. While microphone-based speech perception is limited by airborne noise, mmWave is immune to such interference. However, user-device motion hinders mmWave-based vocal extraction, dispersing vocal signals and introducing distortions. In this paper, we propose mmMUSE , an mmWave-based motion-resilient universal speech enhancement system that integrates mmWave and audio. To mitigate motion interference, we propose a two-stage method for robust vocal vibration extraction. Moreover, by proposing the Vocal-Noise-Ratio metric to assess the prominence of the vocal vibration, we enable real-time voice activity detection. We also design a complex-valued network that includes an attention-based fusion network for cross-modal complementing and a time-frequency masking network for correcting amplitude and phase of speech to isolate noises. Using datasets from 46 participants, mmMUSE outperforms the state-of-the-art speech enhancement models by 26% in SISDR and 34% in STOI on average. It also achieves SISDR improvements of 16.72 dB, 17.93 dB, 14.93 dB, and 18.95 dB in controlled environments involving intense noise, extensive motion, multiple speakers, and various obstructive materials, respectively. Finally, in real-world scenarios, including running, public spaces, and driving, mmMUSE achieves WER below 10%.
- 16 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 12 (12) , 18651-18664
EarOE: Enabling Body-Channel Voice Interaction Interface on Earphones via Occlusion Effect
Nowadays, voice input on earphones has become one of the most paramount human-computer interaction approaches. Traditional voice interaction is built on the air channel, which is highly noise-susceptible and suffers from being falsely triggered by nearby competing users. In our work, we design a noise-resistant voice interaction interface on earphones, named EAROE, which takes advantage of the narrow-bandwidth body channel to reconstruct high-fidelity audible speech. Although promising, directly taking the body channel as a voice interaction interface is non-trivial since the limited bandwidth of body-channel speech causes the original timbre information and linguistic content to be lost. To address these issues, we employ an electro-acoustic (EA) model for occlusion effect-based cross-channel correlation analysis. Based on that, we carefully design an attention-based encoder-decoder network to embrace cross-channel correlation for high-quality wide-bandwidth spectrum synthesis. To accommodate individual differences and improve model generalization, we implement a physics-based data augmentation strategy to expand the scale of the training dataset. Through extensive real-world experiments with 28 participants, EAROE can achieve an average Mel-cepstral distance of 8.08, an average modulation spectra distance of 0.82, and an average log-spectral distance of 11.16, outperforming existing solutions.
- 16 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 12 (12) , 18651-18664
EarOE: Enabling Body-Channel Voice Interaction Interface on Earphones via Occlusion Effect
Nowadays, voice input on earphones has become one of the most paramount human-computer interaction approaches. Traditional voice interaction is built on the air channel, which is highly noise-susceptible and suffers from being falsely triggered by nearby competing users. In our work, we design a noise-resistant voice interaction interface on earphones, named EAROE, which takes advantage of the narrow-bandwidth body channel to reconstruct high-fidelity audible speech. Although promising, directly taking the body channel as a voice interaction interface is non-trivial since the limited bandwidth of body-channel speech causes the original timbre information and linguistic content to be lost. To address these issues, we employ an electro-acoustic (EA) model for occlusion effect-based cross-channel correlation analysis. Based on that, we carefully design an attention-based encoder-decoder network to embrace cross-channel correlation for high-quality wide-bandwidth spectrum synthesis. To accommodate individual differences and improve model generalization, we implement a physics-based data augmentation strategy to expand the scale of the training dataset. Through extensive real-world experiments with 28 participants, EAROE can achieve an average Mel-cepstral distance of 8.08, an average modulation spectra distance of 0.82, and an average log-spectral distance of 11.16, outperforming existing solutions.
- Open Access
- 22 August 2024
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 8 (3) , 1-30
EarSpeech: Exploring In-Ear Occlusion Effect on Earphones for Data-efficient Airborne Speech Enhancement
Earphones have become a popular voice input and interaction device. However, airborne speech is susceptible to ambient noise, making it necessary to improve the quality and intelligibility of speech on earphones in noisy conditions. As the dual-microphone structure (i.e., outer and in-ear microphones) has been widely adopted in earphones (especially ANC earphones), we design EarSpeech which exploits in-ear acoustic sensory as the complementary modality to enable airborne speech enhancement. The key idea of EarSpeech is that in-ear speech is less sensitive to ambient noise and exhibits a correlation with airborne speech. However, due to the occlusion effect, in-ear speech has limited bandwidth, making it challenging to directly correlate with full-band airborne speech. Therefore, we exploit the occlusion effect to carry out theoretical modeling and quantitative analysis of this cross-channel correlation and study how to leverage such cross-channel correlation for speech enhancement. Specifically, we design a series of methodologies including data augmentation, deep learning-based fusion, and noise mixture scheme, to improve the generalization, effectiveness, and robustness of EarSpeech, respectively. Lastly, we conduct real-world experiments to evaluate the performance of our system. Specifically, EarSpeech achieves an average improvement ratio of 27.23% and 13.92% in terms of PESQ and STOI, respectively, and significantly improves SI-SDR by 8.91 dB. Benefiting from data augmentation, EarSpeech can achieve comparable performance with a small-scale dataset that is 40 times less than the original dataset. In addition, we validate the generalization of different users, speech content, and language types, respectively, as well as robustness in the real world via comprehensive experiments. The audio demo of EarSpeech is available on https://github.com/EarSpeech/earspeech.github.io/.
- 22 August 2024
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 8 (3) , 1-30
Open AccessEarSpeech: Exploring In-Ear Occlusion Effect on Earphones for Data-efficient Airborne Speech Enhancement
Earphones have become a popular voice input and interaction device. However, airborne speech is susceptible to ambient noise, making it necessary to improve the quality and intelligibility of speech on earphones in noisy conditions. As the dual-microphone structure (i.e., outer and in-ear microphones) has been widely adopted in earphones (especially ANC earphones), we design EarSpeech which exploits in-ear acoustic sensory as the complementary modality to enable airborne speech enhancement. The key idea of EarSpeech is that in-ear speech is less sensitive to ambient noise and exhibits a correlation with airborne speech. However, due to the occlusion effect, in-ear speech has limited bandwidth, making it challenging to directly correlate with full-band airborne speech. Therefore, we exploit the occlusion effect to carry out theoretical modeling and quantitative analysis of this cross-channel correlation and study how to leverage such cross-channel correlation for speech enhancement. Specifically, we design a series of methodologies including data augmentation, deep learning-based fusion, and noise mixture scheme, to improve the generalization, effectiveness, and robustness of EarSpeech, respectively. Lastly, we conduct real-world experiments to evaluate the performance of our system. Specifically, EarSpeech achieves an average improvement ratio of 27.23% and 13.92% in terms of PESQ and STOI, respectively, and significantly improves SI-SDR by 8.91 dB. Benefiting from data augmentation, EarSpeech can achieve comparable performance with a small-scale dataset that is 40 times less than the original dataset. In addition, we validate the generalization of different users, speech content, and language types, respectively, as well as robustness in the real world via comprehensive experiments. The audio demo of EarSpeech is available on https://github.com/EarSpeech/earspeech.github.io/.