YZYou Zuo

You Zuo

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


You Zuo has a total of 18 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    FeelWave: Enabling Emotion-Aware Voice Interaction through Noise-Robust mmWave Emotion Sensing

    by Lingyu Wang, You Zuo, Dequan Wang, Chenming He, Chengzhen Meng, Xinran Zhang, Xiaoran Fan, Yanyong Zhang


  • Open Access

    mmMUSE: An mmWave-based Motion-resilient Universal Speech Enhancement System

    by Lingyu Wang, Kai Wang, Dequan Wang, You Zuo, Chenming He, Chengzhen Meng, Xiaoran Fan, Haojie Ren, Yanyong Zhang

    Speech enhancement improves the user interaction experience in voice-based smart systems. While microphone-based speech perception is limited by airborne noise, mmWave is immune to such interference. However, user-device motion hinders mmWave-based vocal extraction, dispersing vocal signals and introducing distortions. In this paper, we propose mmMUSE , an mmWave-based motion-resilient universal speech enhancement system that integrates mmWave and audio. To mitigate motion interference, we propose a two-stage method for robust vocal vibration extraction. Moreover, by proposing the Vocal-Noise-Ratio metric to assess the prominence of the vocal vibration, we enable real-time voice activity detection. We also design a complex-valued network that includes an attention-based fusion network for cross-modal complementing and a time-frequency masking network for correcting amplitude and phase of speech to isolate noises. Using datasets from 46 participants, mmMUSE outperforms the state-of-the-art speech enhancement models by 26% in SISDR and 34% in STOI on average. It also achieves SISDR improvements of 16.72 dB, 17.93 dB, 14.93 dB, and 18.95 dB in controlled environments involving intense noise, extensive motion, multiple speakers, and various obstructive materials, respectively. Finally, in real-world scenarios, including running, public spaces, and driving, mmMUSE achieves WER below 10%.


  • EarOE: Enabling Body-Channel Voice Interaction Interface on Earphones via Occlusion Effect

    by Feiyu Han, You Zuo, Weiwei Jiang, Dawei Yan, Yuxin Zhao, Panlong Yang, Yubo Yan

    Nowadays, voice input on earphones has become one of the most paramount human-computer interaction approaches. Traditional voice interaction is built on the air channel, which is highly noise-susceptible and suffers from being falsely triggered by nearby competing users. In our work, we design a noise-resistant voice interaction interface on earphones, named EAROE, which takes advantage of the narrow-bandwidth body channel to reconstruct high-fidelity audible speech. Although promising, directly taking the body channel as a voice interaction interface is non-trivial since the limited bandwidth of body-channel speech causes the original timbre information and linguistic content to be lost. To address these issues, we employ an electro-acoustic (EA) model for occlusion effect-based cross-channel correlation analysis. Based on that, we carefully design an attention-based encoder-decoder network to embrace cross-channel correlation for high-quality wide-bandwidth spectrum synthesis. To accommodate individual differences and improve model generalization, we implement a physics-based data augmentation strategy to expand the scale of the training dataset. Through extensive real-world experiments with 28 participants, EAROE can achieve an average Mel-cepstral distance of 8.08, an average modulation spectra distance of 0.82, and an average log-spectral distance of 11.16, outperforming existing solutions.


  • Open Access

    EarSpeech: Exploring In-Ear Occlusion Effect on Earphones for Data-efficient Airborne Speech Enhancement

    by Feiyu Han, Panlong Yang, You Zuo, Fei Shang, Fenglei Xu, Xiang-Yang Li

    Earphones have become a popular voice input and interaction device. However, airborne speech is susceptible to ambient noise, making it necessary to improve the quality and intelligibility of speech on earphones in noisy conditions. As the dual-microphone structure (i.e., outer and in-ear microphones) has been widely adopted in earphones (especially ANC earphones), we design EarSpeech which exploits in-ear acoustic sensory as the complementary modality to enable airborne speech enhancement. The key idea of EarSpeech is that in-ear speech is less sensitive to ambient noise and exhibits a correlation with airborne speech. However, due to the occlusion effect, in-ear speech has limited bandwidth, making it challenging to directly correlate with full-band airborne speech. Therefore, we exploit the occlusion effect to carry out theoretical modeling and quantitative analysis of this cross-channel correlation and study how to leverage such cross-channel correlation for speech enhancement. Specifically, we design a series of methodologies including data augmentation, deep learning-based fusion, and noise mixture scheme, to improve the generalization, effectiveness, and robustness of EarSpeech, respectively. Lastly, we conduct real-world experiments to evaluate the performance of our system. Specifically, EarSpeech achieves an average improvement ratio of 27.23% and 13.92% in terms of PESQ and STOI, respectively, and significantly improves SI-SDR by 8.91 dB. Benefiting from data augmentation, EarSpeech can achieve comparable performance with a small-scale dataset that is 40 times less than the original dataset. In addition, we validate the generalization of different users, speech content, and language types, respectively, as well as robustness in the real world via comprehensive experiments. The audio demo of EarSpeech is available on https://github.com/EarSpeech/earspeech.github.io/.