Iteratively Improving Speech Recognition and Voice Conversion
Preprint
Abstract
Many existing works on voice conversion (VC) tasks use automatic speech recognition (ASR) models for ensuring linguistic consistency between source and converted samples. However, for the low-data resource domains, training a high-quality ASR remains to be a challenging task. In this work, we propose a novel iterative way of improving both the ASR and VC models. We first train an ASR model which is used to ensure content preservation while training a VC model. In the next iteration, the VC model is used as a data augmentation method to further fine-tune the ASR model and generalize it to diverse speakers. By iteratively leveraging the improved ASR model to train VC model and vice-versa, we experimentally show improvement in both the models. Our proposed framework outperforms the ASR and one-shot VC baseline models on English singing and Hindi speech domains in subjective and objective evaluations in low-data resource settings.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Hierarchical Diffusion Models for Singing Voice Neural Vocoder2023
- Towards high-fidelity singing voice conversion with acoustic reference and contrastive predictive coding2022
- NHSS: A speech and singing parallel databaseSpeech Communication, 2021
- MUCS 2021: Multilingual and Code-Switching ASR Challenges for Low Resource Indian Languages2021
- StarGANv2-VC: A Diverse, Unsupervised, Non-Parallel Framework for Natural-Sounding Voice Conversion2021
- One-Shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization2019
- ESPnet: End-to-End Speech Processing Toolkit2018
- Hybrid CTC/Attention Architecture for End-to-End Speech RecognitionIEEE Journal of Selected Topics in Signal Processing, 2017