A Pitch-Controlled End-to-End Voice Conversion System for Brazilian Portuguese
Authors:
- Universidade Federal do Rio de JaneiroUJUniversidade Federal do Rio de Janeiro
- da Costa Vitor P.DPda Costa Vitor P.
- Netto Sergio L.Netto Sergio L.
- Biscainho Luiz Wagner PBPBiscainho Luiz Wagner P
- Maia RannieryMRMaia Ranniery
- Universidade Federal do Rio Grande do NorteUNUniversidade Federal do Rio Grande do Norte
Open Access
- 1 January 2024
- journal article
- Published by Sociedad Brasileira de Telecomunicacoes in Journal of Communication and Information Systems
- Vol. 39 (1) , 127-136
Abstract
Speech conversion is a technique that modifies the identity of the voice in a speech signal without changing the spoken content. Accurate pitch conversion is a requirement the best speech conversion systems must address, as this characteristic is essential to the correct identification of the target speaker. This work proposes a pitch-controlled end-to-end voice conversion model that combines state-of-the-art ideas from both speaking and singing voice conversion with a novel cost function to ensure artifact-free pitch tracking. The model is trained in Brazilian Portuguese, overcoming the lack of high-quality data by improving a large but flawed dataset with filtering operation. Our model mostly outperforms other popular open source models in both listening tests and objective measurements. In particular, on a 5-point MOS, we obtained the highest speaker similarity score (4.05), and a naturalness score of 3.48, second only to a system whose similarity score was 2.62.Keywords
This publication has 36 references indexed in Scilit:
- The Singing Voice Conversion Challenge 20232023
- Hubert: How Much Can a Bad Teacher Benefit ASR Pre-Training?2021
- Waveglow: A Flow-based Generative Network for Speech Synthesis2019
- Neural Source-filter-based Waveform Model for Statistical Parametric Speech Synthesis2019
- Crepe: A Convolutional Representation for Pitch Estimation2018
- Image-to-Image Translation with Conditional Adversarial Networks2017
- HMM-based sequence-to-frame mapping for voice conversion2010
- Voice conversion through vector quantization2003
- Restructuring speech representations using a pitch-adaptive time–frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible role of a repetitive structure in soundsSpeech Communication, 1999
- Continuous probabilistic transform for voice conversionIEEE Transactions on Speech and Audio Processing, 1998