Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation
Top Cited Papers
- 22 September 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 33, 4044-4054
Abstract
Recent advancements in speech generation have been driven by large-scale training datasets. However, current models struggle to capture the spontaneity and variability inherent in real-world human speech, as they are primarily trained on audio-book datasets limited to formal, read-aloud speaking styles. To address this limitation, we introduce Emilia-Pipe, an open-source preprocessing pipeline designed to extract high-quality training data from valuable yet under-explored in-the-wild sources that capture spontaneous human speech in real-world contexts. Using Emilia-Pipe, we construct Emilia, which comprises over 101k hours of speech across six languages: English, Chinese, German, French, Japanese, and Korean. Furthermore, we expand Emilia to Emilia-Large, a dataset exceeding 216k hours, making it one of the largest open-source speech generation resources available. Extensive experiments show that Emilia-trained models produce markedly more spontaneous, human-like speech than those trained on traditional audio-book datasets, while matching their intelligibility. These models better capture diverse speaker timbres and the full spectrum of real-world conversational styles. Our work also highlights the importance of scaling dataset size for advancing speech generation performance and validates the effectiveness of Emilia for both multilingual and crosslingual speech generation tasks.Keywords
All Related Versions
Funding Information
- Chinese University of Hong Kong, Shenzhen
- National Natural Science Foundation of China (62376237)
- 2023 Shenzhen Stability Science Program
- Shenzhen Science and Technology Program (ZDSYS20230626091302006)
- Program for Guangdong Introducing Innovative and Entrepreneurial Teams (2023ZT10X044)
- Postdoctoral Fellowship Program of China Postdoctoral Science Foundation (GZB20230811)
This publication has 25 references indexed in Scilit:
- SPMIS: An Investigation of Synthetic Spoken Misinformation Detection2024
- AutoPrep: An Automatic Preprocessing Framework for In-The-Wild Speech Data2024
- Libriheavy: A 50,000 Hours ASR Corpus with Punctuation Casing and Context2024
- Yodas: Youtube-Oriented Dataset for Audio and Speech2023
- WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech ProcessingIEEE Journal of Selected Topics in Signal Processing, 2022
- Dnsmos P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors2022
- WENETSPEECH: A 10000+ Hours Multi-Domain Mandarin Corpus for Speech Recognition2022
- How Far Are We from Robust Voice Conversion: A Survey2021
- Libri-Light: A Benchmark for ASR with Limited or No Supervision2020
- LIII. On lines and planes of closest fit to systems of points in spaceJournal of Computers in Education, 1901