Towards Real-World Streaming Speech Translation for Code-Switched Speech
Preprint
Abstract
Code-switching (CS), i.e. mixing different languages in a single sentence, is a common phenomenon in communication and can be challenging in many Natural Language Processing (NLP) settings. Previous studies on CS speech have shown promising results for end-to-end speech translation (ST), but have been limited to offline scenarios and to translation to one of the languages present in the source (\textit{monolingual transcription}). In this paper, we focus on two essential yet unexplored areas for real-world CS speech translation: streaming settings, and translation to a third language (i.e., a language not included in the source). To this end, we extend the Fisher and Miami test and validation datasets to include new targets in Spanish and German. Using this data, we train a model for both offline and streaming ST and we establish baseline results for the two settings mentioned earlier.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges2023
- ArzEn-ST: A Three-way Speech Translation Corpus for Code-Switched Egyptian Arabic-English2022
- End-to-End Speech Translation for Code Switched Speech2022
- Streaming Models for Joint Speech Recognition and Translation2021
- Transformers: State-of-the-Art Natural Language Processing2020
- Re-translation versus Streaming for Simultaneous Translation2020
- MuST-C: a Multilingual Speech Translation Corpus2019
- Low-Latency Neural Speech Translation2018
- A Call for Clarity in Reporting BLEU Scores2018
- Long Short-Term MemoryNeural Computation, 1997