Improving End-to-End Speech Translation for the Low Resource Language Fongbe to French

Authors:
Open Access
Publisher website
Google Scholar
Abstract
This study addresses the challenges of end-to-end (E2E) Speech-to-Text Translation (STT) for the low-resource Fongbe-to-French language pair using a transfer learning approach. We first establish robust baselines by integrating state-of-the-art pretrained speech encoders (HuBERT-147, AfriHuBERT, XLS-R, Whisper) with powerful text decoders (mBART, NLLB). This initial phase identified the AfriHuBERT–NLLB and XLS-R–NLLB combinations as the most competitive E2E configurations. To further enhance performance, we propose and evaluate three hybrid feature fusion strategies, the Bidirectional Co-Attention (BCOAT), the Feature-wise Linear Modulation (FiLM), and the Feature Sum (SUM). These methods are designed to strategically fuse intermediate representations extracted from two distinct and powerful encoders, AfriHuBERT and XLS-R, within the E2E architecture. The fusion process aims to enrich the linguistic and tonal information critical for accurate translation of the tonal Fongbe language. Experimental results demonstrate significant performance gains over the baselines. The BLEU score improved from 26.32 to a peak of 27.78 for the AfriHuBERT–NLLB configuration (using FiLM), and from 26.27 to a maximum of 28.05 for the XLS-R–NLLB configuration (using SUM). These findings confirm that translation quality for tonal languages like Fongbe can be substantially improved by extracting and combining high-quality, complementary features through advanced encoder fusion. Our hybrid feature fusion methods present a substantial advance in speech translation quality within resource-scarce linguistic environments.
Funding Information
  • Partnership for Skills in Applied Sciences, Engineering, and Technology
  • Regional Scholarship and Innovation Fund

This publication has 17 references indexed in Scilit: