Eva Navas
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Eva Navas has a total of 130 co-authors. Here are the top co-authors by number of shared publications:
- Inma Hernaez
HiTZ Basque Center for Language Technology, University of the Basque Country, Ingeniero Torres Quevedo Plaza, 1, 48013 Bilbao, Spain
54 shared publications - Ibon Saratxaga
HiTZ Center University of the Basque Country -UPV/EHU Spain
19 shared publications - Jon Sanchez
University of the Basque Country
17 shared publications Daniel Erro
8 shared publicationsLuis Serrano
7 shared publicationsD. Erro
6 shared publications
Latest Publications
- Open Access
MEG-to-MEG Transfer Learning and Cross-Task Speech/Silence Detection with Limited Data
Data-efficient neural decoding is a central challenge for speech brain-computer interfaces. We present the first demonstration of transfer learning and cross-task decoding for MEG-based speech models spanning perception and production. We pre-train a Conformer-based model on 50 hours of single-subject listening data and fine-tune on just 5 minutes per subject across 18 participants. Transfer learning yields consistent improvements, with in-task accuracy gains of 1-4% and larger cross-task gains of up to 5-6%. Not only does pre-training improve performance within each task, but it also enables reliable cross-task decoding between perception and production. Critically, models trained on speech production decode passive listening above chance, confirming that learned representations reflect shared neural processes rather than task-specific motor activity.
Open AccessMEG-to-MEG Transfer Learning and Cross-Task Speech/Silence Detection with Limited Data
Data-efficient neural decoding is a central challenge for speech brain-computer interfaces. We present the first demonstration of transfer learning and cross-task decoding for MEG-based speech models spanning perception and production. We pre-train a Conformer-based model on 50 hours of single-subject listening data and fine-tune on just 5 minutes per subject across 18 participants. Transfer learning yields consistent improvements, with in-task accuracy gains of 1-4% and larger cross-task gains of up to 5-6%. Not only does pre-training improve performance within each task, but it also enables reliable cross-task decoding between perception and production. Critically, models trained on speech production decode passive listening above chance, confirming that learned representations reflect shared neural processes rather than task-specific motor activity.
- Open Access
Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages
Automatic speech recognition systems have undoubtedly advanced with the integration of multilingual and multitask models such as Whisper, which have shown a promising ability to understand and process speech across a wide range of languages. Despite their robustness, these models often fall short in handling the linguistic distinctions of minority languages. This study addresses this gap by integrating traditional and novel language models with fine-tuned Whisper models to raise their performance in less commonly studied languages. Through rigorous fine-tuning and evaluation across multiple datasets, we demonstrate substantial improvements in word error rate, particularly in low-resource scenarios. Our approach not only does take advantage of the extensive data Whisper was pre-trained on, but also complements its linguistic adaptability by incorporating language models. We obtained improvements up to 51% for in-distribution datasets and up to 34% for out-of-distribution sentences using statistical language models, while large language models provided moderate but consistently robust improvement across diverse linguistic contexts. The findings reveal that, while the integration reliably benefits all model sizes, the extent of improvement varies, highlighting the importance of optimized language model parameters. Finally, we emphasize the importance of selecting appropriate evaluation parameters when reporting the results using transformer-based ASR models. In summary, this research clears the way for more inclusive ASR technologies that perform better across languages by enriching their linguistic knowledge. For further implementation details of this study, the technical documentation and source code are available at http://www.github.com/hitz-zentroa/whisper-lm.
Open AccessWhisper-LM: Improving ASR Models with Language Models for Low-Resource Languages
Automatic speech recognition systems have undoubtedly advanced with the integration of multilingual and multitask models such as Whisper, which have shown a promising ability to understand and process speech across a wide range of languages. Despite their robustness, these models often fall short in handling the linguistic distinctions of minority languages. This study addresses this gap by integrating traditional and novel language models with fine-tuned Whisper models to raise their performance in less commonly studied languages. Through rigorous fine-tuning and evaluation across multiple datasets, we demonstrate substantial improvements in word error rate, particularly in low-resource scenarios. Our approach not only does take advantage of the extensive data Whisper was pre-trained on, but also complements its linguistic adaptability by incorporating language models. We obtained improvements up to 51% for in-distribution datasets and up to 34% for out-of-distribution sentences using statistical language models, while large language models provided moderate but consistently robust improvement across diverse linguistic contexts. The findings reveal that, while the integration reliably benefits all model sizes, the extent of improvement varies, highlighting the importance of optimized language model parameters. Finally, we emphasize the importance of selecting appropriate evaluation parameters when reporting the results using transformer-based ASR models. In summary, this research clears the way for more inclusive ASR technologies that perform better across languages by enriching their linguistic knowledge. For further implementation details of this study, the technical documentation and source code are available at http://www.github.com/hitz-zentroa/whisper-lm.
- Open Access
Electrode Setup for Electromyography-Based Silent Speech Interfaces: A Pilot Study
This paper describes a series of pilot experiments developed to define the electrode setup in order to record a novel parallel electromyography (EMG)–audio database. The main purpose of the database is to provide data useful for the development of an EMG-based silent speech interface for Spanish laryngectomized speakers. Motivated by the scarcity of information in related studies regarding this important decision-making process, we decided to carry out a set of experiments with multiple recording sessions and different setups. We included different electrode types (paired and concentric) and locations targeting different muscles in the face and neck involved in the speech production process. We then analyzed the results obtained in a phone classification task using frame-based phone accuracy. The final setup consists of eight channels with bipolar single-electrode pairs targeting eight specific muscles crucial for capturing speech-related information: the digastric, the depressor anguli oris, the risorius, the levator labii superioris, the masseter, the zygomaticus major, the depressor labii superioris, and the stylohyoid. This setup has been used for the final recordings in the database. By providing insight into the electrode setups that were used in related studies and the optimal setup that resulted from this pilot study, we hope that this research will help future researchers in the field in determining their experimental setup.
Open AccessElectrode Setup for Electromyography-Based Silent Speech Interfaces: A Pilot Study
This paper describes a series of pilot experiments developed to define the electrode setup in order to record a novel parallel electromyography (EMG)–audio database. The main purpose of the database is to provide data useful for the development of an EMG-based silent speech interface for Spanish laryngectomized speakers. Motivated by the scarcity of information in related studies regarding this important decision-making process, we decided to carry out a set of experiments with multiple recording sessions and different setups. We included different electrode types (paired and concentric) and locations targeting different muscles in the face and neck involved in the speech production process. We then analyzed the results obtained in a phone classification task using frame-based phone accuracy. The final setup consists of eight channels with bipolar single-electrode pairs targeting eight specific muscles crucial for capturing speech-related information: the digastric, the depressor anguli oris, the risorius, the levator labii superioris, the masseter, the zygomaticus major, the depressor labii superioris, and the stylohyoid. This setup has been used for the final recordings in the database. By providing insight into the electrode setups that were used in related studies and the optimal setup that resulted from this pilot study, we hope that this research will help future researchers in the field in determining their experimental setup.
- Open Access
- 17 August 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (16) , 9358
NoisenseDB: An Urban Sound Event Database to Develop Neural Classification Systems for Noise-Monitoring Applications
The use of continuous monitoring systems to control aspects such as noise pollution has grown in recent years. The commercial monitoring systems used to date only provide information on noise levels but do not identify the noise sources that generate them. The identification of noise sources is an important aspect in order to apply corrective measures to mitigate the noise levels. In this sense, new technological advances like machine listening can enable the addition of other capabilities to sound monitoring systems such as the detection and classification of noise sources. Despite the increasing development of these systems, researchers have to face some shortcomings. The most frequent ones are on the one hand, the lack of data recorded in real environments and on the other hand, the need for automatic labelling of large volumes of data collected by working monitoring systems. In order to address these needs, in this paper, we present our own sound database recorded in an urban environment. Some baseline results for the database are provided using two original convolutional neural network based sound events classification systems. Additionally, a state of the art transformer-based audio classification system (AST) has been applied to obtain some baseline results. Furthermore, the database has been used for evaluating a semi-supervised strategy to train a classifier for automatic labelling that can be refined by human labellers afterwards.
- 17 August 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (16) , 9358
Open AccessNoisenseDB: An Urban Sound Event Database to Develop Neural Classification Systems for Noise-Monitoring Applications
The use of continuous monitoring systems to control aspects such as noise pollution has grown in recent years. The commercial monitoring systems used to date only provide information on noise levels but do not identify the noise sources that generate them. The identification of noise sources is an important aspect in order to apply corrective measures to mitigate the noise levels. In this sense, new technological advances like machine listening can enable the addition of other capabilities to sound monitoring systems such as the detection and classification of noise sources. Despite the increasing development of these systems, researchers have to face some shortcomings. The most frequent ones are on the one hand, the lack of data recorded in real environments and on the other hand, the need for automatic labelling of large volumes of data collected by working monitoring systems. In order to address these needs, in this paper, we present our own sound database recorded in an urban environment. Some baseline results for the database are provided using two original convolutional neural network based sound events classification systems. Additionally, a state of the art transformer-based audio classification system (AST) has been applied to obtain some baseline results. Furthermore, the database has been used for evaluating a semi-supervised strategy to train a classifier for automatic labelling that can be refined by human labellers afterwards.
- Open Access
- 30 June 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (13) , 7746
Frame-Based Phone Classification Using EMG Signals
This paper evaluates the impact of inter-speaker and inter-session variability on the development of a silent speech interface (SSI) based on electromyographic (EMG) signals from the facial muscles. The final goal of the SSI is to provide a communication tool for Spanish-speaking laryngectomees by generating audible speech from voiceless articulation. However, before moving on to such a complex task, a simpler phone classification task in different modalities regarding speaker and session dependency is performed for this study. These experiments consist of processing the recorded utterances into phone-labeled segments and predicting the phonetic labels using only features obtained from the EMG signals. We evaluate and compare the performance of each model considering the classification accuracy. Results show that the models are able to predict the phonetic label best when they are trained and tested using data from the same session. The accuracy drops drastically when the model is tested with data from a different session, although it improves when more data are added to the training data. Similarly, when the same model is tested on a session from a different speaker, the accuracy decreases. This suggests that using larger amounts of data could help to reduce the impact of inter-session variability, but more research is required to understand if this approach would suffice to account for inter-speaker variability as well.
- 30 June 2023
- Published by MDPI AG in Applied Sciences
- Vol. 13 (13) , 7746
Open AccessFrame-Based Phone Classification Using EMG Signals
This paper evaluates the impact of inter-speaker and inter-session variability on the development of a silent speech interface (SSI) based on electromyographic (EMG) signals from the facial muscles. The final goal of the SSI is to provide a communication tool for Spanish-speaking laryngectomees by generating audible speech from voiceless articulation. However, before moving on to such a complex task, a simpler phone classification task in different modalities regarding speaker and session dependency is performed for this study. These experiments consist of processing the recorded utterances into phone-labeled segments and predicting the phonetic labels using only features obtained from the EMG signals. We evaluate and compare the performance of each model considering the classification accuracy. Results show that the models are able to predict the phonetic label best when they are trained and tested using data from the same session. The accuracy drops drastically when the model is tested with data from a different session, although it improves when more data are added to the training data. Similarly, when the same model is tested on a session from a different speaker, the accuracy decreases. This suggests that using larger amounts of data could help to reduce the impact of inter-session variability, but more research is required to understand if this approach would suffice to account for inter-speaker variability as well.
- Open Access
- 7 June 2023
- Published by Springer Nature
- p. 289-312
Deep Dive Speech Technology
This chapter provides an in-depth account of current research activities and applications in the field of Speech Technology (ST). It discusses technical, scientific, commercial and societal aspects in various ST sub-fields and relates ST to the wider areas of Natural Language Processing and Artificial Intelligence. Furthermore, it outlines breakthroughs needed, main technology visions and provides an outlook towards 2030 as well as a broad view of how ST may fit into and contribute to a wider vision of Deep Natural Language Understanding and Digital Language Equality in Europe. The chapter integrates the views of several companies and institutions involved in research and commercial application of ST.
- 7 June 2023
- Published by Springer Nature
- p. 289-312
Open AccessDeep Dive Speech Technology
This chapter provides an in-depth account of current research activities and applications in the field of Speech Technology (ST). It discusses technical, scientific, commercial and societal aspects in various ST sub-fields and relates ST to the wider areas of Natural Language Processing and Artificial Intelligence. Furthermore, it outlines breakthroughs needed, main technology visions and provides an outlook towards 2030 as well as a broad view of how ST may fit into and contribute to a wider vision of Deep Natural Language Understanding and Digital Language Equality in Europe. The chapter integrates the views of several companies and institutions involved in research and commercial application of ST.
- Open Access
- 7 June 2023
- Published by Springer Nature
- p. 13-38
State-of-the-Art in Language Technology and Language-centric Artificial Intelligence
This chapter landscapes the field of Language Technology (LT) and language- centric AI by assembling a comprehensive state-of-the-art of basic and applied research in the area. It sketches all recent advances in AI, including the most recent deep learning neural technologies. The chapter brings to light not only where language-centric AI as a whole stands, but also where the required resources should be allocated to place European LT at the forefront of the AI revolution. We identify key research areas and gaps that need to be addressed to ensure LT can overcome the current inequalities.
- 7 June 2023
- Published by Springer Nature
- p. 13-38
Open AccessState-of-the-Art in Language Technology and Language-centric Artificial Intelligence
This chapter landscapes the field of Language Technology (LT) and language- centric AI by assembling a comprehensive state-of-the-art of basic and applied research in the area. It sketches all recent advances in AI, including the most recent deep learning neural technologies. The chapter brings to light not only where language-centric AI as a whole stands, but also where the required resources should be allocated to place European LT at the forefront of the AI revolution. We identify key research areas and gaps that need to be addressed to ensure LT can overcome the current inequalities.
- Open Access
- 7 June 2023
- Published by Springer Nature
- p. 95-98
Language Report Basque
Since 1968 Basque has been immersed in a process of revitalisation that has faced formidable obstacles. Nonetheless, significant progress has been made in numerous areas. The Language Technology community widely accepts the standardised language and constructs efficacious LT tools. After thirty years of collaborative work, research has resulted in state-of-the-art technology and robust, broad-coverage NLP for Basque. However, a dramatic difference remains between Basque and other European languages in terms of both the maturity of research and the state of readiness with respect to language technology solutions.
- 7 June 2023
- Published by Springer Nature
- p. 95-98
Open AccessLanguage Report Basque
Since 1968 Basque has been immersed in a process of revitalisation that has faced formidable obstacles. Nonetheless, significant progress has been made in numerous areas. The Language Technology community widely accepts the standardised language and constructs efficacious LT tools. After thirty years of collaborative work, research has resulted in state-of-the-art technology and robust, broad-coverage NLP for Basque. However, a dramatic difference remains between Basque and other European languages in terms of both the maturity of research and the state of readiness with respect to language technology solutions.
- Open Access
- 27 February 2022
- Published by MDPI AG in Applied Sciences
- Vol. 12 (5) , 2473
Automatic Classification of Synthetic Voices for Voice Banking Using Objective Measures
Speech is the most common way of communication among humans. People who cannot communicate through speech due to partial of total loss of the voice can benefit from Alternative and Augmentative Communication devices and Text to Speech technology. One problem of using these technologies is that the included synthetic voices might be impersonal and badly adapted to the user in terms of age, accent or even gender. In this context, the use of synthetic voices from voice banking systems is an attractive alternative. New voices can be obtained applying adaptation techniques using recordings from people with healthy voice (donors) or from the user himself/herself before losing his/her own voice. In this way, the goal is to offer a wide voice catalog to potential users. However, as there is no control over the recording or the adaptation processes, some method to control the final quality of the voice is needed. We present the work developed to automatically select the best synthetic voices using a set of objective measures and a subjective Mean Opinion Score evaluation. A prediction algorithm of the MOS has been build which correlates similarly to the most correlated individual measure.
- 27 February 2022
- Published by MDPI AG in Applied Sciences
- Vol. 12 (5) , 2473
Open AccessAutomatic Classification of Synthetic Voices for Voice Banking Using Objective Measures
Speech is the most common way of communication among humans. People who cannot communicate through speech due to partial of total loss of the voice can benefit from Alternative and Augmentative Communication devices and Text to Speech technology. One problem of using these technologies is that the included synthetic voices might be impersonal and badly adapted to the user in terms of age, accent or even gender. In this context, the use of synthetic voices from voice banking systems is an attractive alternative. New voices can be obtained applying adaptation techniques using recordings from people with healthy voice (donors) or from the user himself/herself before losing his/her own voice. In this way, the goal is to offer a wide voice catalog to potential users. However, as there is no control over the recording or the adaptation processes, some method to control the final quality of the voice is needed. We present the work developed to automatically select the best synthetic voices using a set of objective measures and a subjective Mean Opinion Score evaluation. A prediction algorithm of the MOS has been build which correlates similarly to the most correlated individual measure.
- Open Access
- 7 February 2022
- Published by MDPI AG in Applied Sciences
- Vol. 12 (3) , 1686
Evaluation of Tacotron Based Synthesizers for Spanish and Basque
In this paper, we describe the implementation and evaluation of Text to Speech synthesizers based on neural networks for Spanish and Basque. Several voices were built, all of them using a limited number of data. The system applies Tacotron 2 to compute mel-spectrograms from the input sequence, followed by WaveGlow as neural vocoder to obtain the audio signals from the spectrograms. The limited number of data used for training the models leads to synthesis errors in some sentences. To automatically detect those errors, we developed a new method that is able to find the sentences that have lost the alignment during the inference process. To mitigate the problem, we implemented a guided attention providing the system with the explicit duration of the phonemes. The resulting system was evaluated to assess its robustness, quality and naturalness both with objective and subjective measures. The results reveal the capacity of the system to produce good quality and natural audios.
- 7 February 2022
- Published by MDPI AG in Applied Sciences
- Vol. 12 (3) , 1686
Open AccessEvaluation of Tacotron Based Synthesizers for Spanish and Basque
In this paper, we describe the implementation and evaluation of Text to Speech synthesizers based on neural networks for Spanish and Basque. Several voices were built, all of them using a limited number of data. The system applies Tacotron 2 to compute mel-spectrograms from the input sequence, followed by WaveGlow as neural vocoder to obtain the audio signals from the spectrograms. The limited number of data used for training the models leads to synthesis errors in some sentences. To automatically detect those errors, we developed a new method that is able to find the sentences that have lost the alignment during the inference process. To mitigate the problem, we implemented a guided attention providing the system with the explicit duration of the phonemes. The resulting system was evaluated to assess its robustness, quality and naturalness both with objective and subjective measures. The results reveal the capacity of the system to produce good quality and natural audios.