Bernd T. Meyer

Neural tracking of speech is a well-established phenomenon in neuroscience. However, for speech signals with a fixed structure, significant correlations between speech envelopes and neurophysiological representations occur even for unheard sentences. We exploit a structured speech-in-noise matrix hearing test (Oldenburger Sentence Test, OLSA) to systematically quantify the relationship between acoustic sentence similarity and neural tracking. Simultaneous magnetoencephalography (MEG) and 76-channel electroencephalography (EEG) data, including 16 channels positioned directly around the ears (ear-EEG), were recorded from 21 young adults with normal hearing during the presentation of clean-speech audiobooks and OLSA sentences at six signal-to-noise ratios. A linear decoder trained on audiobooks reconstructed OLSA sentence envelopes. Reconstruction accuracies were compared using a linear mixed model across heard (matched) and unheard (mismatched) sentences of varying acoustic similarity. Significant reconstruction accuracies were achieved across MEG, EEG, and ear-EEG for both matched and mismatched sentences. For mismatched sentences, these accuracies gradually increased with their acoustic similarity to the heard speech data. The high similarity between sentences, which is especially prominent in matrix tests, can cause significant spurious tracking for mismatched stimuli. This effect can reach levels comparable to those of matched sentences and can be mistaken for true neural tracking. Robust neural tracking across modalities further supported the established viability of ear-EEG compared to whole-head systems.

Neural speech tracking is a powerful phenomenon in which brain signals synchronize with the time-varying features of continuous speech. This study benchmarks speech-in-noise tracking performance and the feasibility of objective speech reception threshold (SRT _neuro ) estimation across three sensor modalities: magnetoencephalography (MEG), scalp-electroencephalography (EEG), and ear-EEG. While MEG and scalp-EEG serve as reference measurements providing full-head coverage, ear-EEG offers a more unobtrusive and portable alternative. Twenty-one young, normal-hearing adults participated in simultaneous MEG and 76-channel EEG recordings, which incorporated 16 integrated around-ear electrodes (ear-EEG). During the recordings, participants listened to continuous audiobooks and matrix sentences at fixed intelligibility levels corresponding to individually varied signal-to-noise ratios (SNRs). Neural tracking was quantified via envelope reconstruction using a linear decoder, and psychometric functions were fitted to the data to derive SRT _neuro as the midpoint. We observed consistent envelope tracking across all three modalities, suggesting shared underlying neural processes, with reconstruction accuracies increasing alongside SNR. MEG yielded the highest signal quality, with reconstruction accuracies approximately 1.5 times higher than scalp-EEG and 3 times higher than ear-EEG. Although SRT _neuro was derived within a 5 dB margin of behavioral thresholds for all participants, these neurophysiological estimates did not significantly correlate with individual behavioral SRTs. Furthermore, ear-EEG proved less reliable and exhibited a slight bias. These results demonstrate that MEG, scalp-EEG, and ear-EEG are all viable for capturing robust neural envelope tracking. However, the acoustic envelope alone may be insufficient as a standalone predictor for diagnostics due to high inter-subject variability. Significance Statement: When listening to speech, the brain tracks the rhythm of the signal, specifically the temporal envelope. These neural responses provide insights into speech-in-noise perception and offer a potential objective measure of listening ability. This study is the first to benchmark speech-in-noise tracking across high-resolution MEG, affordable scalp-EEG, and emerging ear-EEG using individualized signal-to-noise ratios derived from psychometric functions. Our comparison reveals a clear performance hierarchy, proving that even ear-centered configurations can capture robust neural signals across varying acoustic conditions relative to high-fidelity references. While high inter-subject variability currently limits the diagnostic utility of standalone envelope tracking, characterizing these trade-offs across different sensor modalities establishes a foundation for advanced, individualized neural hearing assessments.

Epileptic seizures affect around 1% of people worldwide and have an enormous impact on the quality of life as well as the health of each patient. Electroencephalography (EEG) is widely used to diagnose epilepsy and detect seizures. Automatic detection and documentation of epileptic seizures using EEG signals would help neurologists evaluate the course of disease of each patient individually. As scalp EEG systems are not suited to be worn in everyday life situations, there is a need for mobile EEG systems to permanently record EEG signals. An approach for such mobile devices consists of using behind-the-ear (BTE) electrodes, leading to a reduction in electrode channels. To address this reduction, we investigated the influence of different scalp EEG channel arrangements on the detection of epileptic seizures. Raw EEG signals have been used as input for a long short-term memory (LSTM) recurrent neural network (RNN), as well as a combination of a convolutional neural network (CNN) and LSTM to classify ictal and inter-ictal phases. When using all channels of the 10–20 EEG cap system, the CNN-LSTM model achieved a sensitivity of 73%, with fewer than two seizures being falsely detected per hour. The usage of BTE channels as input to the proposed epileptic seizure detection produced a promising sensitivity of 68% with around 10 false alarms per hour.

Deep learning has the potential to enhance speech signals and increase their intelligibility for users of hearing aids. Deep models suited for real-world application should feature a low computational complexity and low processing delay of only a few milliseconds. In this paper, we explore deep speech enhancement that matches these requirements and contrast monaural and binaural processing algorithms in two complex acoustic scenes. Both algorithms are evaluated with objective metrics and in experiments with hearing-impaired listeners performing a speech-in-noise test. Results are compared to two traditional enhancement strategies, i.e., adaptive differential microphone processing and binaural beamforming. While in diffuse noise, all algorithms perform similarly, the binaural deep learning approach performs best in the presence of spatial interferers. Through a post-analysis, this can be attributed to improvements at low SNRs and to precise spatial filtering.

In this paper, we introduce a causal low-latency low-complexity approach for binaural multichannel blind speaker separation in noisy reverberant conditions. The model, referred to as Group Communication Binaural Filter and Sum Network (GCBFSnet) predicts complex filters for filter-and-sum beamforming in the time-frequency domain. We apply Group Communication (GC), i.e., latent model variables are split into groups and processed with a shared sequence model with the aim of reducing the complexity of a simple model only containing one convolutional and one recurrent module. With GC we are able to reduce the size of the model by up to 83% and the complexity up to 73% compared to the model without GC, while mostly retaining performance. Even for the smallest model configuration, GCBFSnet matches the performance of a low-complexity TasNet baseline in most metrics despite the larger size and higher number of required operations of the baseline.

In this paper, we propose a multilingual query-by-example keyword spotting (KWS) system based on a residual neural network. The model is trained as a classifier on a multilingual keyword dataset extracted from Common Voice sentences and fine-tuned using circle loss. We demonstrate the generalization ability of the model to new languages and report a mean reduction in EER of 59.2 % for previously seen and 47.9 % for unseen languages compared to a competitive baseline. We show that the word embeddings learned by the KWS model can be accurately predicted from the phoneme sequences using a simple LSTM model. Our system achieves a promising accuracy for streaming keyword spotting and keyword search on Common Voice audio using just 5 examples per keyword. Experiments on the Hey-Snips dataset show a good performance with a false negative rate of 5.4 % at only 0.1 false alarms per hour.

Listening to the audio of TV broadcast signals can be challenging for hearing-impaired as well as normal-hearing listeners, especially when background sounds are prominent or too loud compared to the speech signal. This can result in a reduced satisfaction and increased listening effort of the listeners. Since the broadcast sound is usually premixed, we perform a subjective evaluation for quantifying the potential of speech enhancement systems based on audio source separation and recurrent neural networks (RNN). Recently, RNNs have shown promising resultsin the context of sound source separation and real-time signal processing. In this paper, we separate the speech from the background signals and remix the separated sounds at a higher signal-to-noise ratio. This differs from classic speech enhancement, where usually only the extracted speech signal is exploited. The subjective evaluation with 20 normal-hearing subjects on real TV-broadcast material shows that our proposed enhancement system is able to reduce the listening effort by around 2 points on a 13-point listening effort rating scale and increases the perceived sound quality compared to the original mixture.

This paper introduces a dual-signal transformation LSTM network (DTLN) for real-time speech enhancement as part of the Deep Noise Suppression Challenge (DNS-Challenge). This approach combines a short-time Fourier transform (STFT) and a learned analysis and synthesis basis in a stacked-network approach with less than one million parameters. The model was trained on 500 h of noisy speech provided by the challenge organizers. The network is capable of real-time processing (one frame in, one frame out) and reaches competitive results. Combining these two types of signal transformations enables the DTLN to robustly extract information from magnitude spectra and incorporate phase information from the learned feature basis. The method shows state-of-the-art performance and outperforms the DNS-Challenge baseline by 0.24 points absolute in terms of the mean opinion score (MOS).