HHHynek Hermansky

Hynek Hermansky

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Hynek Hermansky has a total of 93 co-authors. Here are the top co-authors by number of shared publications:

  • Samuel Thomas

    Human Language Technology Center of Excellence, Center for Language and Speech Processing, Johns Hopkins University, Baltimore, MD USA

    6 shared publications
  • Sriram Ganapathy

    Center for Language and Speech Processing, Department of Electrical and Computer Engineering, The Johns Hopkins University, USA

    4 shared publications
  • Sri Harish Mallidi

    3 shared publications
  • Aren Jansen

    Human Language Technology Center of Excellence, Center for Language and Speech Processing, Johns Hopkins University, Baltimore, MD USA

    2 shared publications
  • Bernd T. Meyer

    2 shared publications
  • Feipeng Li

    School of Information and Communications Engineering and the National Engineering Research Center for Offshore Oil and Gas Exploration, Xian Jiaotong University, Xian, Shaanxi, China

    2 shared publications

Latest Publications

  • Open Access

    Self-paced silent speech brain-computer interface for device control

    by Shiyu Luo, Miguel Angrick, Christopher Coogan, Daniel N. Candrea, Kimberley Wyse-Sookoo, Anouck Schippers, Rohit Ganji, Griffin W. Milsap, William S. Anderson, Chad R. Gordon, et al.

    Brain-computer interfaces can potentially restore autonomy to people with paralysis, including the ability to control their own environment via smart devices in the home. BCI applications for controlling smart devices to date have required visual displays or auditory cues, limiting the user’s ability to autonomously and privately issue commands. In this study, a clinical trial participant with amyotrophic lateral sclerosis (ALS) used a chronically implanted electrocorticographic (ECoG) BCI to control smart devices with self-paced silent speech commands. Across 18 experimental sessions, silently mimed speech commands were detected in real time and decoded with a median accuracy of 97.1% (chance: 7.14%). These results demonstrate that silently attempted speech can be reliably decoded without exogenous timing cues, supporting the feasibility of reliable autonomous smart device control with an implantable BCI.

  • Open Access

    Iterative alignment discovery of speech-associated neural activity

    by Qinwan Rabbani, Samyak Shah, Griffin Milsap, Matthew Fifer, Hynek Hermansky, Nathan Crone

    Objective. Brain–computer interfaces (BCIs) have the potential to preserve or restore speech in patients with neurological disorders that weaken the muscles involved in speech production. However, successful training of low-latency speech synthesis and recognition models requires alignment of neural activity with intended phonetic or acoustic output with high temporal precision. This is particularly challenging in patients who cannot produce audible speech, as ground truth with which to pinpoint neural activity synchronized with speech is not available. Approach. In this study, we present a new iterative algorithm for neural voice activity detection (nVAD) called iterative alignment discovery dynamic time warping (IAD-DTW) that integrates DTW into the loss function of a deep neural network (DNN). The algorithm is designed to discover the alignment between a patient’s electrocorticographic (ECoG) neural responses and their attempts to speak during collection of data for training BCI decoders for speech synthesis and recognition. Main results. To demonstrate the effectiveness of the algorithm, we tested its accuracy in predicting the onset and duration of acoustic signals produced by able-bodied patients with intact speech undergoing short-term diagnostic ECoG recordings for epilepsy surgery. We simulated a lack of ground truth by randomly perturbing the temporal correspondence between neural activity and an initial single estimate for all speech onsets and durations. We examined the model’s ability to overcome these perturbations to estimate ground truth. IAD-DTW showed no notable degradation (Significance. IAD-DTW is computationally inexpensive and can be easily integrated into existing DNN-based nVAD approaches, as it pertains only to the final loss computation. This approach makes it possible to train speech BCI algorithms using ECoG data from patients who are unable to produce audible speech, including those with Locked-In Syndrome.


  • Open Access

    Complex Frequency Domain Linear Prediction: A Tool to Compute Modulation Spectrum of Speech

    by Samik Sadhu, Hynek Hermansky

    Conventional Frequency Domain Linear Prediction (FDLP) technique models the squared Hilbert envelope of speech with varied degrees of approximation which can be sampled at the required frame rate and used as features for Automatic Speech Recognition (ASR). Although previously the complex cepstrum of the conventional FDLP model has been used as compact frame-wise speech features, it has lacked interpretability in the context of the Hilbert envelope. In this paper, we propose a modification of the conventional FDLP model that allows easy interpretability of the complex cepstrum as temporal modulations in an all-pole model approximation of the power of the speech signal. Additionally, our "complex" FDLP yields significant speed-ups in comparison to conventional FDLP for the same degree of approximation.

  • Open Access

    Radically Old Way of Computing Spectra: Applications in End-to-End ASR

    by Samik Sadhu, Hynek Hermansky

    We propose a technique to compute spectrograms using Frequency Domain Linear Prediction (FDLP) that uses all-pole models to fit the squared Hilbert envelope of speech in different frequency sub-bands. The spectrogram of a complete speech utterance is computed by overlap-add of contiguous all-pole model responses. A long context window of 1.5 seconds allows us to capture the low frequency temporal modulations of speech in the spectrogram. For an end-to-end automatic speech recognition task, the FDLP spectrogram performs on par with the standard mel spectrogram features for clean read speech training and test data. For more realistic speech data with train-test domain mismatches or reverberations, FDLP spectrogram shows up to 25% and 22% relative WER improvements over mel spectrogram respectively.


  • Multi-Stream End-to-End Speech Recognition

    by Ruizhi Li, Xiaofei Wang, Sri Harish R. Mallidi, Shinji Watanabe, Takaaki Hori, Hynek Hermansky

    Attention-based methods and Connectionist Temporal Classification (CTC) network have been promising research directions for end-to-end (E2E) Automatic Speech Recognition (ASR). The joint CTC/Attention model has achieved great success by utilizing both architectures during multi-task training and joint decoding. In this work, we present a multi-stream framework based on joint CTC/Attention E2E ASR with parallel streams represented by separate encoders aiming to capture diverse information. On top of the regular attention networks, the Hierarchical Attention Network (HAN) is introduced to steer the decoder toward the most informative encoders. A separate CTC network is assigned to each stream to force monotonic alignments. Two representative framework have been proposed and discussed, which are Multi-Encoder Multi-Resolution (MEM-Res) framework and Multi-Encoder Multi-Array (MEM-Array) framework, respectively. In MEM-Res framework, two heterogeneous encoders with different architectures, temporal resolutions and separate CTC networks work in parallel to extract complimentary information from same acoustics. Experiments are conducted on Wall Street Journal (WSJ) and CHiME-4, resulting in relative Word Error Rate (WER) reduction of 18.0-32.1% and the best WER of 3.6% in the WSJ eval92 test set. The MEM-Array framework aims at improving the far-field ASR robustness using multiple microphone arrays which are activated by separate encoders. Compared with the best single-array results, the proposed framework has achieved relative WER reduction of 3.7% and 9.7% in AMI and DIRHA multi-array corpora, respectively, which also outperforms conventional fusion strategies.


  • Open Access

    General properties of auditory spectro-temporal receptive fields

    by Nagaraj R. Mahajan, Nima Mesgarani, Hynek Hermansky

    The analyses of more than 2000 spectro-temporal receptive fields (STRFs) obtained from the ferret primary auditory cortex revealed their dominant encoding properties along time and frequency domains. Results showed that the peak responses of the STRFs were roughly aligned along the time axis, and enhanced low modulation components of the signal around 3 Hz. On the contrary, the peaks of the STRF population along the frequency axis varied and were distributed widely. Further analyses revealed that there were some general properties along the frequency axis around the best frequency of each STRF. The STRFs enhanced modulation frequencies around 0.25 cycles/octave. These findings are consistent with techniques that have been reported in the literature to result in improvements in speech recognition performance, highlighting the significant potential of biological insights into the design of better machines.


  • Open Access

    Exploring Methods for the Automatic Detection of Errors in Manual Transcription

    by Xiaofei Wang, Jinyi Yang, Ruizhi Li, Samik Sadhu, Hynek Hermansky

    Quality of data plays an important role in most deep learning tasks. In the speech community, transcription of speech recording is indispensable. Since the transcription is usually generated artificially, automatically finding errors in manual transcriptions not only saves time and labors but benefits the performance of tasks that need the training process. Inspired by the success of hybrid automatic speech recognition using both language model and acoustic model, two approaches of automatic error detection in the transcriptions have been explored in this work. Previous study using a biased language model approach, relying on a strong transcription-dependent language model, has been reviewed. In this work, we propose a novel acoustic model based approach, focusing on the phonetic sequence of speech. Both methods have been evaluated on a completely real dataset, which was originally transcribed with errors and strictly corrected manually afterwards.

  • DNN-based performance measures for predicting error rates in automatic speech recognition and optimizing hearing aid parameters

    by Angel Mario Castro Martinez, Lukas Gerlach, Guillermo Payá-Vayá, Hynek Hermansky, Jasper Ooster, Bernd T. Meyer