Gautam Shreedhar Bhat

Deep neural networks (DNNs) have been useful in solving benchmark problems in various domains including audio. DNNs have been used to improve several speech processing algorithms that improve speech perception for hearing impaired listeners. To make use of DNNs to their full potential and to configure models easily, automated machine learning (AutoML) systems are developed, focusing on model optimization. As an application of AutoML to audio and hearing aids, this work presents an AutoML based voice activity detector (VAD) that is implemented on a smartphone as a real-time application. The developed VAD can be used to elevate the performance of speech processing applications like speech enhancement that are widely used in hearing aid devices. The classification model generated by AutoML is computationally fast and has minimal processing delay, which enables an efficient, real-time operation on a smartphone. The steps involved in real-time implementation are discussed in detail. The key contribution of this work include the utilization of AutoML platform for hearing aid applications and the realization of AutoML model on smartphone. The experimental analysis and results demonstrate the significance and importance of using the AutoML for the current approach. The evaluations also show improvements over the state of art techniques and reflect the practical usability of the developed smartphone app in different noisy environments.

Alert signals like sirens and home alarms are important as they warn people of precarious situations. This work presents the detection and separation of these acoustically important alert signals, not to be attenuated as noise, to assist the hearing impaired listeners. The proposed method is based on convolutional neural network (CNN) and convolutional-recurrent neural network (CRNN). The developed method consists of two blocks, the detector block, and the separator block. The entire setup is integrated with speech enhancement (SE) algorithms, and before the compression stage, used in a hearing aid device (HAD) signal processing pipeline. The detector recognizes the presence of alert signal in various noisy environments. The separator block separates the alert signal from the mixture of noisy signals before passing it through SE to ensure minimal or no attenuation of the alert signal. It is implemented on a smartphone as an application that seamlessly works with HADs in real-time. This smartphone assistive setup allows the hearing aid users to know the presence of the alert sounds even when these are out of sight. The algorithm is computationally efficient with a low processing delay. The key contribution of this paper includes the development and integration of alert signal separator block with SE and the realization of the entire setup on a smartphone in real-time. The proposed method is compared with several state-of-the-art techniques through objective measures in various noisy conditions. The experimental analysis demonstrates the effectiveness and practical usefulness of the developed setup in real-world noisy scenarios.

This paper presents a Speech Enhancement (SE) technique based on multi-objective learning convolutional neural network to improve the overall quality of speech perceived by Hearing Aid (HA) users. The proposed method is implemented on a smartphone as an application that performs real-time SE. This arrangement works as an assistive tool to HA. A multi-objective learning architecture including primary and secondary features uses a mapping-based convolutional neural network (CNN) model to remove noise from a noisy speech spectrum. The algorithm is computationally fast and has a low processing delay which enables it to operate seamlessly on a smartphone. The steps and the detailed analysis of real-time implementation are discussed. The proposed method is compared with existing conventional and neural network-based SE techniques through speech quality and intelligibility metrics in various noisy speech conditions. The key contribution of this paper includes the realization of CNN SE model on a smartphone processor that works seamlessly with HA. The experimental results demonstrate significant improvements over the state-of-the-art techniques and reflect the usability of the developed SE application in noisy environments.

Conventional Blind Source Separation (BSS) techniques are computationally complex. This is due to the calculation of the demixing matrix for the entire signal or due to the frequent update of the demixing matrix at every time frame index, making them impractical to use in many real-time applications. In this paper, a robust, neural network based two-microphone sound source localization method is used as a criterion to enhance the efficiency of the Independent Vector Analysis (IVA), a BSS method. IVA is used to separate speech and noise sources which are convolutedly mixed. The practical usability of the proposed method is proved by implementing it on a smartphone to separate speech and noise in real-world scenarios for hearing-aid applications. The experimental results with objective and subjective tests reveal the usefulness of the developed method for real-world applications.

In this paper, we present a Speech Enhancement (SE) technique to improve intelligibility of speech perceived by Hearing Aid users using smartphone as an assistive device. We use the formant frequency information to improve the overall quality and intelligibility of the speech. The proposed SE method is based on new super Gaussian joint maximum a Posteriori (SGJMAP) estimator. Using the priori information of formant frequency locations, the derived gain function has “ tradeoff” factors that allows the smartphone user to customize perceptual preference, by controlling the amount of noise suppression and speech distortion in real-time. The formant frequency information helps the hearing aid user to control the gains over the non-formant frequency band, allowing the HA users to attain more noise suppression while maintaining the speech intelligibility using a smartphone application. Objective intelligibility measures and subjective results reflect the usability of the developed SE application in noisy real world acoustic environment.

This paper presents the minimum variance distortionless response (MVDR) beamformer combined with a Speech Enhancement (SE) gain function as a real-time application running on smartphones that work as an assistive device to Hearing Aids. It has been shown that beamforming techniques improve the Signal to Noise Ratio (SNR) in noisy conditions. In the proposed algorithm, MVDR beamformer is used as an SNR booster for the SE method. The proposed SE gain is based on the Log-Spectral Amplitude estimator to improve the speech quality in the presence of different background noises. Objective evaluation and intelligibility measures support the theoretical analysis and show significant improvements of the proposed method in comparison with existing methods. Subjective test results show the effectiveness of the application in real-world noisy conditions at SNR levels of -5 dB, 0 dB, and 5 dB.

In this work, the coherence between speech and noise signals is used to obtain a Speech Enhancement (SE) gain function, in combination with a Super-Gaussian Joint Maximum a Posteriori (SGJMAP) single microphone SE gain function. The proposed SE method is implemented on a smartphone that works as an assistive device to hearing aids in real-time. Although coherence SE gain function suppresses the background noise well, it distorts the speech. In contrary, SE using SGJMAP improves speech quality with the introduction of musical noise, which we contain by using a post filter. The weighted union of these two gain functions strikes a balance between noise suppression and speech distortion. A “weighting” parameter is introduced in the derived gain function that can be controlled by the smartphone user based on different background noises and their comfort level of hearing, which proves the practical usability of the developed SE method implemented on the smartphone. Objective and subjective measures of the proposed method show significant improvements in both quality and intelligibility compared to the state of the art SE techniques considered in this work for several noisy conditions at the signal to noise ratio levels of -5 dB, 0 dB, and 5 dB.

In this paper, we present a Speech Enhancement (SE) method implemented on a smartphone, and this arrangement functions as an assistive device to hearing aids (HA). Many benchmark single channel SE algorithms implemented on HAs provide considerable improvement in speech quality, while speech intelligibility improvement still remains a prime challenge. The proposed SE method based on Log spectral amplitude estimator improves speech intelligibility in the noisy real world acoustic environment using the priori information of formant frequency locations. The formant frequency information avails us to control the amount of speech distortion in these frequency bands, thereby controlling speech distortion. We introduce a `scaling' parameter for the SE gain function, which controls the gains over the non-formant frequency band, allowing the HA users to customize the playback speech using a smartphone application to their listening preference. Objective intelligibility measures show the effectiveness of the proposed SE method. Subjective results reflect the suitability of the developed Speech Enhancement application in real-world noisy conditions at SNR levels of -5 dB, 0 dB and 5 dB.

Multi-channel compression is one among the most popular techniques of hearing aids of mapping the wide dynamic range of speech signals into the reduced dynamic range of hearing impaired listeners effectively. Most of the Multi-Channel Compressors are particularly designed to manage certain loudness pattern, but it's quite difficult to efficiently capture the dynamically changing acoustics in Real-Time. The proposed method circumvents this problem by providing a contraption to estimate the fluctuating sound pressure level (SPL) and converting it into the desired SPL suitable for Multi-channel compression. This objective of stabilizing the SPL is achieved by incorporating a rapid voice activated detection (VAD), a root mean square (RMS) estimator, a moving average (MA) rectifier and a SPL regulator. Proposed SPL Stabilizer provides the hearing aids compressor with a comparatively constant SPL level input in Real-Time, and there by improves the quality and intelligibility of compression output signal.

In this letter, we derive a new super-Gaussian joint maximum a posteriori (SGJMAP) based single microphone speech enhancement gain function. The developed speech enhancement method is implemented on a smartphone, and this arrangement functions as an assistive device to hearing aids. We introduce a tradeoff parameter in the derived gain function that allows the smartphone user to customize their listening preference, by controlling the amount of noise suppression and speech distortion in real-time based on their level of hearing comfort perceived in noisy real-world acoustic environment. Objective quality and intelligibility measures show the effectiveness of the proposed method in comparison to benchmark techniques considered in this letter. Subjective results reflect the usefulness of the developed speech enhancement application in real-world noisy conditions at signal to noise ratio levels of -5, 0, and 5 dB.