Rintaro Ikeshita
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Rintaro Ikeshita has a total of 24 co-authors. Here are the top co-authors by number of shared publications:
- Tomohiro Nakatani
NTT Communication Science Laboratories, NTT Corporation, Kyoto, Japan
13 shared publications - Shoko Araki
NTT Communication Science Laboratories, NTT Corporation, Kyoto, Japan
7 shared publications - Marc Delcroix
NTT Corporation
4 shared publications - Tsubasa Ochiai
NTT Communication Science Laboratories, NTT Corporation, Kyoto, Japan
4 shared publications - Keisuke Kinoshita
NTT Communication Science Labs., Nippon Telegraph and Telephone Corp., Kyoto, Japan
3 shared publications - Hiroshi Sawada
Communication Science Laboratories, NTT Inc., Kyoto-Fu, Japan
2 shared publications
Latest Publications
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 33, 4336-4340
Auxiliary-Function-Based Dynamic Independent Vector Analysis for Blind Extraction of a Single Moving Source
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 33, 4336-4340
Auxiliary-Function-Based Dynamic Independent Vector Analysis for Blind Extraction of a Single Moving Source
- 29 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 33, 2259-2274
Maximizing Predicted Signal-to-Distortion Ratio: A New Microphone Selection Criterion for Beamforming in Acoustic Sensor Networks
This paper addresses the problem of selecting an effective subset of microphones in acoustic sensor networks (ASNs) for speech enhancement applications. A basic approach to this problem is to select a subset of microphones such that the output signal-to-noise ratio (oSNR) of the beamforming output signal is maximized. However, oSNR does not necessarily correlate well with widely used signal quality measures such as the signal-to-distortion ratio (SDR), perceptual evaluation of speech quality (PESQ), and short-time objective intelligibility (STOI). We here introduce a new measure that predicts SDR for beamforming output signals. This measure, termed predicted SDR (pSDR), demonstrates a better correlation with SDR, PESQ, and STOI compared to oSNR. Whereas the original SDR requires an oracle reference signal for measurements, the proposed pSDR is defined in the short-time Fourier transform domain and can be estimated solely from source covariance matrices already used for obtaining beamformers. Therefore, our pSDR can be immediately used as a measure for selecting microphones instead of oSNR. We also extend conventional subset selection methods to jointly select a reference microphone of beamformers in a unified optimization problem. This extension is crucial because the choice of the reference microphone can significantly affect the quality of enhanced signals in ASNs. Numerical experiments suggest that, as a measure for selecting both a microphone subset and a reference microphone, pSDR is generally better than, or at least comparable to, oSNR in terms of SDR, PESQ, and STOI. We also demonstrate the effectiveness of selecting a microphone subset jointly with a reference microphone of beamformers.
- 29 May 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 33, 2259-2274
Maximizing Predicted Signal-to-Distortion Ratio: A New Microphone Selection Criterion for Beamforming in Acoustic Sensor Networks
This paper addresses the problem of selecting an effective subset of microphones in acoustic sensor networks (ASNs) for speech enhancement applications. A basic approach to this problem is to select a subset of microphones such that the output signal-to-noise ratio (oSNR) of the beamforming output signal is maximized. However, oSNR does not necessarily correlate well with widely used signal quality measures such as the signal-to-distortion ratio (SDR), perceptual evaluation of speech quality (PESQ), and short-time objective intelligibility (STOI). We here introduce a new measure that predicts SDR for beamforming output signals. This measure, termed predicted SDR (pSDR), demonstrates a better correlation with SDR, PESQ, and STOI compared to oSNR. Whereas the original SDR requires an oracle reference signal for measurements, the proposed pSDR is defined in the short-time Fourier transform domain and can be estimated solely from source covariance matrices already used for obtaining beamformers. Therefore, our pSDR can be immediately used as a measure for selecting microphones instead of oSNR. We also extend conventional subset selection methods to jointly select a reference microphone of beamformers in a unified optimization problem. This extension is crucial because the choice of the reference microphone can significantly affect the quality of enhanced signals in ASNs. Numerical experiments suggest that, as a measure for selecting both a microphone subset and a reference microphone, pSDR is generally better than, or at least comparable to, oSNR in terms of SDR, PESQ, and STOI. We also demonstrate the effectiveness of selecting a microphone subset jointly with a reference microphone of beamformers.
- Open Access
- 1 April 2025
- Published by Institute of Electronics, Information and Communications Engineers (IEICE) in IEICE ESS Fundamentals Review
- Vol. 18 (4) , 267-278
Listening to Speech in Mixture: Advances in Source Separation and Target Speech Extraction
複数の音声やそのほかの音が混ざって収録された音響信号から,個々の音を分離して抽出する音源分離,及び特定の話者の音声のみを抽出する目的音声抽出について,最新の技術動向を解説する.これらの技術は,人にとって音声をより聞き取りやすくするだけでなく,後段の音声アプリケーションの性能向上にも寄与する.二つの重要なアプローチとして,信号のモデルに基づく方法とニューラルネットワークに基づく方法をとりあげる.各アプローチの概要と特徴を述べ,代表的な技術として,残響環境下でのブラインド音源分離と声の特徴に基づく目的音声抽出について詳しく紹介する.最後に,この技術分野の今後の展望についても触れる.
- 1 April 2025
- Published by Institute of Electronics, Information and Communications Engineers (IEICE) in IEICE ESS Fundamentals Review
- Vol. 18 (4) , 267-278
Open AccessListening to Speech in Mixture: Advances in Source Separation and Target Speech Extraction
複数の音声やそのほかの音が混ざって収録された音響信号から,個々の音を分離して抽出する音源分離,及び特定の話者の音声のみを抽出する目的音声抽出について,最新の技術動向を解説する.これらの技術は,人にとって音声をより聞き取りやすくするだけでなく,後段の音声アプリケーションの性能向上にも寄与する.二つの重要なアプローチとして,信号のモデルに基づく方法とニューラルネットワークに基づく方法をとりあげる.各アプローチの概要と特徴を述べ,代表的な技術として,残響環境下でのブラインド音源分離と声の特徴に基づく目的音声抽出について詳しく紹介する.最後に,この技術分野の今後の展望についても触れる.
- 22 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 32 (23299290) , 3589-3602
Rethinking Processing Distortions: Disentangling the Impact of Speech Enhancement Errors on Speech Recognition Performance
It is challenging to improve automatic speech recognition (ASR) performance in noisy conditions with a single-channel speech enhancement (SE) front-end. This is generally attributed to the processing distortions caused by the nonlinear processing of single-channel SE front-ends. However, the causes of such degraded ASR performance have not been fully investigated. How to design single-channel SE front-ends in a way that significantly improves ASR performance remains an open research question. In this study, we investigate a signal-level numerical metric that can explain the cause of degradation in ASR performance. To this end, we propose a novel analysis scheme based on the orthogonal projection-based decomposition of SE errors. This scheme manually modifies the ratio of the decomposed interference, noise, and artifact errors, and it enables us to directly evaluate the impact of each error type on ASR performance. Our analysis reveals the particularly detrimental effect of artifact errors on ASR performance compared to the other types of errors. This provides us with a more principled definition of processing distortions that cause the ASR performance degradation. Then, we study two practical approaches for reducing the impact of artifact errors. First, we prove that the simple observation adding (OA) post-processing (i.e., interpolating the enhanced and observed signals) can improve the signal-to-artifact ratio. Second, we propose a novel training objective, called artifact-boosted signal-to-distortion ratio (AB-SDR), which forces the model to estimate the enhanced signals with fewer artifact errors. Through experiments, we confirm that both the OA and AB-SDR approaches are effective in decreasing artifact errors caused by single-channel SE front-ends, allowing them to significantly improve ASR performance.
- 22 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 32 (23299290) , 3589-3602
Rethinking Processing Distortions: Disentangling the Impact of Speech Enhancement Errors on Speech Recognition Performance
It is challenging to improve automatic speech recognition (ASR) performance in noisy conditions with a single-channel speech enhancement (SE) front-end. This is generally attributed to the processing distortions caused by the nonlinear processing of single-channel SE front-ends. However, the causes of such degraded ASR performance have not been fully investigated. How to design single-channel SE front-ends in a way that significantly improves ASR performance remains an open research question. In this study, we investigate a signal-level numerical metric that can explain the cause of degradation in ASR performance. To this end, we propose a novel analysis scheme based on the orthogonal projection-based decomposition of SE errors. This scheme manually modifies the ratio of the decomposed interference, noise, and artifact errors, and it enables us to directly evaluate the impact of each error type on ASR performance. Our analysis reveals the particularly detrimental effect of artifact errors on ASR performance compared to the other types of errors. This provides us with a more principled definition of processing distortions that cause the ASR performance degradation. Then, we study two practical approaches for reducing the impact of artifact errors. First, we prove that the simple observation adding (OA) post-processing (i.e., interpolating the enhanced and observed signals) can improve the signal-to-artifact ratio. Second, we propose a novel training objective, called artifact-boosted signal-to-distortion ratio (AB-SDR), which forces the model to estimate the enhanced signals with fewer artifact errors. Through experiments, we confirm that both the OA and AB-SDR approaches are effective in decreasing artifact errors caused by single-channel SE front-ends, allowing them to significantly improve ASR performance.
- 11 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Signal Processing
- Vol. 72 (1053587X) , 1560-1575
Geometrically-Regularized Fast Independent Vector Extraction by Pure Majorization-Minimization
We propose computationally efficient algorithms for extracting a single source of interest (SOI) using geometrically-regularized independent vector extraction (GR-IVE). Conventional GR-IVE relies on a block majorization-minimization (block MM) algorithm, which successively optimizes each part (block) of the separation matrix based on the minimization of a surrogate function. We here extend the block MM to a pure MM by developing a global optimization algorithm for minimizing the same surrogate function when the number of SOI is one. Our global optimization is reduced to finding the minimum solution of the so-called secular equation. We also study an orthonormal approach to GR-IVE, i.e., when the feasible region is restricted to unitary matrices. For this problem, we also develop a pure MM algorithm for the single SOI case. Numerical experiments for audio source extraction demonstrate that our proposed MM algorithms converge consistently faster than the conventional block MM algorithm for GR-IVE. Also, the proposed orthonormal approach improves the separation accuracy if the regularization parameter is chosen appropriately.
- 11 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Signal Processing
- Vol. 72 (1053587X) , 1560-1575
Geometrically-Regularized Fast Independent Vector Extraction by Pure Majorization-Minimization
We propose computationally efficient algorithms for extracting a single source of interest (SOI) using geometrically-regularized independent vector extraction (GR-IVE). Conventional GR-IVE relies on a block majorization-minimization (block MM) algorithm, which successively optimizes each part (block) of the separation matrix based on the minimization of a surrogate function. We here extend the block MM to a pure MM by developing a global optimization algorithm for minimizing the same surrogate function when the number of SOI is one. Our global optimization is reduced to finding the minimum solution of the so-called secular equation. We also study an orthonormal approach to GR-IVE, i.e., when the feasible region is restricted to unitary matrices. For this problem, we also develop a pure MM algorithm for the single SOI case. Numerical experiments for audio source extraction demonstrate that our proposed MM algorithms converge consistently faster than the conventional block MM algorithm for GR-IVE. Also, the proposed orthonormal approach improves the separation accuracy if the regularization parameter is chosen appropriately.
- 9 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 32 (23299290) , 1157-1172
Blind and Spatially-Regularized Online Joint Optimization of Source Separation, Dereverberation, and Noise Reduction
This paper proposes a computationally efficient joint optimization algorithm that performs online source separation, dereverberation, and noise reduction based on blind and spatially-regularized processing. When applying such online Blind Source Separation (BSS) as online Independent Vector Extraction (IVE) to a speech application, we must focus on the trade-off between the algorithmic delay and separation accuracy, both of which depend on the analysis frame length. In addition, to separate the sources with specified source permutation, researchers introduced spatial regularization based on the Directions-of-Arrival (DOAs) of the sources into IVE. However, the scale ambiguity of IVE often makes the spatial regularization work inappropriately. To solve these problems, we first propose a blind online joint optimization algorithm of IVE and weighted prediction error dereverberation (WPE). This online algorithm can achieve accurate separation even using short analysis frames because reverberation can be reduced using WPE. We then extend the online joint optimization with robust spatial regularization. We reveal that regularizing the scale of the separated signals is very effective in making the DOA-based spatial regularization work reliably. Our experiments confirm that our blind online joint optimization algorithm can significantly improve the separation accuracy with an algorithmic delay of 8 ms. In addition, we confirm that the proposed spatially-regularized online joint optimization algorithm reduces the rate of the source permutation error to zero percent.
- 9 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 32 (23299290) , 1157-1172
Blind and Spatially-Regularized Online Joint Optimization of Source Separation, Dereverberation, and Noise Reduction
This paper proposes a computationally efficient joint optimization algorithm that performs online source separation, dereverberation, and noise reduction based on blind and spatially-regularized processing. When applying such online Blind Source Separation (BSS) as online Independent Vector Extraction (IVE) to a speech application, we must focus on the trade-off between the algorithmic delay and separation accuracy, both of which depend on the analysis frame length. In addition, to separate the sources with specified source permutation, researchers introduced spatial regularization based on the Directions-of-Arrival (DOAs) of the sources into IVE. However, the scale ambiguity of IVE often makes the spatial regularization work inappropriately. To solve these problems, we first propose a blind online joint optimization algorithm of IVE and weighted prediction error dereverberation (WPE). This online algorithm can achieve accurate separation even using short analysis frames because reverberation can be reduced using WPE. We then extend the online joint optimization with robust spatial regularization. We reveal that regularizing the scale of the separated signals is very effective in making the DOA-based spatial regularization work reliably. Our experiments confirm that our blind online joint optimization algorithm can significantly improve the separation accuracy with an algorithmic delay of 8 ms. In addition, we confirm that the proposed spatially-regularized online joint optimization algorithm reduces the rate of the source permutation error to zero percent.
- 8 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 31 (23299290) , 3589-3602
Multi-Frame Full-Rank Spatial Covariance Analysis for Underdetermined Blind Source Separation and Dereverberation
Full-rank spatial covariance analysis (FCA) is a technique for blind source separation (BSS), and can be applied to underdetermined situations where the sources outnumber the microphones. This paper proposes multi-frame FCA as an extension of FCA to improve the BSS performance when the room reverberations are not so short that multiple time frames are needed to cover the dominant parts of the reverberations. There has already been proposed an FCA model that considers delayed source components. However, the existing FCA model does not take the correlation between different time frames into account. In contrast, our new extension models multiple time frames with multivariate Gaussian distributions of larger dimensionality than the existing FCA models, aiming to better model the source components spanning multiple time frames. We derive an expectation-maximization (EM) algorithm to optimize the model parameters. Experimental results show that the proposed multi-frame FCA performed clearly better than the existing FCA techniques in BSS tasks and also joint BSS and blind dereverberation tasks.
- 8 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 31 (23299290) , 3589-3602
Multi-Frame Full-Rank Spatial Covariance Analysis for Underdetermined Blind Source Separation and Dereverberation
Full-rank spatial covariance analysis (FCA) is a technique for blind source separation (BSS), and can be applied to underdetermined situations where the sources outnumber the microphones. This paper proposes multi-frame FCA as an extension of FCA to improve the BSS performance when the room reverberations are not so short that multiple time frames are needed to cover the dominant parts of the reverberations. There has already been proposed an FCA model that considers delayed source components. However, the existing FCA model does not take the correlation between different time frames into account. In contrast, our new extension models multiple time frames with multivariate Gaussian distributions of larger dimensionality than the existing FCA models, aiming to better model the source components spanning multiple time frames. We derive an expectation-maximization (EM) algorithm to optimize the model parameters. Experimental results show that the proposed multi-frame FCA performed clearly better than the existing FCA techniques in BSS tasks and also joint BSS and blind dereverberation tasks.
- Open Access
How Bad Are Artifacts?: Analyzing the Impact of Speech Enhancement Errors on ASR
It is challenging to improve automatic speech recognition (ASR) performance in noisy conditions with single-channel speech enhancement (SE). In this paper, we investigate the causes of ASR performance degradation by decomposing the SE errors using orthogonal projection-based decomposition (OPD). OPD decomposes the SE errors into noise and artifact components. The artifact component is defined as the SE error signal that cannot be represented as a linear combination of speech and noise sources. We propose manually scaling the error components to analyze their impact on ASR. We experimentally identify the artifact component as the main cause of performance degradation, and we find that mitigating the artifact can greatly improve ASR performance. Furthermore, we demonstrate that the simple observation adding (OA) technique (i.e., adding a scaled version of the observed signal to the enhanced speech) can monotonically increase the signal-to-artifact ratio under a mild condition. Accordingly, we experimentally confirm that OA improves ASR performance for both simulated and real recordings. The findings of this paper provide a better understanding of the influence of SE errors on ASR and open the door to future research on novel approaches for designing effective single-channel SE front-ends for ASR.
Open AccessHow Bad Are Artifacts?: Analyzing the Impact of Speech Enhancement Errors on ASR
It is challenging to improve automatic speech recognition (ASR) performance in noisy conditions with single-channel speech enhancement (SE). In this paper, we investigate the causes of ASR performance degradation by decomposing the SE errors using orthogonal projection-based decomposition (OPD). OPD decomposes the SE errors into noise and artifact components. The artifact component is defined as the SE error signal that cannot be represented as a linear combination of speech and noise sources. We propose manually scaling the error components to analyze their impact on ASR. We experimentally identify the artifact component as the main cause of performance degradation, and we find that mitigating the artifact can greatly improve ASR performance. Furthermore, we demonstrate that the simple observation adding (OA) technique (i.e., adding a scaled version of the observed signal to the enhanced speech) can monotonically increase the signal-to-artifact ratio under a mild condition. Accordingly, we experimentally confirm that OA improves ASR performance for both simulated and real recordings. The findings of this paper provide a better understanding of the influence of SE errors on ASR and open the door to future research on novel approaches for designing effective single-channel SE front-ends for ASR.
- 1 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 30 (23299290) , 1032-1047
Switching Independent Vector Analysis and its Extension to Blind and Spatially Guided Convolutional Beamforming Algorithms
This paper develops a framework that can accurately perform denoising, dereverberation, and source separation using a relatively small number of microphones. It has been empirically confirmed that Independent Vector Analysis (IVA) can blindly separate N sources from their sound mixture even with diffuse noise when a sufficiently large number (= M) of microphones are available (i.e., M N). However, the estimation accuracy is seriously degraded when the number of microphones, or more specifically M N (0), decreases. To overcome this IVA limitation, we propose switching IVA (swIVA) in this paper. With swIVA, the time frames of an observed signal with time-varying characteristics are clustered into several groups, each of which can be well handled by IVA with a small number of microphones, and thus accurate estimation can be achieved by individually applying IVA to each group. Conventionally, a switching mechanism was introduced into a Minimum-Variance Distortionless Response (MVDR) beamformer, and this paper extends the mechanism to work with a blind source separation algorithm. To incorporate dereverberation capability, we further extend swIVA to a blind Convolutional beamforming algorithm (swCIVA) that integrates swIVA and switching Weighted Prediction Error-based dereverberation (swWPE) in a jointly optimal way. With swCIVA, two different time-varying characteristics of an observed signal are captured for dereverberation and source separation to achieve effective estimation. We show that both swIVA and swCIVA can be optimized effectively based on blind signal processing, and their performance can be further improved using a spatial guide for initialization. Experiments demonstrate that both the proposed methods largely outperformed conventional IVA and its convolutional beamforming extension (CIVA) in terms of objective signal quality and automatic speech recognition scores when using relatively few microphones.
- 1 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 30 (23299290) , 1032-1047
Switching Independent Vector Analysis and its Extension to Blind and Spatially Guided Convolutional Beamforming Algorithms
This paper develops a framework that can accurately perform denoising, dereverberation, and source separation using a relatively small number of microphones. It has been empirically confirmed that Independent Vector Analysis (IVA) can blindly separate N sources from their sound mixture even with diffuse noise when a sufficiently large number (= M) of microphones are available (i.e., M N). However, the estimation accuracy is seriously degraded when the number of microphones, or more specifically M N (0), decreases. To overcome this IVA limitation, we propose switching IVA (swIVA) in this paper. With swIVA, the time frames of an observed signal with time-varying characteristics are clustered into several groups, each of which can be well handled by IVA with a small number of microphones, and thus accurate estimation can be achieved by individually applying IVA to each group. Conventionally, a switching mechanism was introduced into a Minimum-Variance Distortionless Response (MVDR) beamformer, and this paper extends the mechanism to work with a blind source separation algorithm. To incorporate dereverberation capability, we further extend swIVA to a blind Convolutional beamforming algorithm (swCIVA) that integrates swIVA and switching Weighted Prediction Error-based dereverberation (swWPE) in a jointly optimal way. With swCIVA, two different time-varying characteristics of an observed signal are captured for dereverberation and source separation to achieve effective estimation. We show that both swIVA and swCIVA can be optimized effectively based on blind signal processing, and their performance can be further improved using a spatial guide for initialization. Experiments demonstrate that both the proposed methods largely outperformed conventional IVA and its convolutional beamforming extension (CIVA) in terms of objective signal quality and automatic speech recognition scores when using relatively few microphones.