Takanobu Nishiura
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Takanobu Nishiura has a total of 38 co-authors. Here are the top co-authors by number of shared publications:
- Kenta Iwai
College of Information Science and Engineering, Ritsumeikan University
8 shared publications - Masato Nakayama
College of Information Science and Engineering, Ritsumeikan University, Osaka 567-8570, Japan
8 shared publications - Yuting Geng
College of Information Science and Engineering, Ritsumeikan University, Osaka 567-8570, Japan
5 shared publications Masato Nakayama
3 shared publications- Yoshiharu Soeta
Molecular Biosystems Research Institute, Life Science and Biotechnology, National Institute of Advanced Industrial Science and Technology (AIST), Ikeda 563-8577, Japan
3 shared publications Binh Thien Nguyen
Ritsumeikan University
2 shared publications
Latest Publications
- Open Access
- 24 June 2026
- Published by Elsevier in SSRN Electronic Journal
Amplitude-Projected Diffusion Posterior Sampling for Speech Phase Reconstruction with Zero-Shot Measurement Guidance
Phase reconstruction from a given amplitude spectrogram is a fundamental but ill-posed problem in speech and audio signal processing. Conventional analytical methods explicitly enforce amplitude consistency but lack a learned speech prior, often producing perceptual artifacts. Diffusion-based methods can model natural speech distributions, but the target amplitude is usually used only as a soft condition and is not explicitly satisfied during sampling. To address this limitation, we propose Amplitude-Projected Diffusion Posterior Sampling (AP-DPS) for speech phase reconstruction. AP-DPS is based on an amplitude-conditioned deterministic diffusion model. At each reverse step, the network predicts a clean waveform candidate, which is projected onto the target amplitude spectrogram by replacing its STFT amplitude with the given amplitude while preserving the estimated phase. The projected clean estimate is then used in the DDIM update. AP-DPS can also incorporate an additional measurement through a DPS-style measurement gradient during inference, without retraining the diffusion model. Experiments on the VCTK corpus show that AP-DPS improves phase reconstruction under both measurement-free and measurement-guided settings. With an additional measurement obtained from a signal sampled at 1200 Hz, all compared methods improve across PESQ, eSTOI, and SNR. AP-DPS achieves the best PESQ in both settings, while diffusion-based methods show higher SNR than analytical methods. However, analytical methods remain stronger in eSTOI. Zero-shot experiments further demonstrate that AP-DPS can exploit low-sampling-rate and noisy-waveform measurements without task-specific retraining.
- 24 June 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessAmplitude-Projected Diffusion Posterior Sampling for Speech Phase Reconstruction with Zero-Shot Measurement Guidance
Phase reconstruction from a given amplitude spectrogram is a fundamental but ill-posed problem in speech and audio signal processing. Conventional analytical methods explicitly enforce amplitude consistency but lack a learned speech prior, often producing perceptual artifacts. Diffusion-based methods can model natural speech distributions, but the target amplitude is usually used only as a soft condition and is not explicitly satisfied during sampling. To address this limitation, we propose Amplitude-Projected Diffusion Posterior Sampling (AP-DPS) for speech phase reconstruction. AP-DPS is based on an amplitude-conditioned deterministic diffusion model. At each reverse step, the network predicts a clean waveform candidate, which is projected onto the target amplitude spectrogram by replacing its STFT amplitude with the given amplitude while preserving the estimated phase. The projected clean estimate is then used in the DDIM update. AP-DPS can also incorporate an additional measurement through a DPS-style measurement gradient during inference, without retraining the diffusion model. Experiments on the VCTK corpus show that AP-DPS improves phase reconstruction under both measurement-free and measurement-guided settings. With an additional measurement obtained from a signal sampled at 1200 Hz, all compared methods improve across PESQ, eSTOI, and SNR. AP-DPS achieves the best PESQ in both settings, while diffusion-based methods show higher SNR than analytical methods. However, analytical methods remain stronger in eSTOI. Zero-shot experiments further demonstrate that AP-DPS can exploit low-sampling-rate and noisy-waveform measurements without task-specific retraining.
- Open Access
Acoustic Metadata Design on Object-Based Audio Using Estimated 3D-Position from Visual Image Toward Depth-Directional Sound Image Localization
Multichannel audio is a sound field reproduction technology that uses multiple loudspeakers. Object-based audio is a playback method for multichannel audio that enables the construction of sound images at specified positions using coordinates within the playback space. However, the sound image positions must be manually specified by audio content creators, which increases the production workload, especially for works containing many sound images or feature films. We have previously proposed a method to reduce the workload of content creators by constructing sound images based on object positions in visual images. However, a significant challenge remains since depth localization of the sound image is not accurate enough. This paper aims to improve localization accuracy by changing the range of sound image movement along the depth direction. To confirm the localization accuracy of sound images constructed using the proposed method, we conducted a subjective evaluation experiment. The experiment identified the optimal movement range by presenting participants with visual images synchronized with sound images moving across varying spatial scales. Consequently, we were able to identify the range of sound image movement in the depth direction necessary for presenting sound images with high consistency with the visual images.
Open AccessAcoustic Metadata Design on Object-Based Audio Using Estimated 3D-Position from Visual Image Toward Depth-Directional Sound Image Localization
Multichannel audio is a sound field reproduction technology that uses multiple loudspeakers. Object-based audio is a playback method for multichannel audio that enables the construction of sound images at specified positions using coordinates within the playback space. However, the sound image positions must be manually specified by audio content creators, which increases the production workload, especially for works containing many sound images or feature films. We have previously proposed a method to reduce the workload of content creators by constructing sound images based on object positions in visual images. However, a significant challenge remains since depth localization of the sound image is not accurate enough. This paper aims to improve localization accuracy by changing the range of sound image movement along the depth direction. To confirm the localization accuracy of sound images constructed using the proposed method, we conducted a subjective evaluation experiment. The experiment identified the optimal movement range by presenting participants with visual images synchronized with sound images moving across varying spatial scales. Consequently, we were able to identify the range of sound image movement in the depth direction necessary for presenting sound images with high consistency with the visual images.
- Open Access
- 28 March 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15 (7) , 3744
Virtual Sound Source Construction Based on Direct-to-Reverberant Ratio Control Using Multiple Pairs of Parametric-Array Loudspeakers and Conventional Loudspeakers
We propose a new method for constructing a virtual sound source (VSS) based on the direct-to-reverberant ratio (DRR) of room impulse responses (RIRs), using multiple pairs of parametric-array loudspeakers (PALs) and conventional loudspeakers (hereafter referred to simply as loudspeakers). In this paper, we focus on the differences in the DRRs of the RIRs generated by PALs and loudspeakers. The DRR of an RIR is recognized as a key cue for distance perception. A PAL can achieve super-directivity using an array of ultrasonic transducers. Its RIR exhibits a high DRR, characterized by a large-amplitude direct wave and low-amplitude reverberations. Consequently, a PAL makes the VSS appear to be closer to the listener. In contrast, a loudspeaker causes the VSS to be perceived as farther away because the sound it emits has a low DRR. The proposed method leverages the differences in the DRRs of the RIRs between PALs and loudspeakers. It controls the perceived distance of the VSS by reproducing the desired DRR at the listener’s position through a weighted combination of the RIRs emitted from PALs and loudspeakers into the air. Additionally, the proposed method adjusts the direction of the VSS using vector-based amplitude panning (VBAP). Finally, we have confirmed the effectiveness of the proposed method through evaluation experiments.
- 28 March 2025
- Published by MDPI AG in Applied Sciences
- Vol. 15 (7) , 3744
Open AccessVirtual Sound Source Construction Based on Direct-to-Reverberant Ratio Control Using Multiple Pairs of Parametric-Array Loudspeakers and Conventional Loudspeakers
We propose a new method for constructing a virtual sound source (VSS) based on the direct-to-reverberant ratio (DRR) of room impulse responses (RIRs), using multiple pairs of parametric-array loudspeakers (PALs) and conventional loudspeakers (hereafter referred to simply as loudspeakers). In this paper, we focus on the differences in the DRRs of the RIRs generated by PALs and loudspeakers. The DRR of an RIR is recognized as a key cue for distance perception. A PAL can achieve super-directivity using an array of ultrasonic transducers. Its RIR exhibits a high DRR, characterized by a large-amplitude direct wave and low-amplitude reverberations. Consequently, a PAL makes the VSS appear to be closer to the listener. In contrast, a loudspeaker causes the VSS to be perceived as farther away because the sound it emits has a low DRR. The proposed method leverages the differences in the DRRs of the RIRs between PALs and loudspeakers. It controls the perceived distance of the VSS by reproducing the desired DRR at the listener’s position through a weighted combination of the RIRs emitted from PALs and loudspeakers into the air. Additionally, the proposed method adjusts the direction of the VSS using vector-based amplitude panning (VBAP). Finally, we have confirmed the effectiveness of the proposed method through evaluation experiments.
- Open Access
- 1 March 2025
- Published by Acoustical Society of Japan in Acoustical Science and Technology
- Vol. 46 (2) , 186-190
DNN-based phase estimation for online speech enhancement
This paper presents a DNN-based phase reconstruction algorithm for online speech enhancement. Although various online phase reconstruction algorithms have been proposed, many of them rely on the structure of the clean amplitude. This restricts their performance in speech enhancement applications, where only noisy observations are available. In contrast, our proposed method directly estimates the clean phase from the noisy observation. Several aspects of phase reconstruction and their effects on speech enhancement are also investigated and discussed. Experimental results confirm that our method performs better than conventional online phase reconstruction methods for speech enhancement in all experimental settings.
- 1 March 2025
- Published by Acoustical Society of Japan in Acoustical Science and Technology
- Vol. 46 (2) , 186-190
Open AccessDNN-based phase estimation for online speech enhancement
This paper presents a DNN-based phase reconstruction algorithm for online speech enhancement. Although various online phase reconstruction algorithms have been proposed, many of them rely on the structure of the clean amplitude. This restricts their performance in speech enhancement applications, where only noisy observations are available. In contrast, our proposed method directly estimates the clean phase from the noisy observation. Several aspects of phase reconstruction and their effects on speech enhancement are also investigated and discussed. Experimental results confirm that our method performs better than conventional online phase reconstruction methods for speech enhancement in all experimental settings.
- Open Access
- 1 January 2025
- Published by Acoustical Society of Japan in Acoustical Science and Technology
- p. e25.95
Multichannel feedforward active noise control system with an optical laser microphone and a distant air-conduction microphone for noise reduction in a real system
A conventional feedforward active noise control system suffers from degraded performance of noise reduction due to the causality constraint caused by processing and propagation delays. To address this issue, we propose a multichannel feedforward active noise control system that combines an optical laser microphone and an air-conduction microphone. The proposed system relaxes the causality constraint by acquiring vibration information at the speed of light using the optical laser microphone and enhances coherence using the signal of the air-conduction microphone. Experiments using a laser Doppler vibrometer and an electret condenser microphone demonstrate superior noise reduction compared to the conventional system.
- 1 January 2025
- Published by Acoustical Society of Japan in Acoustical Science and Technology
- p. e25.95
Open AccessMultichannel feedforward active noise control system with an optical laser microphone and a distant air-conduction microphone for noise reduction in a real system
A conventional feedforward active noise control system suffers from degraded performance of noise reduction due to the causality constraint caused by processing and propagation delays. To address this issue, we propose a multichannel feedforward active noise control system that combines an optical laser microphone and an air-conduction microphone. The proposed system relaxes the causality constraint by acquiring vibration information at the speed of light using the optical laser microphone and enhances coherence using the signal of the air-conduction microphone. Experiments using a laser Doppler vibrometer and an electret condenser microphone demonstrate superior noise reduction compared to the conventional system.
- Open Access
- 19 December 2024
- Published by MDPI AG in Applied Sciences
- Vol. 14 (24) , 11911
Enhanced Virtual Sound Source Construction Based on Wave Field Synthesis Using Crossfade Processing with Electro-Dynamic and Parametric Loudspeaker Arrays
Wave field synthesis (WFS) can be used to construct virtual sound sources (VSSs) with a loudspeaker array. Conventional methods using a single type of loudspeaker showed limited performance in distance perception. For example, WFS with electro-dynamic loudspeakers (EDLs) has the advantage of constructing VSSs near the loudspeaker, while WFS with parametric array loudspeakers (PALs) has the advantage of constructing VSSs far from the loudspeaker. In this paper, we propose a VSS construction method utilizing crossfade processing with both EDLs and PALs. The contribution of EDLs and PALs was balanced to better synthesize the target sound field. We carried out experiments to evaluate the sound pressure, frequency characteristic, and sound image perception. The experimental results demonstrated that the proposed method can enhance these aspects of the VSS.
- 19 December 2024
- Published by MDPI AG in Applied Sciences
- Vol. 14 (24) , 11911
Open AccessEnhanced Virtual Sound Source Construction Based on Wave Field Synthesis Using Crossfade Processing with Electro-Dynamic and Parametric Loudspeaker Arrays
Wave field synthesis (WFS) can be used to construct virtual sound sources (VSSs) with a loudspeaker array. Conventional methods using a single type of loudspeaker showed limited performance in distance perception. For example, WFS with electro-dynamic loudspeakers (EDLs) has the advantage of constructing VSSs near the loudspeaker, while WFS with parametric array loudspeakers (PALs) has the advantage of constructing VSSs far from the loudspeaker. In this paper, we propose a VSS construction method utilizing crossfade processing with both EDLs and PALs. The contribution of EDLs and PALs was balanced to better synthesize the target sound field. We carried out experiments to evaluate the sound pressure, frequency characteristic, and sound image perception. The experimental results demonstrated that the proposed method can enhance these aspects of the VSS.
- Open Access
- 13 November 2024
- Published by MDPI AG in Applied Sciences
- Vol. 14 (22) , 10467
Comfortable Sound Design Based on Auditory Masking with Chord Progression and Melody Generation Corresponding to the Peak Frequencies of Dental Treatment Noises
Noise reduction methods have been proposed for various loud noises. However, in a quiet indoor environment, even small noises often cause discomfort. One of the small noises that causes discomfort is noise with resonant frequencies. Since resonant frequencies are often high frequencies, it is difficult to apply conventional active noise control methods to them. To solve this problem, we focused on auditory masking, a phenomenon in which synthesized sounds increase the audible threshold. We have performed several studies on reducing discomfort based on auditory masking. However, it was difficult for comfortable sound design to be achieved using the previously proposed methods, even though they were able to reduce feelings of discomfort. Here, we focus on a pleasant sound: music. Comfortable sound design is made possible by introducing music theory into the design of masker signals. In this paper, we therefore propose comfortable sound design based on auditory masking with chord progression and melody generation to match the peak frequencies of dental treatment noises.
- 13 November 2024
- Published by MDPI AG in Applied Sciences
- Vol. 14 (22) , 10467
Open AccessComfortable Sound Design Based on Auditory Masking with Chord Progression and Melody Generation Corresponding to the Peak Frequencies of Dental Treatment Noises
Noise reduction methods have been proposed for various loud noises. However, in a quiet indoor environment, even small noises often cause discomfort. One of the small noises that causes discomfort is noise with resonant frequencies. Since resonant frequencies are often high frequencies, it is difficult to apply conventional active noise control methods to them. To solve this problem, we focused on auditory masking, a phenomenon in which synthesized sounds increase the audible threshold. We have performed several studies on reducing discomfort based on auditory masking. However, it was difficult for comfortable sound design to be achieved using the previously proposed methods, even though they were able to reduce feelings of discomfort. Here, we focus on a pleasant sound: music. Comfortable sound design is made possible by introducing music theory into the design of masker signals. In this paper, we therefore propose comfortable sound design based on auditory masking with chord progression and melody generation to match the peak frequencies of dental treatment noises.
- Open Access
- 1 November 2024
- Published by Research Institute of Signal Processing, Japan in Journal of Signal Processing
- Vol. 28 (6) , 267-275
Sound-Field Reproduction Based on Virtual Early Reflections Using Parametric and Electrodynamic Loudspeakers
Highly realistic sound-field reproduction systems have been attracting attention owing to their ability to reproduce a sound field. These systems commonly use electrodynamic loudspeakers (EDLs) to construct sound images. The directivity of EDLs is broad, so the constructed sound images become diffused owing to the reverberation characteristics of the room. Therefore, sharp sound-image construction requires a large, complex system. As a more feasible approach, we focus on parametric array loudspeakers (PALs). The directivity of PALs is narrow, so the constructed sound image is sharp. Therefore, we propose a sound-field reproduction system that involves using a PAL to construct a sharp sound image and EDLs to reproduce the sensation of reverberation. To reproduce the sensation of reverberation, the early reflections of the target sound field are calculated using the mirror-image method, and virtual early reflections are produced. Thus, the sharpness of the sound image and the sensation of reverberation can be easily controlled to reproduce a highly realistic sound field. We conducted objective evaluation experiments to verify the effectiveness of the proposed system.
- 1 November 2024
- Published by Research Institute of Signal Processing, Japan in Journal of Signal Processing
- Vol. 28 (6) , 267-275
Open AccessSound-Field Reproduction Based on Virtual Early Reflections Using Parametric and Electrodynamic Loudspeakers
Highly realistic sound-field reproduction systems have been attracting attention owing to their ability to reproduce a sound field. These systems commonly use electrodynamic loudspeakers (EDLs) to construct sound images. The directivity of EDLs is broad, so the constructed sound images become diffused owing to the reverberation characteristics of the room. Therefore, sharp sound-image construction requires a large, complex system. As a more feasible approach, we focus on parametric array loudspeakers (PALs). The directivity of PALs is narrow, so the constructed sound image is sharp. Therefore, we propose a sound-field reproduction system that involves using a PAL to construct a sharp sound image and EDLs to reproduce the sensation of reverberation. To reproduce the sensation of reverberation, the early reflections of the target sound field are calculated using the mirror-image method, and virtual early reflections are produced. Thus, the sharpness of the sound image and the sensation of reverberation can be easily controlled to reproduce a highly realistic sound field. We conducted objective evaluation experiments to verify the effectiveness of the proposed system.
- Open Access
- 14 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 152036-152044
Joint Deep Neural Network for Single-Channel Speech Separation on Masking-Based Training Targets
Single-channel speech separation can be adopted in many applications. Time-frequency (T-F) masking is an effective method for single-channel speech separation. With advancements in deep learning, T-F masks have become used as a training target, achieving notable separation results. Among the numerous masks that have been proposed, the ideal binary mask (IBM), ideal ratio mask (IRM), Wiener filter (WF) and spectral magnitude mask (SMM) are commonly used and have proven effective, though their separation performance varies depending on the speech mixture and separation model. The existing approach mainly utilizes a single network to approximate the mask of the target speech. However, in mixed speech, there are segments where speech is mixed with other speech, segments where speech is mixed with silent intervals, and segments where high signal-to-noise ratio (SNR) speech is mixed due to pauses and variations in the speakers’ intonation and emphasis. In this paper, we attempt to use different networks to handle speech segments containing various mixtures. In addition to the existing network, we introduce a network (using the Rectified Linear Unit as activation functions) to specifically address segments containing a mixture of speech and silence, as well as segments with high SNR speech mixtures. We conducted evaluation experiments on the speech separation of two speakers using the four aforementioned masks as training targets. The performance improvements observed in the evaluation experiments demonstrate the effectiveness of our proposed method based on the joint network compared to the conventional method based on the single network.
- 14 October 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 152036-152044
Open AccessJoint Deep Neural Network for Single-Channel Speech Separation on Masking-Based Training Targets
Single-channel speech separation can be adopted in many applications. Time-frequency (T-F) masking is an effective method for single-channel speech separation. With advancements in deep learning, T-F masks have become used as a training target, achieving notable separation results. Among the numerous masks that have been proposed, the ideal binary mask (IBM), ideal ratio mask (IRM), Wiener filter (WF) and spectral magnitude mask (SMM) are commonly used and have proven effective, though their separation performance varies depending on the speech mixture and separation model. The existing approach mainly utilizes a single network to approximate the mask of the target speech. However, in mixed speech, there are segments where speech is mixed with other speech, segments where speech is mixed with silent intervals, and segments where high signal-to-noise ratio (SNR) speech is mixed due to pauses and variations in the speakers’ intonation and emphasis. In this paper, we attempt to use different networks to handle speech segments containing various mixtures. In addition to the existing network, we introduce a network (using the Rectified Linear Unit as activation functions) to specifically address segments containing a mixture of speech and silence, as well as segments with high SNR speech mixtures. We conducted evaluation experiments on the speech separation of two speakers using the four aforementioned masks as training targets. The performance improvements observed in the evaluation experiments demonstrate the effectiveness of our proposed method based on the joint network compared to the conventional method based on the single network.
- Open Access
- 4 October 2024
- Published by MDPI AG in Information
- Vol. 15 (10) , 608
Threshold-Based Combination of Ideal Binary Mask and Ideal Ratio Mask for Single-Channel Speech Separation
An effective approach to addressing the speech separation problem is utilizing a time–frequency (T-F) mask. The ideal binary mask (IBM) and ideal ratio mask (IRM) have long been widely used to separate speech signals. However, the IBM is better at improving speech intelligibility, while the IRM is better at improving speech quality. To leverage their respective strengths and overcome weaknesses, we propose an ideal threshold-based mask (ITM) to combine these two masks. By adjusting two thresholds, these two masks are combined to jointly act on speech separation. We list the impact of using different threshold combinations on speech separation performance under ideal conditions and discuss a reasonable range for fine tuning the thresholds. By using masks as a training target, to evaluate the effectiveness of the proposed method, we conducted supervised speech separation experiments applying a deep neural network (DNN) and long short-term memory (LSTM), the results of which were measured by three objective indicators: the signal-to-distortion ratio (SDR), signal-to-interference ratio (SIR), and signal-to-artifact ratio improvement (SAR). Experimental results show that the proposed mask combines the strengths of the IBM and IRM and implies that the accuracy of speech separation can potentially be further improved by effectively leveraging the advantages of different masks.
- 4 October 2024
- Published by MDPI AG in Information
- Vol. 15 (10) , 608
Open AccessThreshold-Based Combination of Ideal Binary Mask and Ideal Ratio Mask for Single-Channel Speech Separation
An effective approach to addressing the speech separation problem is utilizing a time–frequency (T-F) mask. The ideal binary mask (IBM) and ideal ratio mask (IRM) have long been widely used to separate speech signals. However, the IBM is better at improving speech intelligibility, while the IRM is better at improving speech quality. To leverage their respective strengths and overcome weaknesses, we propose an ideal threshold-based mask (ITM) to combine these two masks. By adjusting two thresholds, these two masks are combined to jointly act on speech separation. We list the impact of using different threshold combinations on speech separation performance under ideal conditions and discuss a reasonable range for fine tuning the thresholds. By using masks as a training target, to evaluate the effectiveness of the proposed method, we conducted supervised speech separation experiments applying a deep neural network (DNN) and long short-term memory (LSTM), the results of which were measured by three objective indicators: the signal-to-distortion ratio (SDR), signal-to-interference ratio (SIR), and signal-to-artifact ratio improvement (SAR). Experimental results show that the proposed mask combines the strengths of the IBM and IRM and implies that the accuracy of speech separation can potentially be further improved by effectively leveraging the advantages of different masks.