Weitao Yuan
Research Subjects
Weitao Yuan has a total of 30 co-authors. Here are the top co-authors by number of shared publications:
- Shengbei Wang
Tiangong University,Tianjin Key Laboratory of Autonomous Intelligence Technology and Systems,China
10 shared publications - Masashi Unoki
Japan Advanced Institute of Science and Technology, Nomi, Ishikawa, Japan
9 shared publications Shengbei Wang
6 shared publicationsJianming Wang
5 shared publications- Wenwu Wang
School of Computer Science and Electronic Engineering, University of Surrey, Guildford GU2 7XH, UK
5 shared publications Masashi Unoki
Japan Advanced Institute of Science and Technology
4 shared publications
Latest Publications
- 4 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 31 (23299290) , 3206-3220
Unsupervised Deep Unfolded Representation Learning for Singing Voice Separation
Learning effective vocal representations from a waveform mixture is a crucial but challenging task for deep neural network (DNN)-based singing voice separation (SVS). Successful representation learning (RL) depends heavily on well-designed neural architectures and effective general priors. However, DNNs for RL in SVS are mostly built on generic architectures without general priors being systematically considered. To address these issues, we introduce deep unfolding to RL and propose two RL-based models for SVS, deep unfolded representation learning (DURL) and optimal transport DURL (OT-DURL). In both models, we formulate RL as a sequence of optimization problems for signal reconstruction, where three general priors, synthesis, non-negative, and our novel analysis, are incorporated. In DURL and OT-DURL, we take different approaches in penalizing the analysis prior. DURL uses the Euclidean distance as its penalty, while OT-DURL uses a more sophisticated penalty known as the OT distance. We address the optimization problems in DURL and OT-DURL with the first-order operator splitting algorithm and unfold the obtained iterative algorithms to novel encoders, by mapping the synthesis/analysis/non-negative priors to different interpretable sublayers of the encoders. We evaluated these DURL and OT-DURL encoders in the unsupervised informed SVS and supervised Open-Unmix frameworks. Experimental results indicate that (1) the OT-DURL encoder is better than the DURL encoder and (2) both encoders can considerably improve the vocal-signal-separation performance compared with those of the baseline model.
- 4 August 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 31 (23299290) , 3206-3220
Unsupervised Deep Unfolded Representation Learning for Singing Voice Separation
Learning effective vocal representations from a waveform mixture is a crucial but challenging task for deep neural network (DNN)-based singing voice separation (SVS). Successful representation learning (RL) depends heavily on well-designed neural architectures and effective general priors. However, DNNs for RL in SVS are mostly built on generic architectures without general priors being systematically considered. To address these issues, we introduce deep unfolding to RL and propose two RL-based models for SVS, deep unfolded representation learning (DURL) and optimal transport DURL (OT-DURL). In both models, we formulate RL as a sequence of optimization problems for signal reconstruction, where three general priors, synthesis, non-negative, and our novel analysis, are incorporated. In DURL and OT-DURL, we take different approaches in penalizing the analysis prior. DURL uses the Euclidean distance as its penalty, while OT-DURL uses a more sophisticated penalty known as the OT distance. We address the optimization problems in DURL and OT-DURL with the first-order operator splitting algorithm and unfold the obtained iterative algorithms to novel encoders, by mapping the synthesis/analysis/non-negative priors to different interpretable sublayers of the encoders. We evaluated these DURL and OT-DURL encoders in the unsupervised informed SVS and supervised Open-Unmix frameworks. Experimental results indicate that (1) the OT-DURL encoder is better than the DURL encoder and (2) both encoders can considerably improve the vocal-signal-separation performance compared with those of the baseline model.
- 15 June 2023
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 83 (3) , 6711-6729
Speech watermarking based tamper detection and recovery scheme with high tolerable tamper rate
- 15 June 2023
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 83 (3) , 6711-6729
Speech watermarking based tamper detection and recovery scheme with high tolerable tamper rate
- 4 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. abs 1607 6, 1-5
An Improved Optimal Transport Kernel Embedding Method with Gating Mechanism for Singing Voice Separation and Speaker Identification
Singing voice separation (SVS) and speaker identification (SI) are two classic problems in speech signal processing. Deep neural networks (DNNs) solve these two problems by extracting effective representations of the target signal from the input mixture. Since essential features of a signal can be well reflected on its latent geometric structure of the feature distribution, a natural way to address SVS/SI is to extract the geometry-aware and distribution-related features of the target signal. To do this, this work introduces the concept of optimal transport (OT) to SVS/SI and proposes an improved optimal transport kernel embedding (iOTKE) to extract the target-distribution-related features. The iOTKE learns an OT from the input signal to the target signal on the basis of a reference set learned from all training data. Thus it can maintain the feature diversity and preserve the latent geometric structure of the distribution for the target signal. To further improve the feature selection ability, we extend the proposed iOTKE to a gated version, i.e., gated iOTKE (G-iOTKE), by incorporating a lightweight gating mechanism. The gating mechanism controls effective information flow and enables the proposed method to select important features for a specific input signal. We evaluated the proposed G-iOTKE on SVS/SI. Experimental results showed that the proposed method provided better results than other models.
- 4 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. abs 1607 6, 1-5
An Improved Optimal Transport Kernel Embedding Method with Gating Mechanism for Singing Voice Separation and Speaker Identification
Singing voice separation (SVS) and speaker identification (SI) are two classic problems in speech signal processing. Deep neural networks (DNNs) solve these two problems by extracting effective representations of the target signal from the input mixture. Since essential features of a signal can be well reflected on its latent geometric structure of the feature distribution, a natural way to address SVS/SI is to extract the geometry-aware and distribution-related features of the target signal. To do this, this work introduces the concept of optimal transport (OT) to SVS/SI and proposes an improved optimal transport kernel embedding (iOTKE) to extract the target-distribution-related features. The iOTKE learns an OT from the input signal to the target signal on the basis of a reference set learned from all training data. Thus it can maintain the feature diversity and preserve the latent geometric structure of the distribution for the target signal. To further improve the feature selection ability, we extend the proposed iOTKE to a gated version, i.e., gated iOTKE (G-iOTKE), by incorporating a lightweight gating mechanism. The gating mechanism controls effective information flow and enables the proposed method to select important features for a specific input signal. We evaluated the proposed G-iOTKE on SVS/SI. Experimental results showed that the proposed method provided better results than other models.
- Open Access
- 30 August 2021
- Published by International Speech Communication Association
- p. 3041-3045
Crossfire Conditional Generative Adversarial Networks for Singing Voice Extraction
- 30 August 2021
- Published by International Speech Communication Association
- p. 3041-3045
Open AccessCrossfire Conditional Generative Adversarial Networks for Singing Voice Extraction
- 6 June 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2700-2704
Synchronous Multi-Bit Audio Watermarking Based on Phase Shifting
Audio watermarking has been developed to protect the copyright of audio signals. We considered the use of the distribution of the phase spectrum and propose an effective multi-bit audio watermarking method based on phase shifting. The proposed method is implemented on the basis of a frame-wise framework. The phase spectrum of each frame is first calculated, then, in accordance with the embedded bit number of each frame, the phase bins are segmented into several subsets. An artificial phase pattern is constructed in one of these subsets to carry a specific multi-bit watermark. In the water-mark extraction process, each frame is synchronized by identifying the phase pattern, then watermarks are extracted. We investigate the performance of the proposed method, and the results indicated that the proposed method balances the trade-off of inaudibility and robustness. From comparative evaluations, the proposed method also performed better than several other audio watermarking methods.
- 6 June 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2700-2704
Synchronous Multi-Bit Audio Watermarking Based on Phase Shifting
Audio watermarking has been developed to protect the copyright of audio signals. We considered the use of the distribution of the phase spectrum and propose an effective multi-bit audio watermarking method based on phase shifting. The proposed method is implemented on the basis of a frame-wise framework. The phase spectrum of each frame is first calculated, then, in accordance with the embedded bit number of each frame, the phase bins are segmented into several subsets. An artificial phase pattern is constructed in one of these subsets to carry a specific multi-bit watermark. In the water-mark extraction process, each frame is synchronized by identifying the phase pattern, then watermarks are extracted. We investigate the performance of the proposed method, and the results indicated that the proposed method balances the trade-off of inaudibility and robustness. From comparative evaluations, the proposed method also performed better than several other audio watermarking methods.
- 15 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 29 (23299290) , 807-822
Evolving Multi-Resolution Pooling CNN for Monaural Singing Voice Separation
Monaural singing voice separation (MSVS) is a challenging task and has been extensively studied. Deep neural networks (DNNs) are current state-of-the-art methods for MSVS. However, they are often designed manually, which is time-consuming and error-prone. They are also pre-defined, thus cannot adapt their structures to the training data. To address these issues, we first designed a multi-resolution convolutional neural network (CNN) for MSVS called multi-resolution pooling CNN (MRP-CNN), which uses various-sized pooling operators to extract multi-resolution features. We then introduced Neural Architecture Search (NAS) to extend the MRP-CNN to the evolving MRP-CNN (E-MRP-CNN) to automatically search for effective MRP-CNN structures using genetic algorithms optimized in terms of a single objective taking into account only separation performance and multiple objectives taking into account both separation performance and model complexity. The E-MRP-CNN using the multi-objective algorithm gives a set of Pareto-optimal solutions, each providing a trade-off between separation performance and model complexity. Evaluations on the MIR-1 K, DSD100, and MUSDB18 datasets were used to demonstrate the advantages of the E-MRP-CNN over several recent baselines.
- 15 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 29 (23299290) , 807-822
Evolving Multi-Resolution Pooling CNN for Monaural Singing Voice Separation
Monaural singing voice separation (MSVS) is a challenging task and has been extensively studied. Deep neural networks (DNNs) are current state-of-the-art methods for MSVS. However, they are often designed manually, which is time-consuming and error-prone. They are also pre-defined, thus cannot adapt their structures to the training data. To address these issues, we first designed a multi-resolution convolutional neural network (CNN) for MSVS called multi-resolution pooling CNN (MRP-CNN), which uses various-sized pooling operators to extract multi-resolution features. We then introduced Neural Architecture Search (NAS) to extend the MRP-CNN to the evolving MRP-CNN (E-MRP-CNN) to automatically search for effective MRP-CNN structures using genetic algorithms optimized in terms of a single objective taking into account only separation performance and multiple objectives taking into account both separation performance and model complexity. The E-MRP-CNN using the multi-objective algorithm gives a set of Pareto-optimal solutions, each providing a trade-off between separation performance and model complexity. Evaluations on the MIR-1 K, DSD100, and MUSDB18 datasets were used to demonstrate the advantages of the E-MRP-CNN over several recent baselines.
- 5 August 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 28 (23299290) , 2349-2363
Multi-Subspace Echo Hiding Based on Time-Frequency Similarities of Audio Signals
Audio watermarking plays an important role in copyright protection. Echo hiding, one of the most effective techniques for audio watermarking, has been studied for decades. However, the conventional echo hiding has been criticized for its weak security, as watermarks can be easily extracted by means of cepstrum analysis even without any prior knowledge. This paper explores the time-frequency (T-F) characteristics of the repetition structures in an audio signal to improve the security of conventional echo hiding. In our approach, the original audio signal is first converted into a high-dimensional T-F representation, and then by clustering the time frames that have similar T-F characteristics into the same subspace, the original audio is decomposed into a union of subspaces with each corresponding to one time-domain subsignal. Paired and opposite echo kernels are applied to energy-balanced subsignals for watermark embedding, which significantly improves the security. In the watermark extraction process, the subspaces are recovered on the basis of the T-F similarities and cepstrum analysis is utilized to extract watermarks. The proposed embedding and extraction schemes thus offer a new approach for echo hiding. The results of experiments demonstrate the effectiveness of our approach with respect to inaudibility, security, and robustness.
- 5 August 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 28 (23299290) , 2349-2363
Multi-Subspace Echo Hiding Based on Time-Frequency Similarities of Audio Signals
Audio watermarking plays an important role in copyright protection. Echo hiding, one of the most effective techniques for audio watermarking, has been studied for decades. However, the conventional echo hiding has been criticized for its weak security, as watermarks can be easily extracted by means of cepstrum analysis even without any prior knowledge. This paper explores the time-frequency (T-F) characteristics of the repetition structures in an audio signal to improve the security of conventional echo hiding. In our approach, the original audio signal is first converted into a high-dimensional T-F representation, and then by clustering the time frames that have similar T-F characteristics into the same subspace, the original audio is decomposed into a union of subspaces with each corresponding to one time-domain subsignal. Paired and opposite echo kernels are applied to energy-balanced subsignals for watermark embedding, which significantly improves the security. In the watermark extraction process, the subspaces are recovered on the basis of the T-F similarities and cepstrum analysis is utilized to extract watermarks. The proposed embedding and extraction schemes thus offer a new approach for echo hiding. The results of experiments demonstrate the effectiveness of our approach with respect to inaudibility, security, and robustness.
- 1 September 2019
- Published by Elsevier in Speech Communication
- Vol. 112, 1-14
Detection of speech tampering using sparse representations and spectral manipulations based information hiding
- 1 September 2019
- Published by Elsevier in Speech Communication
- Vol. 112, 1-14
Detection of speech tampering using sparse representations and spectral manipulations based information hiding
- 19 August 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 26 (10) , 1481-1485
A Skip Attention Mechanism for Monaural Singing Voice Separation
This work proposes a simple but effective attention mechanism, namely Skip Attention (SA), for monaural singing voice separation (MSVS). First, the SA, embedded in the convolutional encoder-decoder network (CEDN), realizes an attention-driven and dependency modeling for the repetitive structures of the music source. Second, the SA, replacing the popular skip connection in the CEDN, effectively controls the flow of the low-level (vocal and musical) features to the output and improves the feature sensitivity and accuracy for MSVS. Finally, we implement the proposed SA on the Stacked Hourglass Network (SHN), namely Skip Attention SHN (SA-SHN). Quantitative and qualitative evaluation results have shown that the proposed SA-SHN achieves significant performance improvement on the MIR-1K dataset (compared to the state-of-the-art SHN) and competitive MSVS performance on the DSD100 dataset (compared to the state-of-the-art DenseNet), even without using any data augmentation methods.
- 19 August 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 26 (10) , 1481-1485
A Skip Attention Mechanism for Monaural Singing Voice Separation
This work proposes a simple but effective attention mechanism, namely Skip Attention (SA), for monaural singing voice separation (MSVS). First, the SA, embedded in the convolutional encoder-decoder network (CEDN), realizes an attention-driven and dependency modeling for the repetitive structures of the music source. Second, the SA, replacing the popular skip connection in the CEDN, effectively controls the flow of the low-level (vocal and musical) features to the output and improves the feature sensitivity and accuracy for MSVS. Finally, we implement the proposed SA on the Stacked Hourglass Network (SHN), namely Skip Attention SHN (SA-SHN). Quantitative and qualitative evaluation results have shown that the proposed SA-SHN achieves significant performance improvement on the MIR-1K dataset (compared to the state-of-the-art SHN) and competitive MSVS performance on the DSD100 dataset (compared to the state-of-the-art DenseNet), even without using any data augmentation methods.