Marco A. Martinez Ramirez
Research Subjects
Marco A. Martinez Ramirez has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Emmanouil Benetos
Centre for Digital Music Queen Mary University of London UK
3 shared publications Joshua D. Reiss
Queen Mary University of London,Centre for Digital Music,UK
3 shared publications- Joshua D. Reiss
Queen Mary University of London
2 shared publications Bhusan Chettri
1 shared publicationBob L. Sturm
1 shared publication- Daniel Stoller
Queen Mary University of London
1 shared publication
Latest Publications
- Open Access
- 6 June 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 66-70
Differentiable Signal Processing With Black-Box Audio Effects
We present a data-driven approach to automate audio signal processing by incorporating stateful third-party, audio effects as layers within a deep neural network. We then train a deep encoder to analyze input audio and control effect parameters to perform the desired signal manipulation, requiring only input-target paired audio data as supervision. To train our network with non-differentiable black-box effects layers, we use a fast, parallel stochastic gradient approximation scheme within a standard auto differentiation graph, yielding efficient end-to-end backpropagation. We demonstrate the power of our approach with three separate automatic audio production applications: tube amplifier emulation, automatic removal of breaths and pops from voice recordings, and automatic music mastering. We validate our results with a subjective listening test, showing our approach not only can enable new automatic audio effects tasks, but can yield results comparable to a specialized, state-of-the-art commercial solution for music mastering.
- 6 June 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 66-70
Open AccessDifferentiable Signal Processing With Black-Box Audio Effects
We present a data-driven approach to automate audio signal processing by incorporating stateful third-party, audio effects as layers within a deep neural network. We then train a deep encoder to analyze input audio and control effect parameters to perform the desired signal manipulation, requiring only input-target paired audio data as supervision. To train our network with non-differentiable black-box effects layers, we use a fast, parallel stochastic gradient approximation scheme within a standard auto differentiation graph, yielding efficient end-to-end backpropagation. We demonstrate the power of our approach with three separate automatic audio production applications: tube amplifier emulation, automatic removal of breaths and pops from voice recordings, and automatic music mastering. We validate our results with a subjective listening test, showing our approach not only can enable new automatic audio effects tasks, but can yield results comparable to a specialized, state-of-the-art commercial solution for music mastering.
- 1 May 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 241-245
Modeling Plate and Spring Reverberation Using A DSP-Informed Deep Neural Network
Plate and spring reverberators are electromechanical systems first used and researched as means to substitute real room reverberation. Currently, they are often used in music production for aesthetic reasons due to their particular sonic characteristics. The modeling of these audio processors and their perceptual qualities is difficult since they use mechanical elements together with analog electronics resulting in an extremely complex response. Based on digital reverberators that use sparse FIR filters, we propose a signal processing-informed deep learning architecture for the modeling of artificial reverberators. We explore the capabilities of deep neural networks to learn such highly nonlinear electromechanical responses and we perform modeling of plate and spring reverberators. In order to measure the performance of the model, we conduct a perceptual evaluation experiment and we also analyze how the given task is accomplished and what the model is actually learning.
- 1 May 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 241-245
Modeling Plate and Spring Reverberation Using A DSP-Informed Deep Neural Network
Plate and spring reverberators are electromechanical systems first used and researched as means to substitute real room reverberation. Currently, they are often used in music production for aesthetic reasons due to their particular sonic characteristics. The modeling of these audio processors and their perceptual qualities is difficult since they use mechanical elements together with analog electronics resulting in an extremely complex response. Based on digital reverberators that use sparse FIR filters, we propose a signal processing-informed deep learning architecture for the modeling of artificial reverberators. We explore the capabilities of deep neural networks to learn such highly nonlinear electromechanical responses and we perform modeling of plate and spring reverberators. In order to measure the performance of the model, we conduct a perceptual evaluation experiment and we also analyze how the given task is accomplished and what the model is actually learning.
- Open Access
- 16 January 2020
- Published by MDPI AG in Applied Sciences
- Vol. 10 (2) , 638
Deep Learning for Black-Box Modeling of Audio Effects
Virtual analog modeling of audio effects consists of emulating the sound of an audio processor reference device. This digital simulation is normally done by designing mathematical models of these systems. It is often difficult because it seeks to accurately model all components within the effect unit, which usually contains various nonlinearities and time-varying components. Most existing methods for audio effects modeling are either simplified or optimized to a very specific circuit or type of audio effect and cannot be efficiently translated to other types of audio effects. Recently, deep neural networks have been explored as black-box modeling strategies to solve this task, i.e., by using only input–output measurements. We analyse different state-of-the-art deep learning models based on convolutional and recurrent neural networks, feedforward WaveNet architectures and we also introduce a new model based on the combination of the aforementioned models. Through objective perceptual-based metrics and subjective listening tests we explore the performance of these models when modeling various analog audio effects. Thus, we show virtual analog models of nonlinear effects, such as a tube preamplifier; nonlinear effects with memory, such as a transistor-based limiter and nonlinear time-varying effects, such as the rotating horn and rotating woofer of a Leslie speaker cabinet.
- 16 January 2020
- Published by MDPI AG in Applied Sciences
- Vol. 10 (2) , 638
Open AccessDeep Learning for Black-Box Modeling of Audio Effects
Virtual analog modeling of audio effects consists of emulating the sound of an audio processor reference device. This digital simulation is normally done by designing mathematical models of these systems. It is often difficult because it seeks to accurately model all components within the effect unit, which usually contains various nonlinearities and time-varying components. Most existing methods for audio effects modeling are either simplified or optimized to a very specific circuit or type of audio effect and cannot be efficiently translated to other types of audio effects. Recently, deep neural networks have been explored as black-box modeling strategies to solve this task, i.e., by using only input–output measurements. We analyse different state-of-the-art deep learning models based on convolutional and recurrent neural networks, feedforward WaveNet architectures and we also introduce a new model based on the combination of the aforementioned models. Through objective perceptual-based metrics and subjective listening tests we explore the performance of these models when modeling various analog audio effects. Thus, we show virtual analog models of nonlinear effects, such as a tube preamplifier; nonlinear effects with memory, such as a transistor-based limiter and nonlinear time-varying effects, such as the rotating horn and rotating woofer of a Leslie speaker cabinet.
- 15 September 2019
- Published by International Speech Communication Association
Ensemble Models for Spoofing Detection in Automatic Speaker Verification
- 15 September 2019
- Published by International Speech Communication Association
Ensemble Models for Spoofing Detection in Automatic Speaker Verification
- Open Access
A general-purpose deep learning approach to model time-varying audio effects
Audio processors whose parameters are modified periodically over time are often referred as time-varying or modulation based audio effects. Most existing methods for modeling these type of effect units are often optimized to a very specific circuit and cannot be efficiently generalized to other time-varying effects. Based on convolutional and recurrent neural networks, we propose a deep learning architecture for generic black-box modeling of audio processors with long-term memory. We explore the capabilities of deep neural networks to learn such long temporal dependencies and we show the network modeling various linear and nonlinear, time-varying and time-invariant audio effects. In order to measure the performance of the model, we propose an objective metric based on the psychoacoustics of modulation frequency perception. We also analyze what the model is actually learning and how the given task is accomplished.
Open AccessA general-purpose deep learning approach to model time-varying audio effects
Audio processors whose parameters are modified periodically over time are often referred as time-varying or modulation based audio effects. Most existing methods for modeling these type of effect units are often optimized to a very specific circuit and cannot be efficiently generalized to other time-varying effects. Based on convolutional and recurrent neural networks, we propose a deep learning architecture for generic black-box modeling of audio processors with long-term memory. We explore the capabilities of deep neural networks to learn such long temporal dependencies and we show the network modeling various linear and nonlinear, time-varying and time-invariant audio effects. In order to measure the performance of the model, we propose an objective metric based on the psychoacoustics of modulation frequency perception. We also analyze what the model is actually learning and how the given task is accomplished.
- 1 May 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 171-175
Modeling Nonlinear Audio Effects with End-to-end Deep Neural Networks
In the context of music production, distortion effects are mainly used for aesthetic reasons and are usually applied to electric musical instruments. Most existing methods for nonlinear modeling are often either simplified or optimized to a very specific circuit. In this work, we investigate deep learning architectures for audio processing and we aim to find a general purpose end-to-end deep neural network to perform modeling of nonlinear audio effects. We show the network modeling various nonlinearities and we discuss the generalization capabilities among different instruments.
- 1 May 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 171-175
Modeling Nonlinear Audio Effects with End-to-end Deep Neural Networks
In the context of music production, distortion effects are mainly used for aesthetic reasons and are usually applied to electric musical instruments. Most existing methods for nonlinear modeling are often either simplified or optimized to a very specific circuit. In this work, we investigate deep learning architectures for audio processing and we aim to find a general purpose end-to-end deep neural network to perform modeling of nonlinear audio effects. We show the network modeling various nonlinearities and we discuss the generalization capabilities among different instruments.
- 1 October 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Stem audio mixing as a content-based transformation of audio features
Multitrack audio mixing is an essential part of music production and one of the first steps consist on processing individual stems from raw recordings. In this paper, we investigate this stage as a content-based transformation. We explore which audio features are relevant to interpret this specific process and which set of features gets modified by the mixing of stems in the most consistent way. We show that the number of features can be reduced with a procedure based on the permutation importance method of random forest classifiers. Thus, the selected audio features are used to train various classification models and we analyse which set of features lead to a better classification accuracy. We conclude that the underlying characteristics of manipulating raw recordings into individual stems can be described by this selected set of features.
- 1 October 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Stem audio mixing as a content-based transformation of audio features
Multitrack audio mixing is an essential part of music production and one of the first steps consist on processing individual stems from raw recordings. In this paper, we investigate this stage as a content-based transformation. We explore which audio features are relevant to interpret this specific process and which set of features gets modified by the mixing of stems in the most consistent way. We show that the number of features can be reduced with a procedure based on the permutation importance method of random forest classifiers. Thus, the selected audio features are used to train various classification models and we analyse which set of features lead to a better classification accuracy. We conclude that the underlying characteristics of manipulating raw recordings into individual stems can be described by this selected set of features.