Motion informed audio source separation
Authors:
- 1 March 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6-10
Abstract
In this paper we tackle the problem of single channel audio source separation driven by descriptors of the sounding object's motion. As opposed to previous approaches, motion is included as a soft-coupling constraint within the nonnegative matrix factorization framework. The proposed method is applied to a multimodal dataset of instruments in string quartet performance recordings where bow motion information is used for separation of string instruments. We show that the approach offers better source separation result than an audio-based baseline and the state-of-the-art multimodal-based approaches on these very challenging music mixtures.Keywords
This publication has 20 references indexed in Scilit:
- An interactive audio source separation framework based on non-negative matrix factorization2014
- An overview of informed audio source separation2013
- Score informed audio source separation using constrained nonnegative matrix factorization and score synthesis2013
- Algorithms for Nonnegative Matrix Factorization with the β-DivergenceNeural Computation, 2011
- A Musically Motivated Mid-Level Representation for Pitch Estimation and Musical Audio Source SeparationIEEE Journal of Selected Topics in Signal Processing, 2011
- Blind Audiovisual Source Separation Based on Sparse Redundant RepresentationsIEEE Transactions on Multimedia, 2010
- Separation by “humming”: User-guided sound extraction from monophonic mixtures2009
- Harmony in Motion2007
- Performance measurement in blind audio source separationIEEE Transactions on Audio, Speech, and Language Processing, 2006
- Real-time speaker localization and speech separation by audio-visual integration2003