Unsupervised Incremental Online Learning and Prediction of Musical Audio Signals
Authors:
- 15 February 2016
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 24 (5) , 863-874
Abstract
Guided by the idea that musical human-computer interaction may become more effective, intuitive, and creative when basing its computer part on cognitively more plausible learning principles, we employ unsupervised incremental online learning (i.e. clustering) to build a system that predicts the next event in a musical sequence, given as audio input. The flow of the system is as follows: 1) segmentation by onset detection, 2) timbre representation of each segment by Mel frequency cepstrum coefficients, 3) discretization by incremental clustering, yielding a tree of different sound classes (e.g. timbre categories/instruments) that can grow or shrink on the fly driven by the instantaneous sound events, resulting in a discrete symbol sequence, 4) extraction of statistical regularities of the symbol sequence, using hierarchical N-grams and the newly introduced conceptual Boltzmann machine that adapt to the dynamically changing clustering tree in 3) , and 5) prediction of the next sound event in the sequence, given the last $n$ previous events. The system's robustness is assessed with respect to complexity and noisiness of the signal. Clustering in isolation yields an adjusted Rand index (ARI) of 82.7%/85.7% for data sets of singing voice and drums. Onset detection jointly with clustering achieve an ARI of 81.3%/76.3% and the prediction of the entire system yields an ARI of 27.2%/39.2%.
Keywords
Funding Information
- European Commission (EmCAP FP6-IST 013123, INSPIRE-ITN FP7-PEOPLE-2011-290000)
This publication has 26 references indexed in Scilit:
- Hierarchical Bayesian nonparametric models with applications2010
- Predictive models for musicConnection Science, 2009
- What/when causal expectation modelling applied to audio signalsConnection Science, 2009
- Combining multiple clusterings using evidence accumulationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2005
- Using Factor Oracles for Machine ImprovisationSoft Computing, 2004
- Phonological acquisition in Optimality Theory: the early stages2004
- The Continuator: Musical Interaction With StyleJournal of New Music Research, 2003
- Variable length Markov chainsThe Annals of Statistics, 1999
- Multiple viewpoint systems for music predictionJournal of New Music Research, 1995
- Neural Network Music Composition by Prediction: Exploring the Benefits of Psychoacoustic Constraints and Multi-scale ProcessingConnection Science, 1994