Validity in Music Information Research Experiments
Preprint
Abstract
Validity is the truth of an inference made from evidence, such as data collected in an experiment, and is central to working scientifically. Given the maturity of the domain of music information research (MIR), validity in our opinion should be discussed and considered much more than it has been so far. Considering validity in one's work can improve its scientific and engineering value. Puzzling MIR phenomena like adversarial attacks and performance glass ceilings become less mysterious through the lens of validity. In this article, we review the subject of validity in general, considering the four major types of validity from a key reference: Shadish et al. 2002. We ground our discussion of these types with a prototypical MIR experiment: music classification using machine learning. Through this MIR experimentalists can be guided to make valid inferences from data collected from their experiments.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- Constructing adversarial examples to investigate the plausibility of explanations in deep audio and image classifiersNeural Computing and Applications, 2022
- A Revision of the Campbellian Validity SystemScandinavian Journal of Educational Research, 2020
- Automatic chord label personalization through deep learning of shared harmonic interval profilesNeural Computing and Applications, 2018
- The Problem of Limited Inter-rater Agreement in Modelling Music SimilarityJournal of New Music Research, 2016
- The State of the Art Ten Years After a State of the Art: Future Research in Music Information RetrievalJournal of New Music Research, 2014
- The neglected user in music information retrieval researchJournal of Intelligent Information Systems, 2013
- Evaluation in Music Information RetrievalJournal of Intelligent Information Systems, 2013
- Rhythm and Transforms, Perception and MathematicsCommunications in Computer and Information Science, 2009
- The Music Information Retrieval Evaluation eXchange (MIREX)D-Lib Magazine, 2006
- Statistical evaluation of music information retrieval experimentsJournal of New Music Research, 2006