Applying Deep Learning Techniques to Estimate Patterns of Musical Gesture
Open Access
- 5 January 2021
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Psychology
- Vol. 11
Abstract
Repetitive practice is one of the most important factors in improving the performance of motor skills. This paper focuses on the analysis and classification of forearm gestures in the context of violin playing. We recorded five experts and three students performing eight traditional classical violin bow-strokes: martelé, staccato, detaché, ricochet, legato, trémolo, collé, and col legno. To record inertial motion information, we utilized the Myo sensor, which reports a multidimensional time-series signal. We synchronized inertial motion recordings with audio data to extract the spatiotemporal dynamics of each gesture. Applying state-of-the-art deep neural networks, we implemented and compared different architectures where convolutional neural networks (CNN) models demonstrated recognition rates of 97.147%, 3DMultiHeaded_CNN models showed rates of 98.553%, and rates of 99.234% were demonstrated by CNN_LSTM models. The collected data (quaternion of the bowing arm of a violinist) contained sufficient information to distinguish the bowing techniques studied, and deep learning methods were capable of learning the movement patterns that distinguish these techniques. Each of the learning algorithms investigated (CNN, 3DMultiHeaded_CNN, and CNN_LSTM) produced high classification accuracies which supported the feasibility of training classifiers. The resulting classifiers may provide the foundation of a digital assistant to enhance musicians' time spent practicing alone, providing real-time feedback on the accuracy and consistency of their musical gestures in performance.Keywords
This publication has 28 references indexed in Scilit:
- Bow Gesture Classification to Identify Three Different Expertise Levels: A Machine Learning Approach2020
- Hand Gesture Recognition Using Compact CNN via Surface Electromyography SignalsSensors, 2020
- Bowing Gestures Classification in Violin Performance: A Machine Learning ApproachFrontiers in Psychology, 2019
- Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer LearningIEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019
- PEEK - An LSTM Recurrent Network for Motion Classification from Sparse Data2018
- Ensembles of Deep LSTM Learners for Activity Recognition using WearablesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2017
- Large Scale Population Assessment of Physical Activity Using Wrist Worn Accelerometers: The UK Biobank StudyPLOS ONE, 2017
- Named data networking-based smart homeICT Express, 2016
- Adaptive Gesture Recognition with Variation Estimation for Interactive SystemsACM Transactions on Interactive Intelligent Systems, 2014
- Beyond recognition2013