A Sequence-to-Sequence Framework Based on Transformer With Masked Language Model for Optical Music Recognition
Authors:
Open Access
- 9 November 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 118243-118252
Abstract
Optical music recognition technology is of great significance in the development of the digital music. In recent years, the convolutional recurrent neural network framework with connectionist temporal classification has been used in music recognition. However, its loss function is calculated in serial mode, which leads to low efficiency in training and difficulty in convergence. Additionally, because of the gradient disappearance of excessive long music sequences, the existing music recognition models are hard to learn the relationships between musical symbols, resulting in high sequence error rate. Therefore, we propose a sequence-to-sequence framework based on transformer with masked language model to deal with these problems. The context representation between musical symbols can be captured further by the self-attention module in the transformer, which will reduce the sequence error rate. In addition, we refer to the masked language model and design a mask matrix to predict each musical symbol in a parallel way, so as to speed up the training process. Our experiments are carried out on the printed images of music stave dataset, and the results show that our proposed method is training-efficient and has great improvement in sequence accuracy rate.Keywords
Funding Information
- Natural Science Foundation of Hunan Province, China (2020JJ5372, 2019JJ50363)
- Scientific Research Projects of Department of Education of Hunan Province (18C0005)
This publication has 27 references indexed in Scilit:
- Clustering-based k-nearest neighbor classification for large-scale data with neural codes representationPattern Recognition, 2018
- Visualizing and Understanding Neural Machine Translation2017
- TeMu-app: Music characters recognition using HOG and SVM2015
- Show and tell: A neural image caption generator2015
- Deep neural network based instrument extraction from music2015
- Optical music recognition: state-of-the-art and open issuesInternational Journal of Multimedia Information Retrieval, 2012
- Staff Detection and Removal2011
- Staff Detection with Stable PathsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2009
- Connectionist temporal classification2006
- A Fast HMM Algorithm Based on Stroke Lengths for On-Line Recognition of Handwritten Music Scores2004