Noisy and Unbalanced Multimodal Document Classification: Textbook Exercises as a Use Case
Open Access
- 20 September 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 71-78
Abstract
In order to foster inclusive education, automatic systems that can adapt textbooks to make them accessible to children with Developmental Coordination Disorder (DCD) are necessary. In this context, we propose a task to classify exercises according to their DCD adaptation type. We introduce a challenging exercise dataset extracted from French textbooks, with two major difficulties: limited and unbalanced, noisy data. To set a baseline on the dataset, we use state-of-the-art models combined through early and late fusion techniques to take advantage of text and vision/layout modalities. Our approach achieves an overall accuracy of 0.802. However, the experiments show the difficulty of the task, especially for minority classes, where the accuracy drops to 0.583.Keywords
Funding Information
- ANR (21-CE38-0014)
This publication has 18 references indexed in Scilit:
- LayoutLM: Pre-training of Text and Layout for Document Image Understanding2020
- Learning Asr-Robust Contextualized Embeddings for Spoken Language Understanding2020
- CamemBERT: a Tasty French Language Model2020
- ICDAR2019 Competition on Scanned Receipt OCR and Information Extraction2019
- FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents2019
- Universal Language Model Fine-tuning for Text Classification2018
- Deep Contextualized Word Representations2018
- Focal Loss for Dense Object Detection2017
- Evaluation of deep convolutional nets for document image classification and retrieval2015
- Building a test collection for complex document information processing2006