A Quantum Annealing Instance Selection Approach for Efficient and Effective Transformer Fine-Tuning
Open Access
- 2 August 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 205-214
Abstract
Deep Learning approaches have become pervasive in recent years due to their ability to solve complex tasks. However, these models need huge datasets for proper training and good generalization. This translates into high training and fine-tuning time, even several days for the most complex models and large datasets. In this work, we present a novel quantum Instance Selection (IS) approach that allows to significantly reduce the size of the training datasets (by up to 28%) while maintaining the model's effectiveness, thus promoting (training) speedups and scalability. Our solution is innovative in the sense that it exploits a different computing paradigm - Quantum Annealing (QA) - a specific Quantum Computing paradigm that can be used to tackle optimization problems. To the best of our knowledge, there have been no prior attempts to tackle the IS problem using QA. Furthermore, we propose a new Quadratic Unconstrained Binary Optimization formulation specific for the IS problem, which is a contribution in itself. Through an extensive set of experiments with several Text Classification benchmarks, we empirically demonstrate our quantum solution's feasibility and competitiveness with the current state-of-the-art IS solutions.Keywords
Funding Information
- Amazon Web Services
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
- Fundação de Amparo à Pesquisa do Estado de São Paulo
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
- the Research Support Foundation of the State of Minas Gerais
This publication has 31 references indexed in Scilit:
- qCLEF: A Proposal to Evaluate Quantum Annealing for Information Retrieval and Recommender Systems2023
- An automated approach to aspect-based sentiment analysis of apps reviews using machine and deep learningAutomated Software Engineering, 2023
- On the class separability of contextual embeddings representations – or “The classifier does not matter when the (text) representation is so good!”Information Processing & Management, 2023
- Quantum bridge analytics I: a tutorial on formulating and using QUBO modelsAnnals of Operations Research, 2022
- Pruning and quantization for deep neural network acceleration: A surveyNeurocomputing, 2021
- On the cost-effectiveness of neural and non-neural approaches and representations for text classification: A comprehensive comparative studyInformation Processing & Management, 2021
- Quantum annealing for combinatorial clusteringQuantum Information Processing, 2018
- Three new instance selection methods based on local sets: A comparative study with several approaches from a bi-objective perspectivePattern Recognition, 2015
- A systematic analysis of performance measures for classification tasksInformation Processing & Management, 2009
- An information-theoretic perspective of tf–idf measuresInformation Processing & Management, 2003