An Ensemble Approach to Question Classification: Integrating Electra Transformer, GloVe, and LSTM
Preprint
Abstract
This paper introduces a novel ensemble approach for question classification using state-of-the-art models -- Electra, GloVe, and LSTM. The proposed model is trained and evaluated on the TREC dataset, a well-established benchmark for question classification tasks. The ensemble model combines the strengths of Electra, a transformer-based model for language understanding, GloVe, a global vectors for word representation, and LSTM, a recurrent neural network variant, providing a robust and efficient solution for question classification. Extensive experiments were carried out to compare the performance of the proposed ensemble approach with other cutting-edge models, such as BERT, RoBERTa, and DistilBERT. Our results demonstrate that the ensemble model outperforms these models across all evaluation metrics, achieving an accuracy of 0.8 on the test set. These findings underscore the effectiveness of the ensemble approach in enhancing the performance of question classification tasks, and invite further exploration of ensemble methods in natural language processing.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Performance Evaluation of Swin Vision Transformer Model Using Gradient Accumulation Optimization Technique2023
- Natural language processing applied to mental illness detection: a narrative reviewnpj Digital Medicine, 2022
- Enhancing Convolutional Neural Network using Hu’s MomentsInternational Journal of Advanced Computer Science and Applications, 2020
- A Multi-layer Bidirectional Transformer Encoder for Pre-trained Word Embedding: A Survey of BERT2020
- A survey on ensemble learningFrontiers of Computer Science, 2019
- Sentiment analysis based on improved pre-trained word embeddingsExpert Systems with Applications, 2019
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- Ensemble learning: A surveyWIREs Data Mining and Knowledge Discovery, 2018
- Glove: Global Vectors for Word Representation2014
- Long Short-Term Memory2012