An Ensemble Approach to Question Classification: Integrating Electra Transformer, GloVe, and LSTM
Preprint
Abstract
Natural Language Processing (NLP) has emerged as a crucial technology for understanding and generating human language, playing an essential role in tasks such as machine translation, sentiment analysis, and more pertinently, question classification. As a subfield within NLP, question classification focuses on determining the type of information being sought, a fundamental step for downstream applications like question answering systems. This study presents an innovative ensemble approach for question classification, combining the strengths of Electra, GloVe, and LSTM models. Rigorously tested on the well-regarded TREC dataset, the model demonstrates how the integration of these disparate technologies can lead to superior results. Electra brings in its transformer-based capabilities for complex language understanding, GloVe offers global vector representations for capturing word-level semantics, and LSTM contributes its sequence learning abilities to model long-term dependencies. By fusing these elements strategically, our ensemble model delivers a robust and efficient solution for the complex task of question classification. Through rigorous comparisons with well-known models like BERT, RoBERTa, and DistilBERT, the ensemble approach verifies its effectiveness by attaining an 80% accuracy score on the test dataset.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- A hybrid machine learning model for classifying gene mutations in cancer using LSTM, BiLSTM, CNN, GRU, and GloVeSystems and Soft Computing, 2024
- Performance Evaluation of Swin Vision Transformer Model Using Gradient Accumulation Optimization Technique2023
- Natural language processing applied to mental illness detection: a narrative reviewnpj Digital Medicine, 2022
- Enhancing Convolutional Neural Network using Hu’s MomentsInternational Journal of Advanced Computer Science and Applications, 2020
- A Multi-layer Bidirectional Transformer Encoder for Pre-trained Word Embedding: A Survey of BERT2020
- A survey on ensemble learningFrontiers of Computer Science, 2019
- Sentiment analysis based on improved pre-trained word embeddingsExpert Systems with Applications, 2019
- Ensemble learning: A surveyWIREs Data Mining and Knowledge Discovery, 2018
- Long Short-Term Memory2012
- Question classification using support vector machines2003