Multi Text Classification Model Based on BRET-CNN-BiLSTM
Authors:
- 8 July 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 184-189
Abstract
When the Bert pre-trained model is used for the Chinese text classification task, the internal parameters of the model are relatively fixed. In the process of training smaller data sets, it is easy to cause over-fitting phenomenon. Therefore, a multiple model structure based on Bert-CNN-BiLSTM is proposed. In this structure, the Bert model is used as the text information extractor, after the output features of Bert model, multi head attention and TEXT-CNN model are used for further feature information extraction, low-dimensional feature vectors with more dense semantic information are generated and spliced to improve the information entropy of text vectors. Finally, the BiLSTM with self-attention is used to extract the information of different words in text information and then classify the text information, and the loss function with Flooding mechanism is used to carry out back propagation to further prevent the over-fitting problem of the model on smaller data sets and enhance the generalization ability of the model. Compared with the traditional Bert, LSTM, TEXT-CNN models, the accuracy, precision, recall and F1 measure of this model are all better than those of traditional Bert, LSTM and TEXT-CNN models. Experiments show that the model can effectively extract the feature information from the text and improve the accuracy of the text classification task.Keywords
This publication has 12 references indexed in Scilit:
- A Brief Survey of Word Embedding and Its Recent Development2021
- Investigating the Performance of Fine-tuned Text Classification Models Based-on Bert2020
- Text Classification Based on Convolutional Neural Network and Attention Model2020
- Improving text classification with weighted word embeddings via a multi-channel TextCNN modelNeurocomputing, 2019
- Sentiment Analysis of Comment Texts Based on BiLSTMIEEE Access, 2019
- Bidirectional LSTM with attention mechanism and convolutional layer for text classificationNeurocomputing, 2019
- Research on Text Classification Based on CNN and LSTM2019
- Text classification based on LSTM and attention2018
- Deep Residual Learning for Image Recognition2016
- Convolutional Neural Networks for Sentence Classification2014