Multi-Label Classification of Chinese Rural Poverty Governance Texts Based on XLNet and Bi-LSTM Fused Hierarchical Attention Mechanism
Open Access
- 21 June 2023
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 13 (13) , 7377
Abstract
Hierarchical multi-label text classification (HMTC) is a highly relevant and widely discussed topic in the era of big data, particularly for efficiently classifying extensive amounts of text data. This study proposes the HTMC-PGT framework for poverty governance’s single-path hierarchical multi-label classification problem. The framework simplifies the HMTC problem into training and combination problems of multi-class classifiers in the classifier tree. Each independent classifier in this framework uses an XLNet pretrained model to extract char-level semantic embeddings of text and employs a hierarchical attention mechanism integrated with Bi-LSTM (BiLSTM + HA) to extract semantic embeddings at the document level for classification purposes. Simultaneously, this study proposes that the structure uses transfer learning (TL) between classifiers in the classifier tree. The experimental results show that the proposed XLNet + BiLSTM + HA + FC + TL model achieves micro-P, micro-R, and micro-F1 values of 96.1%, which is 7.5~38.1% higher than those of other baseline models. The HTMC-PGT framework based on XLNet, BiLSTM + HA, and transfer learning (TL) between classifier tree nodes proposed in this study solves the hierarchical multi-label classification problem of poverty governance text (PGT). It provides a new idea for solving the traditional HMTC problem.Keywords
Funding Information
- National Key Research and Development Program of China (2021ZD0110901, CAAS-ASTIP-2023-AII)
- Science and Technology Innovation Project of the Chinese Academy of Agricultural Sciences (2021ZD0110901, CAAS-ASTIP-2023-AII)
This publication has 35 references indexed in Scilit:
- Collaborating with the Machines: A Hybrid Method for Classifying Policy DocumentsPolicy Studies Journal, 2018
- Bag of Tricks for Efficient Text Classification2017
- Reduction strategies for hierarchical multi-label classification in protein function predictionBMC Bioinformatics, 2016
- Hierarchical Attention Networks for Document Classification2016
- Multi-Label Hierarchical Classification using a Competitive Neural Network for protein function prediction2012
- A survey of hierarchical classification across different application domainsData Mining and Knowledge Discovery, 2010
- A Survey on Transfer LearningIEEE Transactions on Knowledge and Data Engineering, 2009
- Refined experts2009
- Decision trees for hierarchical multi-label classificationMachine Learning, 2008
- Learning Classifiers Using Hierarchically Structured Class Taxonomies2005