Few-Shot Transfer Learning for Text Classification With Lightweight Word Embedding Based Models
Authors:
Open Access
- 18 April 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 7, 53296-53304
Abstract
Many deep learning architectures have been employed to model the semantic compositionality for text sequences, requiring a huge amount of supervised data for parameters training, making it unfeasible in situations where numerous annotated samples are not available or even do not exist. Different from data-hungry deep models, lightweight word embedding-based models could represent text sequences in a plug-and-play way due to their parameter-free property. In this paper, a modified hierarchical pooling strategy over pre-trained word embeddings is proposed for text classification in a few-shot transfer learning way. The model leverages and transfers knowledge obtained from some source domains to recognize and classify the unseen text sequences with just a handful of support examples in the target problem domain. The extensive experiments on five datasets including both English and Chinese text demonstrate that the simple word embedding-based models (SWEMs) with parameter-free pooling operations are able to abstract and represent the semantic text. The proposed modified hierarchical pooling method exhibits significant classification performance in the few-shot transfer learning tasks compared with other alternative methods.Keywords
Funding Information
- National Natural Science Foundation of China (61703417)
This publication has 11 references indexed in Scilit:
- Few-shot learning for short text classificationMultimedia Tools and Applications, 2018
- Bag of Tricks for Efficient Text Classification2017
- A Decomposable Attention Model for Natural Language Inference2016
- Deep Unordered Composition Rivals Syntactic Methods for Text Classification2015
- Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks2015
- A Convolutional Neural Network for Modelling Sentences2014
- Glove: Global Vectors for Word Representation2014
- Convolutional Neural Networks for Sentence Classification2014
- A Survey on Transfer LearningIEEE Transactions on Knowledge and Data Engineering, 2009
- DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline StereoIEEE Transactions on Pattern Analysis and Machine Intelligence, 2009