Framework for Deep Learning-Based Language Models Using Multi-Task Learning in Natural Language Understanding: A Systematic Literature Review and Future Directions
Top Cited Papers
Open Access
- 7 February 2022
- journal article
- review article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 17078-17097
Abstract
Learning human languages is a difficult task for a computer. However, Deep Learning (DL) techniques have enhanced performance significantly for almost all-natural language processing (NLP) tasks. Unfortunately, these models cannot be generalized for all the NLP tasks with similar performance. NLU (Natural Language Understanding) is a subset of NLP including tasks, like machine translation, dialogue-based systems, natural language inference, text entailment, sentiment analysis, etc. The advancement in the field of NLU is the collective performance enhancement in all these tasks. Even though MTL (Multi-task Learning) was introduced before Deep Learning, it has gained significant attention in the past years. This paper aims to identify, investigate, and analyze various language models used in NLU and NLP to find directions for future research. The Systematic Literature Review (SLR) is prepared using the literature search guidelines proposed by Kitchenham and Charters on various language models between 2011 and 2021. This SLR points out that the unsupervised learning method-based language models show potential performance improvement. However, they face the challenge of designing the general-purpose framework for the language model, which will improve the performance of multi-task NLU and the generalized representation of knowledge. Combining these approaches may result in a more efficient and robust multi-task NLU. This SLR proposes building steps for a conceptual framework to achieve goals of enhancing the performance of language models in the field of NLU.Keywords
This publication has 58 references indexed in Scilit:
- Document Modeling with Gated Recurrent Neural Network for Sentiment Classification2015
- Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks2015
- A Convolutional Neural Network for Modelling Sentences2014
- Glove: Global Vectors for Word Representation2014
- Convolutional Neural Networks for Sentence Classification2014
- SemEval-2014 Task 1: Evaluation of Compositional Distributional Semantic Models on Full Sentences through Semantic Relatedness and Textual Entailment2014
- Transforming Auto-Encoders2011
- Multitask Learning1998
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Indexing by latent semantic analysisJournal of the American Society for Information Science, 1990