TAQS: An Arabic Question Similarity System Using Transfer Learning of BERT With BiLSTM
Open Access
- 16 August 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 91509-91523
Abstract
With the rapid increase of Arabic content on the web comes an increased need for short and accurate answers to queries. Machine question answering has appeared as an important emerging field for progress in natural language processing techniques. Machine learning performance surpasses that of humans in some areas, such as natural language processing and text analysis, especially with large amounts of data. There are two main contributions of this research. First, we propose the Tawasul Arabic question similarity (TAQS) system with four Arabic semantic question similarity models using deep learning techniques. Second, we curated and used an Arabic customer service question-similarity dataset with a 44,404 entries of question–answer pairs, called “Tawasul.” For TAQS, first, we use transfer learning to extract the contextualized bidirectional encoder representations from transformers (BERT) embedding with bidirectional long short-term memory (BiLSTM) in two different ways. Specifically, we propose two architectures: the BERT contextual representation with BiLSTM (BERT-BiLSTM) and the hybrid transfer BERT contextual representation with BiLSTM (HT-BERT-BiLSTM). The hybrid transfer representation combines two transfer learning techniques. Second, we fine-tuned two versions of bidirectional encoder representations from transformers for Arabic language (AraBERT). The results show that the HT-BERT-BiLSTM with the features of Layer 12 reaches an accuracy of 94.45%, where the fine-tuning of AraBERTv2 and AraBERTv0.2 achieve 93.10% and 93.90% accuracy, respectively, for the Tawasul dataset. Our proposed TAQS model surpassed the performance of the state-of-the-art BiLSTM with SkipGram by a gain of 43.19% in accuracy.Keywords
Funding Information
- Deanship of Scientific Research at Imam Mohammad Ibn Saud Islamic University through the Graduate Students Research Support Program
This publication has 43 references indexed in Scilit:
- Sirius2015
- Arabic Question Answering: Systems, Resources, Tools, and Future TrendsArabian Journal for Science and Engineering, 2014
- An Overview of Evaluation Methods in TREC Ad Hoc Information Retrieval and TREC Question Answering2007
- QARAB2002
- Natural language question answering: the view from hereNatural Language Engineering, 2001
- Bridging the lexical chasm2000
- Question answering with Textual CBR1998
- Long Short-Term MemoryNeural Computation, 1997
- Auto-FAQ: an experiment in cyberspace leveragingComputer Networks and ISDN Systems, 1995
- A knowledge based Arabic question answering system (AQAS)ACM SIGART Bulletin, 1993