Ngo Xuan Bach
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Ngo Xuan Bach has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
- Tu Minh Phuong
Posts and Telecommunications Institute of Technology, Hanoi, Vietnam
9 shared publications Hoang Thi Thu Uyen
Department of Computer Science, Posts and Telecommunications Institute of Technology, Hanoi, Vietnam
3 shared publicationsTu Minh Phuong
Posts and Telecommunications Institute of Technology
3 shared publications- Ha-Thanh Nguyen
National Institute of Informatics
2 shared publications - Le-Minh Nguyen
Information Science, Japan Advanced Institute of Science and Technology, Nomi, Japan
2 shared publications - Manh-Kien Phi
Department of Computer Science, Posts and Telecommunications Institute of Technology, Hanoi, Vietnam
2 shared publications
Latest Publications
- Open Access
- 28 September 2023
- Published by IOS Press
- p. 2370-2377
Joint Multiple Intent Detection and Slot Filling with Supervised Contrastive Learning and Self-Distillation
Multiple intent detection and slot filling are two fundamental and crucial tasks in spoken language understanding. Motivated by the fact that the two tasks are closely related, joint models that can detect intents and extract slots simultaneously are preferred to individual models that perform each task independently. The accuracy of a joint model depends heavily on the ability of the model to transfer information between the two tasks so that the result of one task can correct the result of the other. In addition, since a joint model has multiple outputs, how to train the model effectively is also challenging. In this paper, we present a method for multiple intent detection and slot filling by addressing these challenges. First, we propose a bidirectional joint model that explicitly employs intent information to recognize slots and slot features to detect intents. Second, we introduce a novel method for training the proposed joint model using supervised contrastive learning and self-distillation. Experimental results on two benchmark datasets MixATIS and MixSNIPS show that our method outperforms state-of-the-art models in both tasks. The results also demonstrate the contributions of both bidirectional design and the training method to the accuracy improvement. Our source code is available at https://github.com/anhtunguyen98/BiSLU.
- 28 September 2023
- Published by IOS Press
- p. 2370-2377
Open AccessJoint Multiple Intent Detection and Slot Filling with Supervised Contrastive Learning and Self-Distillation
Multiple intent detection and slot filling are two fundamental and crucial tasks in spoken language understanding. Motivated by the fact that the two tasks are closely related, joint models that can detect intents and extract slots simultaneously are preferred to individual models that perform each task independently. The accuracy of a joint model depends heavily on the ability of the model to transfer information between the two tasks so that the result of one task can correct the result of the other. In addition, since a joint model has multiple outputs, how to train the model effectively is also challenging. In this paper, we present a method for multiple intent detection and slot filling by addressing these challenges. First, we propose a bidirectional joint model that explicitly employs intent information to recognize slots and slot features to detect intents. Second, we introduce a novel method for training the proposed joint model using supervised contrastive learning and self-distillation. Experimental results on two benchmark datasets MixATIS and MixSNIPS show that our method outperforms state-of-the-art models in both tasks. The results also demonstrate the contributions of both bidirectional design and the training method to the accuracy improvement. Our source code is available at https://github.com/anhtunguyen98/BiSLU.
- Open Access
- 1 June 2023
- Published by Walter de Gruyter GmbH in Cybernetics and Information Technologies
- Vol. 23 (2) , 72-86
Joint Reference and Relation Extraction from Legal Documents with Enhanced Decoder Input
This paper deals with an important task in legal text processing, namely reference and relation extraction from legal documents, which includes two subtasks: 1) reference extraction; 2) relation determination. Motivated by the fact that two subtasks are related and share common information, we propose a joint learning model that solves simultaneously both subtasks. Our model employs a Transformer-based encoder-decoder architecture with non-autoregressive decoding that allows relaxing the sequentiality of traditional seq2seq models and extracting references and relations in one inference step. We also propose a method to enrich the decoder input with learnable meaningful information and therefore, improve the model accuracy. Experimental results on a dataset consisting of 5031 legal documents in Vietnamese with 61,446 references show that our proposed model performs better results than several strong baselines and achieves an F 1 score of 99.4% for the joint reference and relation extraction task.
- 1 June 2023
- Published by Walter de Gruyter GmbH in Cybernetics and Information Technologies
- Vol. 23 (2) , 72-86
Open AccessJoint Reference and Relation Extraction from Legal Documents with Enhanced Decoder Input
This paper deals with an important task in legal text processing, namely reference and relation extraction from legal documents, which includes two subtasks: 1) reference extraction; 2) relation determination. Motivated by the fact that two subtasks are related and share common information, we propose a joint learning model that solves simultaneously both subtasks. Our model employs a Transformer-based encoder-decoder architecture with non-autoregressive decoding that allows relaxing the sequentiality of traditional seq2seq models and extracting references and relations in one inference step. We also propose a method to enrich the decoder input with learnable meaningful information and therefore, improve the model accuracy. Experimental results on a dataset consisting of 5031 legal documents in Vietnamese with 61,446 references show that our proposed model performs better results than several strong baselines and achieves an F 1 score of 99.4% for the joint reference and relation extraction task.
- Open Access
Analyzing Vietnamese Legal Questions Using Deep Neural Networks with Biaffine Classifiers
In this paper, we propose using deep neural networks to extract important information from Vietnamese legal questions, a fundamental task towards building a question answering system in the legal domain. Given a legal question in natural language, the goal is to extract all the segments that contain the needed information to answer the question. We introduce a deep model that solves the task in three stages. First, our model leverages recent advanced autoencoding language models to produce contextual word embeddings, which are then combined with character-level and POS-tag information to form word representations. Next, bidirectional long short-term memory networks are employed to capture the relations among words and generate sentence-level representations. At the third stage, borrowing ideas from graph-based dependency parsing methods which provide a global view on the input sentence, we use biaffine classifiers to estimate the probability of each pair of start-end words to be an important segment. Experimental results on a public Vietnamese legal dataset show that our model outperforms the previous work by a large margin, achieving 94.79% in the F1 score. The results also prove the effectiveness of using contextual features extracted from pre-trained language models combined with other types of features such as character-level and POS-tag features when training on a limited dataset.
Open AccessAnalyzing Vietnamese Legal Questions Using Deep Neural Networks with Biaffine Classifiers
In this paper, we propose using deep neural networks to extract important information from Vietnamese legal questions, a fundamental task towards building a question answering system in the legal domain. Given a legal question in natural language, the goal is to extract all the segments that contain the needed information to answer the question. We introduce a deep model that solves the task in three stages. First, our model leverages recent advanced autoencoding language models to produce contextual word embeddings, which are then combined with character-level and POS-tag information to form word representations. Next, bidirectional long short-term memory networks are employed to capture the relations among words and generate sentence-level representations. At the third stage, borrowing ideas from graph-based dependency parsing methods which provide a global view on the input sentence, we use biaffine classifiers to estimate the probability of each pair of start-end words to be an important segment. Experimental results on a public Vietnamese legal dataset show that our model outperforms the previous work by a large margin, achieving 94.79% in the F1 score. The results also prove the effectiveness of using contextual features extracted from pre-trained language models combined with other types of features such as character-level and POS-tag features when training on a limited dataset.
- 27 December 2022
- Published by Springer Nature in Artificial Intelligence and Law
- Vol. 32 (1) , 57-86
Attentive deep neural networks for legal document retrieval
- 27 December 2022
- Published by Springer Nature in Artificial Intelligence and Law
- Vol. 32 (1) , 57-86
Attentive deep neural networks for legal document retrieval
- Open Access
Attentive Deep Neural Networks for Legal Document Retrieval
Legal text retrieval serves as a key component in a wide range of legal text processing tasks such as legal question answering, legal case entailment, and statute law retrieval. The performance of legal text retrieval depends, to a large extent, on the representation of text, both query and legal documents. Based on good representations, a legal text retrieval model can effectively match the query to its relevant documents. Because legal documents often contain long articles and only some parts are relevant to queries, it is quite a challenge for existing models to represent such documents. In this paper, we study the use of attentive neural network-based text representation for statute law document retrieval. We propose a general approach using deep neural networks with attention mechanisms. Based on it, we develop two hierarchical architectures with sparse attention to represent long sentences and articles, and we name them Attentive CNN and Paraformer. The methods are evaluated on datasets of different sizes and characteristics in English, Japanese, and Vietnamese. Experimental results show that: i) Attentive neural methods substantially outperform non-neural methods in terms of retrieval performance across datasets and languages; ii) Pretrained transformer-based models achieve better accuracy on small datasets at the cost of high computational complexity while lighter weight Attentive CNN achieves better accuracy on large datasets; and iii) Our proposed Paraformer outperforms state-of-the-art methods on COLIEE dataset, achieving the highest recall and F2 scores in the top-N retrieval task.
Open AccessAttentive Deep Neural Networks for Legal Document Retrieval
Legal text retrieval serves as a key component in a wide range of legal text processing tasks such as legal question answering, legal case entailment, and statute law retrieval. The performance of legal text retrieval depends, to a large extent, on the representation of text, both query and legal documents. Based on good representations, a legal text retrieval model can effectively match the query to its relevant documents. Because legal documents often contain long articles and only some parts are relevant to queries, it is quite a challenge for existing models to represent such documents. In this paper, we study the use of attentive neural network-based text representation for statute law document retrieval. We propose a general approach using deep neural networks with attention mechanisms. Based on it, we develop two hierarchical architectures with sparse attention to represent long sentences and articles, and we name them Attentive CNN and Paraformer. The methods are evaluated on datasets of different sizes and characteristics in English, Japanese, and Vietnamese. Experimental results show that: i) Attentive neural methods substantially outperform non-neural methods in terms of retrieval performance across datasets and languages; ii) Pretrained transformer-based models achieve better accuracy on small datasets at the cost of high computational complexity while lighter weight Attentive CNN achieves better accuracy on large datasets; and iii) Our proposed Paraformer outperforms state-of-the-art methods on COLIEE dataset, achieving the highest recall and F2 scores in the top-N retrieval task.
- 9 December 2022
- Published by Springer Nature
- p. 648-660
Combining User Specific and Global News Features for Neural News Recommendation
- 9 December 2022
- Published by Springer Nature
- p. 648-660
Combining User Specific and Global News Features for Neural News Recommendation
- 7 December 2021
- Published by Springer Nature
- p. 513-525
Analyzing Vietnamese Legal Questions Using Deep Neural Networks with Biaffine Classifiers
- 7 December 2021
- Published by Springer Nature
- p. 513-525
Analyzing Vietnamese Legal Questions Using Deep Neural Networks with Biaffine Classifiers
- Open Access
Leveraging Foreign Language Labeled Data for Aspect-Based Opinion Mining
Aspect-based opinion mining is the task of identifying sentiment at the aspect level in opinionated text, which consists of two subtasks: aspect category extraction and sentiment polarity classification. While aspect category extraction aims to detect and categorize opinion targets such as product features, sentiment polarity classification assigns a sentiment label, i.e. positive, negative, or neutral, to each identified aspect. Supervised learning methods have been shown to deliver better accuracy for this task but they require labeled data, which is costly to obtain, especially for resource-poor languages like Vietnamese. To address this problem, we present a supervised aspect-based opinion mining method that utilizes labeled data from a foreign language (English in this case), which is translated to Vietnamese by an automated translation tool (Google Translate). Because aspects and opinions in different languages may be expressed by different words, we propose using word embeddings, in addition to other features, to reduce the vocabulary difference between the original and translated texts, thus improving the effectiveness of aspect category extraction and sentiment polarity classification processes. We also introduce an annotated corpus of aspect categories and sentiment polarities extracted from restaurant reviews in Vietnamese, and conduct a series of experiments on the corpus. Experimental results demonstrate the effectiveness of the proposed approach.
Open AccessLeveraging Foreign Language Labeled Data for Aspect-Based Opinion Mining
Aspect-based opinion mining is the task of identifying sentiment at the aspect level in opinionated text, which consists of two subtasks: aspect category extraction and sentiment polarity classification. While aspect category extraction aims to detect and categorize opinion targets such as product features, sentiment polarity classification assigns a sentiment label, i.e. positive, negative, or neutral, to each identified aspect. Supervised learning methods have been shown to deliver better accuracy for this task but they require labeled data, which is costly to obtain, especially for resource-poor languages like Vietnamese. To address this problem, we present a supervised aspect-based opinion mining method that utilizes labeled data from a foreign language (English in this case), which is translated to Vietnamese by an automated translation tool (Google Translate). Because aspects and opinions in different languages may be expressed by different words, we propose using word embeddings, in addition to other features, to reduce the vocabulary difference between the original and translated texts, thus improving the effectiveness of aspect category extraction and sentiment polarity classification processes. We also introduce an annotated corpus of aspect categories and sentiment polarities extracted from restaurant reviews in Vietnamese, and conduct a series of experiments on the corpus. Experimental results demonstrate the effectiveness of the proposed approach.
- Open Access
- 1 March 2020
- Published by Walter de Gruyter GmbH in Cybernetics and Information Technologies
- Vol. 20 (1) , 112-128
Question Analysis towards a Vietnamese Question Answering System in the Education Domain
Building a computer system, which can automatically answer questions in the human language, speech or text, is a long-standing goal of the Artificial Intelligence (AI) field. Question analysis, the task of extracting important information from the input question, is the first and crucial step towards a question answering system. In this paper, we focus on the task of Vietnamese question analysis in the education domain. Our goal is to extract important information expressed by named entities in an input question, such as university names, campus names, major names, and teacher names. We present several extraction models that utilize the advantages of both traditional statistical methods with handcrafted features and more recent advanced deep neural networks with automatically learned features. Our best model achieves 88.11% in the F 1 score on a corpus consisting of 3,600 Vietnamese questions collected from the fan page of the International School, Vietnam National University, Hanoi.
- 1 March 2020
- Published by Walter de Gruyter GmbH in Cybernetics and Information Technologies
- Vol. 20 (1) , 112-128
Open AccessQuestion Analysis towards a Vietnamese Question Answering System in the Education Domain
Building a computer system, which can automatically answer questions in the human language, speech or text, is a long-standing goal of the Artificial Intelligence (AI) field. Question analysis, the task of extracting important information from the input question, is the first and crucial step towards a question answering system. In this paper, we focus on the task of Vietnamese question analysis in the education domain. Our goal is to extract important information expressed by named entities in an input question, such as university names, campus names, major names, and teacher names. We present several extraction models that utilize the advantages of both traditional statistical methods with handcrafted features and more recent advanced deep neural networks with automatically learned features. Our best model achieves 88.11% in the F 1 score on a corpus consisting of 3,600 Vietnamese questions collected from the fan page of the International School, Vietnam National University, Hanoi.
- 23 August 2019
- Published by Springer Nature in Lecture Notes in Computer Science
- p. 206-219
A POS Tagging Model for Vietnamese Social Media Text Using BiLSTM-CRF with Rich Features
- 23 August 2019
- Published by Springer Nature in Lecture Notes in Computer Science
- p. 206-219
A POS Tagging Model for Vietnamese Social Media Text Using BiLSTM-CRF with Rich Features