DINEX: Interpretable NLP via Diversified Natural Language Explanations
Authors:
Open Access
- 16 October 2024
- book chapter
- Published by IOS Press
- p. 2798-2805
Abstract
Natural Language Explanations (NLE) are becoming increasingly important in Interpretable Natural Language Processing, which can clarify the reasoning process and improve performance. It is essential to utilize human-authored gold explanations to improve the quality of explanations produced by the generator. However, the gold standard explanations within the same dataset are produced by a fixed group of annotators, leading to a more homogeneous perspective and style. This has inspired us to improve the quality of generated explanations by enhancing the diversity of the training set. Based on this, we introduce DINEX, a two-stage framework comprising a diversified explanation generator and an explanation-aware predictor, suitable for any task related to NLE. The first stage of DINEX augments the generator’s training set through two approaches: Semantic Similarity Sampling (SSS) and Structural Variety Generation (SVG). This enables the generator to learn how to produce NLE from diverse perspectives and styles. In the second stage, DINEX improves the predictor’s ability to capture the complementary semantics between contexts and explanations. It also reduces the impact of noise on predictions through dynamic perturbations. We conduct experiments on four datasets in the domains of Question Answering and Reasoning. The results show that DINEX achieves an average performance improvement of 5.49%, establishing a new state-of-the-art on the ComVE dataset. Evaluations by human evaluators and Large Language Models (LLMs) demonstrate that DINEX-generated explanations surpass the baseline in quality across 62.4% of the test samples.Keywords
This publication has 17 references indexed in Scilit:
- Towards Faithful Model Explanation in NLP: A SurveyComputational Linguistics, 2024
- A survey on XAI and natural language explanationsInformation Processing & Management, 2022
- A survey on sentiment analysis methods, applications, and challengesArtificial Intelligence Review, 2022
- Investigating the Benefits of Free-Form Rationales2022
- A Survey of Data Augmentation Approaches for NLP2021
- Explanations for CommonsenseQA: New Dataset and Models2021
- Data Augmentation using Pre-trained Transformer Models2020
- SemEval-2020 Task 4: Commonsense Validation and Explanation2020
- Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering2020
- Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?2020