Improving Seq2Seq Grammatical Error Correction via Decoding Interventions
Authors:
- Houquan ZhouHZHouquan Zhou
- Zhenghua LiZLZhenghua Li
- Min ZhangMin Zhang
- Bo ZhangBZBo Zhang
- Chen LiCLChen Li
- Ji ZhangJZJi Zhang
- Fei HuangFHFei Huang
Preprint
Abstract
The sequence-to-sequence (Seq2Seq) approach has recently been widely used in grammatical error correction (GEC) and shows promising performance. However, the Seq2Seq GEC approach still suffers from two issues. First, a Seq2Seq GEC model can only be trained on parallel data, which, in GEC task, is often noisy and limited in quantity. Second, the decoder of a Seq2Seq GEC model lacks an explicit awareness of the correctness of the token being generated. In this paper, we propose a unified decoding intervention framework that employs an external critic to assess the appropriateness of the token to be generated incrementally, and then dynamically influence the choice of the next token. We discover and investigate two types of critics: a pre-trained left-to-right language model critic and an incremental target-side grammatical error detector critic. Through extensive experiments on English and Chinese datasets, our framework consistently outperforms strong baselines and achieves results competitive with state-of-the-art methods.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- LM-Critic: Language Models for Unsupervised Grammatical Error Correction2021
- DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts2021
- A Simple Recipe for Multilingual Grammatical Error Correction2021
- Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding2021
- Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction2020
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension2020
- GECToR – Grammatical Error Correction: Tag, Not Rewrite2020
- Enabling Robust Grammatical Error Correction in New Domains: Data Sets, Metrics, and AnalysesTransactions of the Association for Computational Linguistics, 2019
- An Analysis of Incorporating an External Language Model into a Sequence-to-Sequence Model2018
- Rethinking the Inception Architecture for Computer Vision2016