Slide, Constrain, Parse, Repeat: Synchronous SlidingWindows for Document AMR Parsing
Preprint
Abstract
The sliding window approach provides an elegant way to handle contexts of sizes larger than the Transformer's input window, for tasks like language modeling. Here we extend this approach to the sequence-to-sequence task of document parsing. For this, we exploit recent progress in transition-based parsing to implement a parser with synchronous sliding windows over source and target. We develop an oracle and a parser for document-level AMR by expanding on Structured-BART such that it leverages source-target alignments and constrains decoding to guarantee synchronicity and consistency across overlapping windows. We evaluate our oracle and parser using the Abstract Meaning Representation (AMR) parsing 3.0 corpus. On the Multi-Sentence development set of AMR 3.0, we show that our transition oracle loses only 8\% of the gold cross-sentential links despite using a sliding window. In practice, this approach also results in a high-quality document-level parser with manageable memory requirements. Our proposed system performs on par with the state-of-the-art pipeline approach for document-level AMR parsing task on Multi-Sentence AMR 3.0 corpus while maintaining sentence-level parsing performance.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Efficient Long-Text Understanding with Short-Text ModelsTransactions of the Association for Computational Linguistics, 2023
- DocAMR: Multi-Sentence AMR Representation and Evaluation2022
- SpanBERT: Improving Pre-training by Representing and Predicting SpansTransactions of the Association for Computational Linguistics, 2020
- Transition-based Parsing with Stack-Transformers2020
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- Transformer-XL: Attentive Language Models beyond a Fixed-Length Context2019
- Rewarding Smatch: Transition-Based AMR Parsing with Reinforcement Learning2019
- Morphological Inflection Generation with Hard Monotonic Attention2017