Enhancing Continual Relation Extraction via Classifier Decomposition
Preprint
Abstract
Continual relation extraction (CRE) models aim at handling emerging new relations while avoiding catastrophically forgetting old ones in the streaming data. Though improvements have been shown by previous CRE studies, most of them only adopt a vanilla strategy when models first learn representations of new relations. In this work, we point out that there exist two typical biases after training of this vanilla strategy: classifier bias and representation bias, which causes the previous knowledge that the model learned to be shaded. To alleviate those biases, we propose a simple yet effective classifier decomposition framework that splits the last FFN layer into separated previous and current classifiers, so as to maintain previous knowledge and encourage the model to learn more robust representations at this training stage. Experimental results on two standard benchmarks show that our proposed framework consistently outperforms the state-of-the-art CRE models, which indicates that the importance of the first training stage to CRE models may be underestimated. Our code is available at https://github.com/hemingkx/CDec.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentation2022
- Consistent Representation Learning for Continual Relation Extraction2022
- Behind the Scenes2021
- Refining Sample Embeddings with Relation Prototypes to Enhance Continual Relation Extraction2021
- Continual Relation Learning via Episodic Memory Activation and Reconsolidation2020
- Sentence Embedding Alignment for Lifelong Relation Extraction2019
- FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation2018
- Position-aware Attention and Supervised Data Improve Slot Filling2017