ZoomNet for Topic-Oriented Fragment Recognition in Long Documents
Open Access
- 11 April 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 39545-39554
Abstract
This work introduces a new information extraction task called Topic-Oriented Fragment Recognition (TOFR) , whose goal is to recognize information related to a specific topic in long documents from professional fields. In this paper, we introduce two TOFR datasets to study the problems of processing long documents. We propose a novel neural framework named Zooming Network (ZoomNet) , which overcomes the challenge of combining information over long distances with limited computing resources by flexibly switching between skimming and intensive reading in processing long documents. In general, ZoomNet first establishes a hierarchical representation aligned to the text structure, which relieves the conflict between local information and extensive contextual information. Then, it synthesizes different levels of information to assign tags via multi-scale actions. We combine supervised and reinforcement learning methods to train our model. Experiments show that the proposed model outperforms several state-of-the-art sequence labeling models, including BiLSTM-CRF, BERT, XLNET, RoBERTa, and ELECTRA, on both TOFR datasets with big margins.Keywords
Funding Information
- National Key Research and Development Program of China (2021ZD0200300)
- Beijing Academy of Artificial Intelligence
- institute guoqiang at tsinghua university
- Beijing Innovation Center for Future Chips
- IDG/Mcgovern Institute for Brain Research at Tsinghua University
- National Natural Science Foundation of China (61836004)
This publication has 18 references indexed in Scilit:
- Learning to Skim Text2017
- End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF2016
- Hierarchical Attention Networks for Document Classification2016
- Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings2015
- On the Properties of Neural Machine Translation: Encoder–Decoder Approaches2014
- Sequence labeling with multiple annotatorsMachine Learning, 2013
- An analysis of active learning strategies for sequence labeling tasks2008
- Introduction to the CoNLL-2000 shared task2000
- Learning long-term dependencies in NARX recurrent neural networksIEEE Transactions on Neural Networks, 1996
- Learning long-term dependencies with gradient descent is difficultIEEE Transactions on Neural Networks, 1994