InstructIE: A Bilingual Instruction-based Information Extraction Dataset
Authors:
Preprint
Abstract
Traditional information extraction (IE) methodologies, constrained by pre-defined classes and static training paradigms, often falter in adaptability, especially in the dynamic world. To bridge this gap, we explore an instruction-based IE paradigm in this paper, leveraging the substantial cross-task generalization capabilities of Large Language Models (LLMs). We observe that most existing IE datasets tend to be overly redundant in their label sets, which leads to the inclusion of numerous labels not directly relevant to the extraction content when constructing instructions. To tackle this issue, we introduce a bilingual theme-centric IE instruction dataset (Chinese and English), InstructIE, and for the first time, incorporate a theme scheme design that effectively simplifies the label structure. Furthermore, we develop an innovative framework named KG2Instruction, which is specifically designed for the automatic generation of such datasets. Experimental evaluations based on InstructIE reveal that while current models show promise in Instruction-based IE tasks, opportunities for their potential optimization also emerge. The dataset is available at https://huggingface.co/datasets/zjunlp/InstructIE.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- Universal Information Extraction as Unified Semantic MatchingProceedings of the AAAI Conference on Artificial Intelligence, 2023
- WebIE: Faithful and Robust Information Extraction on the Web2023
- InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning2022
- Cross-Task Generalization via Natural Language Crowdsourcing Instructions2022
- Unified Structure Generation for Universal Information Extraction2022
- The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoders2021
- Event Extraction by Answering (Almost) Natural Questions2020
- A Unified MRC Framework for Named Entity Recognition2020
- DuIE: A Large-Scale Chinese Dataset for Information Extraction2019
- Modeling Relations and Their Mentions without Labeled Text2010