Proposition from the Perspective of Chinese Language: A Chinese Proposition Classification Evaluation Benchmark
Preprint
Abstract
Existing propositions often rely on logical constants for classification. Compared with Western languages that lean towards hypotaxis such as English, Chinese often relies on semantic or logical understanding rather than logical connectives in daily expressions, exhibiting the characteristics of parataxis. However, existing research has rarely paid attention to this issue. And accurately classifying these propositions is crucial for natural language understanding and reasoning. In this paper, we put forward the concepts of explicit and implicit propositions and propose a comprehensive multi-level proposition classification system based on linguistics and logic. Correspondingly, we create a large-scale Chinese proposition dataset PEACE from multiple domains, covering all categories related to propositions. To evaluate the Chinese proposition classification ability of existing models and explore their limitations, We conduct evaluations on PEACE using several different methods including the Rule-based method, SVM, BERT, RoBERTA, and ChatGPT. Results show the importance of properly modeling the semantic features of propositions. BERT has relatively good proposition classification capability, but lacks cross-domain transferability. ChatGPT performs poorly, but its classification ability can be improved by providing more proposition information. Many issues are still far from being resolved and require further study.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text2022
- MERIt: Meta-Path Guided Contrastive Learning for Logical Reasoning2022
- The Possible, the Plausible, and the Desirable: Event-Based Modality Detection for Language Processing2021
- DAGN: Discourse-Aware Graph Network for Logical Reasoning2021
- LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning2020
- Data-driven classification of the certainty of scholarly assertionsPeerJ, 2020
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- Using uncertainty to link and rank evidence from biomedical literature for model curationBioinformatics, 2017
- The BioScope corpus: biomedical texts annotated for uncertainty, negation and their scopesBMC Bioinformatics, 2008