NegatER: Unsupervised Discovery of Negatives in Commonsense Knowledge Bases
Preprint
Abstract
Codifying commonsense knowledge in machines is a longstanding goal of artificial intelligence. Recently, much progress toward this goal has been made with automatic knowledge base (KB) construction techniques. However, such techniques focus primarily on the acquisition of positive (true) KB statements, even though negative (false) statements are often also important for discriminative reasoning over commonsense KBs. As a first step toward the latter, this paper proposes NegatER, a framework that ranks potential negatives in commonsense KBs using a contextual language model (LM). Importantly, as most KBs do not contain negatives, NegatER relies only on the positive knowledge in the LM and does not require ground-truth negative examples. Experiments demonstrate that, compared to multiple contrastive data augmentation approaches, NegatER yields negatives that are more grammatical, coherent, and informative -- leading to statistically significant accuracy improvements in a challenging KB completion task and confirming that the positive knowledge in LMs can be "re-purposed" to generate negative knowledge.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language ModelsTransactions of the Association for Computational Linguistics, 2020
- How Much Knowledge Can You Pack Into the Parameters of a Language Model?2020
- Structure Aware Negative Sampling in Knowledge Graphs2020
- Transformers: State-of-the-Art Natural Language Processing2020
- Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly2020
- Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks2020
- Billion-Scale Similarity Search with GPUsIEEE Transactions on Big Data, 2019
- Commonsense reasoning and commonsense knowledge in artificial intelligenceCommunications of the ACM, 2015
- Negative Knowledge: Understanding Professional Learning and ExpertiseVocations and Learning, 2008
- ConceptNet — A Practical Commonsense Reasoning Tool-KitBT Technology Journal, 2004