Harnessing Deep Neural Networks with Logic Rules
Preprint
Abstract
Combining deep neural networks with structured logic rules is desirable to harness flexibility and reduce uninterpretability of the neural models. We propose a general framework capable of enhancing various types of neural networks (e.g., CNNs and RNNs) with declarative first-order logic rules. Specifically, we develop an iterative distillation method that transfers the structured information of logic rules into the weights of neural networks. We deploy the framework on a CNN for sentiment analysis, and an RNN for named entity recognition. With a few highly intuitive rules, we obtain substantial improvements and achieve state-of-the-art or comparable results to previous best-performing systems.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Named Entity Recognition with Bidirectional LSTM-CNNsTransactions of the Association for Computational Linguistics, 2016
- Mastering the game of Go with deep neural networks and tree searchNature, 2016
- MGNC-CNN: A Simple Approach to Exploiting Multiple Word Embeddings for Sentence Classification2016
- Human-level concept learning through probabilistic program inductionScience, 2015
- Deep neural networks are easily fooled: High confidence predictions for unrecognizable images2015
- Fast relational learning using bottom clause propositionalization with artificial neural networksMachine Learning, 2013