Contrastive Graph Representations for Logical Formulas Embedding
- 31 December 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 35 (4) , 3563-3574
Abstract
Currently, the non-transparent computing process of deep learning has become a significant reason hindering its further development. The Neural-Symbolic (NS) system formed by integrating logic rules into neural networks has attracted increasing attention owing to its direct interpretability. Embedding symbolic logical formulas into a low-dimensional continuous space provides an effective way for the NS system. However, current studies are all constrained by the modeling ability for its syntactic structure and fail to preserve the intrinsic semantics in embeddings, which causes poor performance on downstream reasoning tasks. To this end, this paper proposes a novel method of Con trastive G raph R epresentations (ConGR) for logical formulas embedding. First, to improve the modeling ability for the syntactic structure, ConGR introduces a densely connected graph convolutional network (GCN) with an attention mechanism to process syntax parsing graphs of formulas. In this way, discriminative local and global embeddings of formulas are obtained at the syntax level. Second, the contrastive instances (positive or negative) for each anchor formula are generated by the transformation under the guidance of logical properties. To preserve semantic information, two types of contrast, global-local and global-global, are carried out to refine formula embeddings. Extensive experiments demonstrate that ConGR obtains superior performance against state-of-the-art baselines on entailment checking and premise selection datasets.Keywords
Funding Information
- National Key Research and Development Program of China (2020AAA0108800)
- National Natural Science Foundation of China (62137002, 61937001, 62176209, 62176207, 62106190, 62050194)
- National Natural Science Foundation of China (61721002)
- Innovation Research Team of Ministry of Education (IRT_17R86)
- China Postdoctoral Science Foundation (2020M683493)
- China Knowledge Centre for Engineering Science and Technology
- Fundamental Research Funds for the Central Universities (xzy022021048, xhj032021013-02)
This publication has 22 references indexed in Scilit:
- Tangent-CFT2019
- Densely Connected Convolutional Networks2017
- Deep Residual Learning for Image Recognition2016
- Mastering the game of Go with deep neural networks and tree searchNature, 2016
- Deep learningNature, 2015
- Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks2015
- Deep Learning: Methods and ApplicationsFoundations and Trends® in Signal Processing, 2014
- A Knowledge Compilation MapJournal of Artificial Intelligence Research, 2002
- Symbolic interpretation of artificial neural networksIEEE Transactions on Knowledge and Data Engineering, 1999
- Holographic reduced representationsIEEE Transactions on Neural Networks, 1995