Probabilistic Neural–Symbolic Models With Inductive Posterior Constraints
- 1 August 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (2) , 2667-2679
Abstract
Neural–symbolic models provide a powerful tool to tackle complex visual reasoning tasks by combining symbolic program execution for reasoning and deep representation learning for visual recognition. A probabilistic formulation of such models with stochastic latent variables can obtain an interpretable and legible reasoning system with less supervision. However, it is still nontrivial to generate reasonable symbolic structures without the guidance of domain knowledge, since it generally involves an optimization problem with both continuous and discrete variables. Despite the challenges, the interpretability of such symbolic structures provides an interface to regularize their generation by domain knowledge. In this article, we propose to incorporate the available domain knowledge into the learning process of probabilistic neural–symbolic (PNS) models via posterior constraints that directly regularize the structure posterior. In this way, our model is able to identify a middle point where the structure generation process mainly learns from data but also selectively borrows information from domain knowledge. We further present inductive reasoning where the posterior constraints can be automatically reweighted to handle noisy annotations. The experimental results show that our method achieves state-of-the-art performance on major abstract reasoning datasets and enjoys good generalization capability and data efficiency.Keywords
Funding Information
- National Key Research and Development Program of China (2017YFA0700904, 2020AAA0106000, 2020AAA0104304, 2020AAA0106302, 2021YFB2701000)
- NSFC Projects (62061136001, 62076145, 62076147, U19B2034, U1811461, U19A2081, 61972224)
- NSF project (JQ19016)
- BNRist (BNR2022RC01006)
- Tsinghua Institute for Guo Qiang
- Beijing Academy of Artificial Intelligence
- Tsinghua-OPPO Joint Research Center for Future Terminal Technology
- High Performance Computing Center, Tsinghua University
This publication has 23 references indexed in Scilit:
- CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning2017
- Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering2017
- Image Question Answering Using Convolutional Neural Network with Dynamic Parameter Prediction2016
- Neural Module Networks2016
- Learning to Compose Neural Networks for Question Answering2016
- SUN RGB-D: A RGB-D scene understanding benchmark suite2015
- An anthropomorphic method for progressive matrix problemsCognitive Systems Research, 2013
- A computational model for solving problems from the Raven’s Progressive Matrices intelligence test using iconic visual representationsCognitive Systems Research, 2013
- Raven Progressive Matrices2003
- A general framework for object detection2002