Position Paper: Generalized grammar rules and structure-based generalization beyond classical equivariance for lexical tasks and transduction
Preprint
Abstract
Compositional generalization is one of the main properties which differentiates lexical learning in humans from state-of-art neural networks. We propose a general framework for building models that can generalize compositionally using the concept of Generalized Grammar Rules (GGRs), a class of symmetry-based compositional constraints for transduction tasks, which we view as a transduction analogue of equivariance constraints in physics-inspired tasks. Besides formalizing generalized notions of symmetry for language transduction, our framework is general enough to contain many existing works as special cases. We present ideas on how GGRs might be implemented, and in the process draw connections to reinforcement learning and other areas of research.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Human-like systematic generalization through a meta-learning neural networkNature, 2023
- Compositionality in Computational LinguisticsAnnual Review of Linguistics, 2023
- The Paradox of the Compositionality of Natural Language: A Neural Machine Translation Case Study2022
- Meta-Learning to Compositionally Generalize2021
- Compositionality Decomposed: How do Neural Networks Generalise?Journal of Artificial Intelligence Research, 2020
- RNNs can generate bounded hierarchical languages with optimal memory2020
- From words-as-mappings to words-as-cues: the role of language in semantic knowledgeLanguage, Cognition and Neuroscience, 2017
- Inductive Inference: Theory and MethodsACM Computing Surveys, 1983
- Universal grammarTheoria, 1970
- Syntactic Structures1957