A general framework for interpretable neural learning based on local information-theoretic goal functions
Open Access
- 5 March 2025
- journal article
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 122 (10)
Abstract
Despite the impressive performance of biological and artificial networks, an intuitive understanding of how their local learning dynamics contribute to network-level task solutions remains a challenge to this date. Efforts to bring learning to a more local scale indeed lead to valuable insights, however, a general constructive approach to describe local learning goals that is both interpretable and adaptable across diverse tasks is still missing. We have previously formulated a local information processing goal that is highly adaptable and interpretable for a model neuron with compartmental structure. Building on recent advances in Partial Information Decomposition (PID), we here derive a corresponding parametric local learning rule, which allows us to introduce “infomorphic” neural networks. We demonstrate the versatility of these networks to perform tasks from supervised, unsupervised, and memory learning. By leveraging the interpretable nature of the PID framework, infomorphic networks represent a valuable tool to advance our understanding of the intricate structure of local learning.Keywords
All Related Versions
Funding Information
- Volkswagen Foundation (2047003)
- Volkswagen Foundation (2047005)
- European Commission (HBP-SP3.1-SGA1-T3.6.1)
- Deutsche Forschungsgemeinschaft (EXC 2067/1 - 390729940)
- Niedersächsisches Ministerium für Wissenschaft und Kultur (2047005)
- Deutsche Forschungsgemeinschaft (RTG 2906 Curiosity)
This publication has 69 references indexed in Scilit:
- Bio-inspired, task-free continual learning through activity regularizationBiological Cybernetics, 2023
- Hyperparameter optimization: Foundations, algorithms, best practices, and open challengesWIREs Data Mining and Knowledge Discovery, 2023
- Fundamentals of Artificial Neural Networks and Deep Learning2022
- Explainable artificial intelligence: an analytical reviewWIREs Data Mining and Knowledge Discovery, 2021
- Partial information decomposition as a unified approach to the specification of neural goal functionsBrain and Cognition, 2017
- Difference Target Propagation2015
- Coherent Infomax as a Computational Goal for Neural SystemsBulletin of Mathematical Biology, 2010
- Simplified neuron model as a principal component analyzerJournal of Mathematical Biology, 1982
- Generality of the functional structure of the neocortexThe Science of Nature, 1977
- A Mathematical Theory of CommunicationBell System Technical Journal, 1948