Inductive bias anchors a scaling law for biological learning
Preprint
Abstract
The speed of animal learning varies substantially across individuals and task domains, and does not always track intuitive notions of task complexity. A nominally simple binary decision task can take thousands of rewards to reach proficiency, whereas seemingly complex naturalistic tasks can be learned within dozens. A long-standing explanation is that learning speed reflects the alignment between an animal’s inductive biases and the task at hand. However, without a precise notion of alignment, it is difficult to translate this idea into quantitative predictions. Here, modeling an animal as a Bayesian learner that improves through iterative reward feedback, we use statistical learning theory to derive a scaling law for biological learning. The law makes two predictions. First, inductive bias multiplicatively sets the speed of learning through a single quantity, the effective complexity . This quantity measures how well the task aligns with the animal’s prior, and corresponds to the number of bits required to represent the task solution under that prior. We demonstrate this multiplicative scaling in a human motor learning dataset, where behavioral measurements before learning can precisely predict subsequent learning speed. Second, task performance improves as a power law in experience. We validate this power law in a large-scale mouse learning dataset and show that a two-parameter rescaling collapses individual learning curves onto a universal function predicted by the theory. These results suggest that behavioral measurements before learning can be used to predict task-dependent learning speeds, and they explain why learning speed reflects alignment with prior knowledge rather than nominal task complexity.This publication has 40 references indexed in Scilit:
- A neural circuit architecture for rapid learning in goal-directed navigationNeuron, 2024
- Learning, fast and slowCurrent Opinion in Neurobiology, 2022
- High-Dimensional Statistics2019
- Refining the law of practice.Psychological Review, 2018
- Building machines that learn and think like peopleBehavioral and Brain Sciences, 2016
- Understanding Machine Learning2014
- Probabilistic models of cognition: exploring representations and inductive biasesTrends in Cognitive Sciences, 2010
- Simplicity and probability in causal explanationCognitive Psychology, 2007
- Modeling by shortest data descriptionAutomatica, 1978
- On the generality of the laws of learning.Psychological Review, 1970