Fisher information flow in artificial neural networks
Preprint
Abstract
The estimation of continuous parameters from measured data plays a central role in many fields of physics. A key tool in understanding and improving such estimation processes is the concept of Fisher information, which quantifies how information about unknown parameters propagates through a physical system and determines the ultimate limits of precision. With Artificial Neural Networks (ANNs) gradually becoming an integral part of many measurement systems, it is essential to understand how they process and transmit parameter-relevant information internally. Here, we present a method to monitor the flow of Fisher information through an ANN performing a parameter estimation task, tracking it from the input to the output layer. We show that optimal estimation performance corresponds to the maximal transmission of Fisher information, and that training beyond this point results in information loss due to overfitting. This provides a model-free stopping criterion for network training-eliminating the need for a separate validation dataset. To demonstrate the practical relevance of our approach, we apply it to a network trained on data from an imaging experiment, highlighting its effectiveness in a realistic physical setting.Keywords
All Related Versions
- Version 1, 2025-09-02, ArXiv
- Version 2, 2025-09-24, ArXiv
- Published version: Physical Review X, 15 (3).
This publication has 56 references indexed in Scilit:
- Surpassing early stopping: A novel correlation-based stopping criterion for neural networksNeurocomputing, 2023
- Early stopping by correlating online indicators in neural networksNeural Networks, 2022
- Cramér–Rao bound‐informed training of neural networks for quantitative MRIMagnetic Resonance in Medicine, 2022
- Information loss in a non-linear neuronal modelPhysica A: Statistical Mechanics and its Applications, 2019
- Deep learningNature, 2015
- What are Extreme Learning Machines? Filling the Gap Between Frank Rosenblatt’s Dream and John von Neumann’s PuzzleCognitive Computation, 2015
- Advances in quantum metrologyNature Photonics, 2011
- Tuning curve sharpening for orientation selectivity: coding efficiency and the impact of correlationsNature Neuroscience, 2004
- Minimum mean square estimation and neural networksNeurocomputing, 1999
- The perceptron: A probabilistic model for information storage and organization in the brain.Psychological Review, 1958