An image reconstruction framework for characterizing initial visual encoding
Open Access
- 17 January 2022
- journal article
- research article
- Published by eLife Sciences Publications, Ltd in eLife
- Vol. 11
Abstract
We developed an image-computable observer model of the initial visual encoding that operates on natural image input, based on the framework of Bayesian image reconstruction from the excitations of the retinal cone mosaic. Our model extends previous work on ideal observer analysis and evaluation of performance beyond psychophysical discrimination, takes into account the statistical regularities of the visual environment, and provides a unifying framework for answering a wide range of questions regarding the visual front end. Using the error in the reconstructions as a metric, we analyzed variations of the number of different photoreceptor types on human retina as an optimal design problem. In addition, the reconstructions allow both visualization and quantification of information loss due to physiological optics and cone mosaic sampling, and how these vary with eccentricity. Furthermore, in simulations of color deficiencies and interferometric experiments, we found that the reconstructed images provide a reasonable proxy for modeling subjects’ percepts. Lastly, we used the reconstruction-based observer for the analysis of psychophysical threshold, and found notable interactions between spatial frequency and chromatic direction in the resulting spatial contrast sensitivity function. Our method is widely applicable to experiments and applications in which the initial visual encoding plays an important role.Keywords
All Related Versions
Funding Information
- Facebook Reality Labs
This publication has 109 references indexed in Scilit:
- Optimal defocus estimation in individual natural imagesProceedings of the National Academy of Sciences, 2011
- Cone photoreceptor contributions to noise and correlations in the retinal outputNature Neuroscience, 2011
- Contributions of ideal observer theory to vision researchVision Research, 2011
- Bayesian model of dynamic image stabilization in the visual systemProceedings of the National Academy of Sciences, 2010
- Functional connectivity in the retina at the resolution of photoreceptorsNature, 2010
- Design of a Trichromatic Cone ArrayPLOS Computational Biology, 2010
- Optimal design of photoreceptor mosaics: Why we do not see color at nightVisual Neuroscience, 2009
- Trichromatic reconstruction from the interleaved cone mosaic: Bayesian model and the color appearance of small spotsJournal of Vision, 2008
- Design of a Neuronal ArrayJournal of Neuroscience, 2008
- Noise characteristics and prior expectations in human visual speed perceptionNature Neuroscience, 2006