Wilson S. Geisler

The dynamic range of contrast-encoding in the early visual system has been investigated at both single-neuron and population levels in animals using oriented stimuli such as gratings and Gabor patches. However, contrast-encoding of unoriented stimuli such as Gaussians has been less explored, even though such stimuli can evoke a large population response. In studies that employ Gaussians, contrast response functions (CRFs) of neurons and neural populations in primary visual cortex are typically characterized across a limited range of Weber contrasts up to 100%, which may not adequately reflect the statistics of contrast in natural scenes. Indeed, our analysis shows that locations with contrasts far exceeding 100% Weber contrast are ubiquitous in natural scenes. Thus, our current understanding of contrast encoding remains incomplete in the context of natural environments. Here, we measured spiking activities of individual neurons using electrophysiology and population responses using calcium imaging in V1 of fixating macaques while Gaussian stimuli were presented over a wide range of Weber contrasts, up to 900%. Both the average spiking response and the average population response continued to increase robustly above 100% Weber contrast. Only a small minority of neurons saturate below 100% Weber contrast. These results demonstrate that the dynamic range of contrast-encoding in V1 is broader than previously assumed and aligns more closely with the statistics of contrast in natural scenes.

Perceptual systems in humans and many other animals are able to segment scenes into regions that are likely to be physically meaningful. This ability depends on having low-level mechanisms that can accurately categorize whether local image patches are samples from the same or different kinds of texture. We find that using spatial proximity as a proxy for same-different ground truth makes it possible to train accurate decision variables and bounds directly from arbitrary natural images with no feedback. We also find that performance can be further improved by using proximity as a ground truth for adjusting the final decision variables and bounds for the current image/scene. These surprising findings result from the simple fact that under a wide range of conditions proximity discrimination (near vs. far) and texture discrimination (same vs. different) have mathematically identical decision bounds if the same image features are used for both tasks. We used the decision variables and bounds trained on natural images as the initial steps in a hierarchical Bayesian observer (HBO) model of texture discrimination [9]. Given the relative simplicity of this HBO model, it did an excellent job of segmenting images having randomly shaped regions containing arbitrary natural textures. We suggest that the proximity proxy is something that natural selection could discover and exploit for any same-different task where the task-relevant stimulus features also vary systematically with distance in space and/or time. For example, natural selection could have created developmental learning/plasticity mechanisms that exploit the proximity proxy.

Visual search is a fundamental natural task for humans and other animals. We investigated the decision processes that humans use in covert (single fixation) search with briefly presented displays having well-separated potential target locations. Performance was compared with the Bayesian-optimal decision process under the assumption that the information from the different potential target locations is statistically independent. Surprisingly, humans performed slightly better than optimal, despite humans' substantial loss of sensitivity in the fovea ("foveal neglect") and the implausibility of the human brain replicating the optimal computations. We show that three factors can quantitatively explain these seemingly paradoxical results. Most importantly, simple and fixed heuristic decision rules reach near optimal search performance. Second, foveal neglect primarily affects only the central potential target location. Finally, spatially correlated neural noise can cause search performance to exceed that predicted for independent noise. These findings have broad implications for understanding visual search tasks and other identification tasks in humans and other animals.

Many natural tasks require the visual system to classify image patches accurately into target categories, including the category of no target. Natural target categories often involve high levels of within-category variability (uncertainty), making it challenging to uncover the underlying computational mechanisms. Here, we describe these tasks as identification from a set of exhaustive, mutually exclusive target categories, each partitioned into mutually exclusive subcategories. We derive the optimal decision rule and present a computational method to simulate performance for moderately large and complex tasks. We focus on the detection of an additive wavelet target in white noise with five dimensions of stimulus uncertainty: target amplitude, orientation, scale, background contrast, and spatial pattern. We compare the performance of the ideal observer with various heuristic observers. We find that a properly normalized heuristic MAX observer (SNN-MAX) approximates optimal performance. We also find that a convolutional neural network trained on this task approaches but does not reach optimal performance, even with extensive training. We measured human performance on a task with three of these dimensions of uncertainty (orientation, scale, and background pattern). Results show that the pattern of hits and correct rejections for the ideal and SNN-MAX observers (but not a simple MAX observer) aligns with the data. Additionally, we measured performance without scale and orientation uncertainty and found that the effect of uncertainty on performance was less than predicted by any model. This unexpectedly small effect can largely be explained by incorporating biologically plausible levels of intrinsic position uncertainty into the models.

When detecting targets under natural conditions, the visual system almost always faces multiple, simultaneous, dimensions of extrinsic uncertainty. This study focused on the simultaneous uncertainty about target amplitude and background contrast. These dimensions have a large effect on detection and vary greatly in natural scenes. We measured the human performance for detecting a sine-wave target in white noise and natural-scene backgrounds for two levels of prior probability of the target being present. We derived and tested the ideal observer for white-noise backgrounds, a special case of a template-matching observer that dynamically moves its criterion with the background contrast (the DTM observer) and two simpler models with a fixed criterion: the template-matching (TM) observer and the normalized template-matching (NTM) observer that normalizes template response by background contrast. Simulations show that, when the target prior is low, the performance of the NTM observer is near optimal and the TM observer is near chance, suggesting that manipulating the target prior is valuable for distinguishing among models. Surprisingly, we found that the NTM and DTM observers better explain human performance than the TM observer for both target priors in both background types. We argue that the visual system most likely exploits contrast normalization, rather than dynamic criterion adjustment, to deal with simultaneous background contrast and target amplitude uncertainty. Finally, our findings show that the data collected under high levels of uncertainty have a rich structure capable of discriminating between models, providing an alternative approach for studying high dimensions of uncertainty.

A number of recent studies have been directed at measuring and modeling detection of targets at specific locations in natural backgrounds, a key subtask of visual search in natural environments. A useful approach is to bin natural background patches into joint histograms with bins along specific background dimensions. By measuring psychometric functions in a sparse subset of these bins, it is possible to estimate how the included dimensions jointly affect detectability over the whole space of natural backgrounds. In previous studies, we found that threshold is proportional to the product of the background luminance, contrast, and similarity; a result predicted by a simple template-matching observer with divisive normalization along each of the dimensions. The measure of similarity was the cosine similarity of the amplitude spectra of the target and background (S_A)—a phase-invariant measure. Here, we investigated the effect of the cosine similarity of the target and background images (S_I|A)—a phase-dependent measure. We found that threshold decreases monotonically with S_I|A in agreement with a recent study (Rideaux et al., 2022). In contrast, the template-matching observer predicts threshold to be a U-shaped function of S_I|A reaching a minimum when the target and background are orthogonal (S_I|A = 0). Surprisingly, when the template-matching observer includes a small amount of intrinsic position uncertainty (measured in a separate experiment) the pattern of thresholds is explained.

Binocular stereo cues are important for discriminating 3D surface orientation, especially at near distances. We devised a single-interval task where observers discriminated the slant of a densely textured planar test surface relative to a textured planar surround reference surface. Although surfaces were rendered with correct perspective, the stimuli were designed so that the binocular cues dominated performance. Slant discrimination performance was measured as a function of the reference slant and the level of uncorrelated white noise added to the test-plane images in the left and right eyes. We compared human performance with an approximate ideal observer (planar matching [PM]) and two subideal observers. The PM observer uses the image in one eye and back projection to predict a test image in the other eye for all possible slants, tilts, and distances. The estimated slant, tilt, and distance are determined by the prediction that most closely matches the measured image in the other eye. The first subideal observer (local planar matching [LPM]) applies PM over local neighborhoods and then pools estimates across the test plane. The second suboptimal observer (local frontoparallel matching [LFM]) uses only location disparity. We find that the ideal observer (PM) and the first subideal observer (LPM) outperforms the second subideal observer (LFM), demonstrating the additional benefit of pattern disparities. We also find that all three model observers can account for human performance, if two free parameters are included: a fixed small level of internal estimation noise, and a fixed overall efficiency scalar on slant discriminability.

Univariate and multivariate normal probability distributions are widely used when modeling decisions under uncertainty. Computing the performance of such models requires integrating these distributions over specific domains, which can vary widely across models. Besides some special cases where these integrals are easy to calculate, there exist no general analytical expressions, standard numerical methods, or software for these integrals. Here we present mathematical results and open-source software that provide (a) the probability in any domain of a normal in any dimensions with any parameters; (b) the probability density, cumulative distribution, and inverse cumulative distribution of any function of a normal vector; (c) the classification errors among any number of normal distributions, the Bayes-optimal discriminability index, and relation to the receiver operating characteristic (ROC); (d) dimension reduction and visualizations for such problems; and (e) tests for how reliably these methods may be used on given data. We demonstrate these tools with vision research applications of detecting occluding objects in natural scenes and detecting camouflage.

Visual systems evolve to process the stimuli that arise in the organism’s natural environment, and hence, to fully understand the neural computations in the visual system, it is important to measure behavioral and neural responses to natural visual stimuli. Here, we measured psychometric and neurometric functions in the macaque monkey for detection of a windowed sine‐wave target in uniform backgrounds and in natural backgrounds of various contrasts. The neurometric functions were obtained by near‐optimal decoding of voltage‐sensitive-dye-imaging (VSDI) responses at the retinotopic scale in primary visual cortex (V1). The results were compared with previous human psychophysical measurements made under the same conditions. We found that human and macaque behavioral thresholds followed the generalized Weber’s law as function of contrast, and that both the slopes and the intercepts of the threshold as a function of background contrast match each other up to a single scale factor. We also found that the neurometric thresholds followed the generalized Weber’s law with slopes and intercepts matching the behavioral slopes and intercepts up to a single scale factor. We conclude that human and macaque ability to detect targets in natural backgrounds are affected in the same way by background contrast, that these effects are consistent with population decoding at the retinotopic scale by down‐stream circuits, and that the macaque monkey is an appropriate animal model for gaining an understanding of the neural mechanisms in humans for detecting targets in natural backgrounds. Finally, we discuss limitations of the current study and potential next steps. NEW & NOTEWORTHY We measured macaque detection performance in natural images and compared their performance to the detection sensitivity of neurophysiological responses recorded in the primary visual cortex (V1), and to the performance of human subjects. We found that 1) human and macaque behavioral performances are in quantitative agreement and 2) are consistent with near‐optimal decoding of V1 population responses.