David C. Knill
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
David C. Knill has a total of 12 co-authors. Here are the top co-authors by number of shared publications:
- Alexandre Pouget
Department of Basic Neuroscience, University of Geneva, Geneva, Switzerland
1 shared publication Amulya Bondada
Department of Brain and Cognitive Sciences University of Rochester 14618 Rochester New York
1 shared publication- Anna Seydell-Greenwald
Georgetown University Medical Center
1 shared publication - Anne-Marie Brouwer
TNO (Netherlands Organization for Applied Scientific Research), Utrecht, The Netherlands
1 shared publication - B. Hu
Center for Visual Science, University of Rochester, Rochester, NY 14627, USA
1 shared publication Daniel Kersten
Department of Psychology University of Minnesota Minneapolis-St. Paul 55455 Minnesota
1 shared publication
Latest Publications
- Open Access
- 1 June 2011
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 11 (7) , 23
Binocular and monocular depth cues in online feedback control of 3D pointing movement
Previous work has shown that humans continuously use visual feedback of the hand to control goal-directed movements online. In most studies, visual error signals were predominantly in the image plane and, thus, were available in an observer's retinal image. We investigate how humans use visual feedback about finger depth provided by binocular and monocular depth cues to control pointing movements. When binocularly viewing a scene in which the hand movement was made in free space, subjects were about 60 ms slower in responding to perturbations in depth than in the image plane. When monocularly viewing a scene designed to maximize the available monocular cues to finger depth (motion, changing size, and cast shadows), subjects showed no response to perturbations in depth. Thus, binocular cues from the finger are critical to effective online control of hand movements in depth. An optimal feedback controller that takes into account the low peripheral stereoacuity and inherent ambiguity in cast shadows can explain the difference in response time in the binocular conditions and lack of response in monocular conditions.
- 1 June 2011
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 11 (7) , 23
Open AccessBinocular and monocular depth cues in online feedback control of 3D pointing movement
Previous work has shown that humans continuously use visual feedback of the hand to control goal-directed movements online. In most studies, visual error signals were predominantly in the image plane and, thus, were available in an observer's retinal image. We investigate how humans use visual feedback about finger depth provided by binocular and monocular depth cues to control pointing movements. When binocularly viewing a scene in which the hand movement was made in free space, subjects were about 60 ms slower in responding to perturbations in depth than in the image plane. When monocularly viewing a scene designed to maximize the available monocular cues to finger depth (motion, changing size, and cast shadows), subjects showed no response to perturbations in depth. Thus, binocular cues from the finger are critical to effective online control of hand movements in depth. An optimal feedback controller that takes into account the low peripheral stereoacuity and inherent ambiguity in cast shadows can explain the difference in response time in the binocular conditions and lack of response in monocular conditions.
- Open Access
- 26 January 2011
- Published by Society for Neuroscience in Journal of Neuroscience
- Vol. 31 (4) , 1219-1237
Flexible, Task-Dependent Use of Sensory Feedback to Control Hand Movements
We tested whether changing accuracy demands for simple pointing movements leads humans to adjust the feedback control laws that map sensory signals from the moving hand to motor commands. Subjects made repeated pointing movements in a virtual environment to touch a button whose shape varied randomly from trial to trial—between squares, rectangles oriented perpendicular to the movement path, and rectangles oriented parallel to the movement path. Subjects performed the task on a horizontal table but saw the target configuration and a virtual rendering of their pointing finger through a mirror mounted between a monitor and the table. On one-third of trials, the position of the virtual finger was perturbed by ±1 cm either in the movement direction or perpendicular to the movement direction when the finger passed behind an occluder. Subjects corrected quickly for the perturbations despite not consciously noticing them; however, they corrected almost twice as much for perturbations aligned with the narrow dimension of a target than for perturbations aligned with the long dimension. These changes in apparent feedback gain appeared in the kinematic trajectories soon after the time of the perturbations, indicating that they reflect differences in the feedback control law used throughout the duration of movements. The results indicate that the brain adjusts its feedback control law for individual movements “on demand” to fit task demands. Simulations of optimal control laws for a two-joint arm show that accuracy demands alone, coupled with signal-dependent noise, lead to qualitatively the same behavior.
- 26 January 2011
- Published by Society for Neuroscience in Journal of Neuroscience
- Vol. 31 (4) , 1219-1237
Open AccessFlexible, Task-Dependent Use of Sensory Feedback to Control Hand Movements
We tested whether changing accuracy demands for simple pointing movements leads humans to adjust the feedback control laws that map sensory signals from the moving hand to motor commands. Subjects made repeated pointing movements in a virtual environment to touch a button whose shape varied randomly from trial to trial—between squares, rectangles oriented perpendicular to the movement path, and rectangles oriented parallel to the movement path. Subjects performed the task on a horizontal table but saw the target configuration and a virtual rendering of their pointing finger through a mirror mounted between a monitor and the table. On one-third of trials, the position of the virtual finger was perturbed by ±1 cm either in the movement direction or perpendicular to the movement direction when the finger passed behind an occluder. Subjects corrected quickly for the perturbations despite not consciously noticing them; however, they corrected almost twice as much for perturbations aligned with the narrow dimension of a target than for perturbations aligned with the long dimension. These changes in apparent feedback gain appeared in the kinematic trajectories soon after the time of the perturbations, indicating that they reflect differences in the feedback control law used throughout the duration of movements. The results indicate that the brain adjusts its feedback control law for individual movements “on demand” to fit task demands. Simulations of optimal control laws for a two-joint arm show that accuracy demands alone, coupled with signal-dependent noise, lead to qualitatively the same behavior.
- Open Access
- 1 January 2010
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 10 (4) , 1-27
Adapting internal statistical models for interpreting visual cues to depth
The informativeness of sensory cues depends critically on statistical regularities in the environment. However, statistical regularities vary between different object categories and environments. We asked whether and how the brain changes the prior assumptions about scene statistics used to interpret visual depth cues when stimulus statistics change. Subjects judged the slants of stereoscopically presented figures by adjusting a virtual probe perpendicular to the surface. In addition to stereoscopic disparities, the aspect ratio of the stimulus in the image provided a “figural compression” cue to slant, whose reliability depends on the distribution of aspect ratios in the world. As we manipulated this distribution from regular to random and back again, subjects' reliance on the compression cue relative to stereoscopic cues changed accordingly. When we randomly interleaved stimuli from shape categories (ellipses and diamonds) with different statistics, subjects gave less weight to the compression cue for figures from the category with more random aspect ratios. Our results demonstrate that relative cue weights vary rapidly as a function of recently experienced stimulus statistics and that the brain can use different statistical models for different object categories. We show that subjects' behavior is consistent with that of a broad class of Bayesian learning models.
- 1 January 2010
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 10 (4) , 1-27
Open AccessAdapting internal statistical models for interpreting visual cues to depth
The informativeness of sensory cues depends critically on statistical regularities in the environment. However, statistical regularities vary between different object categories and environments. We asked whether and how the brain changes the prior assumptions about scene statistics used to interpret visual depth cues when stimulus statistics change. Subjects judged the slants of stereoscopically presented figures by adjusting a virtual probe perpendicular to the surface. In addition to stereoscopic disparities, the aspect ratio of the stimulus in the image provided a “figural compression” cue to slant, whose reliability depends on the distribution of aspect ratios in the world. As we manipulated this distribution from regular to random and back again, subjects' reliance on the compression cue relative to stereoscopic cues changed accordingly. When we randomly interleaved stimuli from shape categories (ellipses and diamonds) with different statistics, subjects gave less weight to the compression cue for figures from the category with more random aspect ratios. Our results demonstrate that relative cue weights vary rapidly as a function of recently experienced stimulus statistics and that the brain can use different statistical models for different object categories. We show that subjects' behavior is consistent with that of a broad class of Bayesian learning models.
- Open Access
- 1 February 2009
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 9 (2) , 11
Cue integration outside central fixation: A study of grasping in depth
We assessed the usefulness of stereopsis across the visual field by quantifying how retinal eccentricity and distance from the horopter affect humans' relative dependence on monocular and binocular cues about 3D orientation. The reliabilities of monocular and binocular cues both decline with eccentricity, but the reliability of binocular information decreases more rapidly. Binocular cue reliability also declines with increasing distance from the horopter, whereas the reliability of monocular cues is virtually unaffected. We measured how subjects integrated these cues to orient their hands when grasping oriented discs at different eccentricities and distances from the horopter. Subjects relied increasingly less on binocular disparity as targets' retinal eccentricity and distance from the horopter increased. The measured cue influences were consistent with what would be predicted from the relative cue reliabilities at the various target locations. Our results showed that relative reliability affects how cues influence motor control and that stereopsis is of limited use in the periphery and away from the horopter because monocular cues are more reliable in these regions.
- 1 February 2009
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 9 (2) , 11
Open AccessCue integration outside central fixation: A study of grasping in depth
We assessed the usefulness of stereopsis across the visual field by quantifying how retinal eccentricity and distance from the horopter affect humans' relative dependence on monocular and binocular cues about 3D orientation. The reliabilities of monocular and binocular cues both decline with eccentricity, but the reliability of binocular information decreases more rapidly. Binocular cue reliability also declines with increasing distance from the horopter, whereas the reliability of monocular cues is virtually unaffected. We measured how subjects integrated these cues to orient their hands when grasping oriented discs at different eccentricities and distances from the horopter. Subjects relied increasingly less on binocular disparity as targets' retinal eccentricity and distance from the horopter increased. The measured cue influences were consistent with what would be predicted from the relative cue reliabilities at the various target locations. Our results showed that relative reliability affects how cues influence motor control and that stereopsis is of limited use in the periphery and away from the horopter because monocular cues are more reliable in these regions.
- Open Access
- 1 January 2009
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 9 (1) , 24
Humans use visual and remembered information about object location to plan pointing movements
We investigated whether humans use a target's remembered location to plan reaching movements to targets according to the relative reliabilities of visual and remembered information. Using their index finger, subjects moved a virtual object from one side of a table to the other, and then went back to a target. In some trials, the target shifted unnoticed while the finger made the first movement. We regressed subjects' movement trajectories against the initial and shifted target locations to infer the weights that subjects gave to remembered and visual locations. We measured the reliability of vision and memory by adding conditions in which the target only appeared after subjects made the first movement (vision only) and in which the target was initially present but disappeared during the first movement (memory only). When both visual and remembered information were available, movement trajectories were biased to the remembered target location. The different weights that subjects gave to memory and visual information on average matched the weights predicted by the variance associated with the use of vision and memory alone. This suggests that humans integrate remembered information about object locations with peripheral visual information by taking into account the relative reliability of the two sources of information.
- 1 January 2009
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 9 (1) , 24
Open AccessHumans use visual and remembered information about object location to plan pointing movements
We investigated whether humans use a target's remembered location to plan reaching movements to targets according to the relative reliabilities of visual and remembered information. Using their index finger, subjects moved a virtual object from one side of a table to the other, and then went back to a target. In some trials, the target shifted unnoticed while the finger made the first movement. We regressed subjects' movement trajectories against the initial and shifted target locations to infer the weights that subjects gave to remembered and visual locations. We measured the reliability of vision and memory by adding conditions in which the target only appeared after subjects made the first movement (vision only) and in which the target was initially present but disappeared during the first movement (memory only). When both visual and remembered information were available, movement trajectories were biased to the remembered target location. The different weights that subjects gave to memory and visual information on average matched the weights predicted by the variance associated with the use of vision and memory alone. This suggests that humans integrate remembered information about object locations with peripheral visual information by taking into account the relative reliability of the two sources of information.
- Open Access
- 1 May 2007
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 7 (8) , 13
Learning Bayesian priors for depth perception
How the visual system learns the statistical regularities (e.g., symmetry) needed to interpret pictorial cues to depth is one of the outstanding questions in perceptual science. We test the hypothesis that the visual system can adapt its model of the statistics of planar figures for estimating three-dimensional surface orientation. In particular, we test whether subjects, when placed in an environment containing a large proportion of randomly shaped ellipses, learn to give less weight to a prior bias to interpret ellipses as slanted circles when making slant judgments of stereoscopically viewed ellipses. In a first experiment, subjects placed a cylinder onto a stereoscopically viewed, slanted, elliptical surface. In this experiment, subjects received full haptic feedback about the true orientation of the surface at the end of the movement. When test stimuli containing small conflicts between the circle interpretation as figure and the slant suggested by stereoscopic disparities were intermixed with stereoscopically viewed circles, subjects gave the same weight to the circle interpretation over the course of five daily sessions. When the same test stimuli were intermixed with stereoscopic views of randomly shaped ellipses, however, subjects gave progressively lower weights to the circle interpretation of test stimuli over five daily sessions. In a second experiment, subjects showed the same effect when they made perceptual judgments of slant without receiving feedback, showing that feedback is not required for learning. We describe a Bayesian model for combining multiple visual cues to adapt the priors underlying pictorial depth cues that qualitatively accounts for the observed behavior.
- 1 May 2007
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 7 (8) , 13
Open AccessLearning Bayesian priors for depth perception
How the visual system learns the statistical regularities (e.g., symmetry) needed to interpret pictorial cues to depth is one of the outstanding questions in perceptual science. We test the hypothesis that the visual system can adapt its model of the statistics of planar figures for estimating three-dimensional surface orientation. In particular, we test whether subjects, when placed in an environment containing a large proportion of randomly shaped ellipses, learn to give less weight to a prior bias to interpret ellipses as slanted circles when making slant judgments of stereoscopically viewed ellipses. In a first experiment, subjects placed a cylinder onto a stereoscopically viewed, slanted, elliptical surface. In this experiment, subjects received full haptic feedback about the true orientation of the surface at the end of the movement. When test stimuli containing small conflicts between the circle interpretation as figure and the slant suggested by stereoscopic disparities were intermixed with stereoscopically viewed circles, subjects gave the same weight to the circle interpretation over the course of five daily sessions. When the same test stimuli were intermixed with stereoscopic views of randomly shaped ellipses, however, subjects gave progressively lower weights to the circle interpretation of test stimuli over five daily sessions. In a second experiment, subjects showed the same effect when they made perceptual judgments of slant without receiving feedback, showing that feedback is not required for learning. We describe a Bayesian model for combining multiple visual cues to adapt the priors underlying pictorial depth cues that qualitatively accounts for the observed behavior.
- Open Access
- 2 April 2007
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 7 (7) , 5-24
Robust cue integration: A Bayesian model and evidence from cue-conflict studies with stereoscopic and figure cues to slant
Most research on depth cue integration has focused on stimulus regimes in which stimuli contain the small cue conflicts that one might expect to normally arise from sensory noise. In these regimes, linear models for cue integration provide a good approximation to system performance. This article focuses on situations in which large cue conflicts can naturally occur in stimuli. We describe a Bayesian model for nonlinear cue integration that makes rational inferences about scenes across the entire range of possible cue conflicts. The model derives from the simple intuition that multiple properties of scenes or causal factors give rise to the image information associated with most cues. To make perceptual inferences about one property of a scene, an ideal observer must necessarily take into account the possible contribution of these other factors to the information provided by a cue. In the context of classical depth cues, large cue conflicts most commonly arise when one or another cue is generated by an object or scene that violates the strongest form of constraint that makes the cue informative. For example, when binocularly viewing a slanted trapezoid, the slant interpretation of the figure derived by assuming that the figure is rectangular may conflict greatly with the slant suggested by stereoscopic disparities. An optimal Bayesian estimator incorporates the possibility that different constraints might apply to objects in the world and robustly integrates cues with large conflicts by effectively switching between different internal models of the prior constraints underlying one or both cues. We performed two experiments to test the predictions of the model when applied to estimating surface slant from binocular disparities and the compression cue (the aspect ratio of figures in an image). The apparent weight that subjects gave to the compression cue decreased smoothly as a function of the conflict between the cues but did not shrink to zero; that is, subjects did not fully veto the compression cue at large cue conflicts. A Bayesian model that assumes a mixed prior distribution of figure shapes in the world, with a large proportion being very regular and a smaller proportion having random shapes, provides a good quantitative fit for subjects' performance. The best fitting model parameters are consistent with the sensory noise to be expected in measurements of figure shape, further supporting the Bayesian model as an account of robust cue integration.
- 2 April 2007
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 7 (7) , 5-24
Open AccessRobust cue integration: A Bayesian model and evidence from cue-conflict studies with stereoscopic and figure cues to slant
Most research on depth cue integration has focused on stimulus regimes in which stimuli contain the small cue conflicts that one might expect to normally arise from sensory noise. In these regimes, linear models for cue integration provide a good approximation to system performance. This article focuses on situations in which large cue conflicts can naturally occur in stimuli. We describe a Bayesian model for nonlinear cue integration that makes rational inferences about scenes across the entire range of possible cue conflicts. The model derives from the simple intuition that multiple properties of scenes or causal factors give rise to the image information associated with most cues. To make perceptual inferences about one property of a scene, an ideal observer must necessarily take into account the possible contribution of these other factors to the information provided by a cue. In the context of classical depth cues, large cue conflicts most commonly arise when one or another cue is generated by an object or scene that violates the strongest form of constraint that makes the cue informative. For example, when binocularly viewing a slanted trapezoid, the slant interpretation of the figure derived by assuming that the figure is rectangular may conflict greatly with the slant suggested by stereoscopic disparities. An optimal Bayesian estimator incorporates the possibility that different constraints might apply to objects in the world and robustly integrates cues with large conflicts by effectively switching between different internal models of the prior constraints underlying one or both cues. We performed two experiments to test the predictions of the model when applied to estimating surface slant from binocular disparities and the compression cue (the aspect ratio of figures in an image). The apparent weight that subjects gave to the compression cue decreased smoothly as a function of the conflict between the cues but did not shrink to zero; that is, subjects did not fully veto the compression cue at large cue conflicts. A Bayesian model that assumes a mixed prior distribution of figure shapes in the world, with a large proportion being very regular and a smaller proportion having random shapes, provides a good quantitative fit for subjects' performance. The best fitting model parameters are consistent with the sensory noise to be expected in measurements of figure shape, further supporting the Bayesian model as an account of robust cue integration.
- Open Access
- 2 March 2007
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 7 (5)
Sensorimotor Processing and Goal-Directed Movement.
- 2 March 2007
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 7 (5)
Open AccessSensorimotor Processing and Goal-Directed Movement.
- Open Access
- 1 February 2005
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 5 (2) , 2
Reaching for visual cues to depth: The brain combines depth cues differently for motor control and perception
Vision provides a number of cues about the three-dimensional (3D) layout of objects in a scene that could be used for planning and controlling goal-directed behaviors such as pointing, grasping, and placing objects. An emerging consensus from the perceptual work is that the visual brain is a near-optimal Bayesian estimator of object properties, for example, by integrating cues in a way that accounts for differences in their reliability. We measured how the visuomotor system integrates binocular and monocular cues to 3D surface orientation to guide the placement of objects on a slanted surface. Subjects showed qualitatively similar results to those found in perceptual studies-they gave more weight to binocular cues at low slants and more weight to monocular cues like texture at high slants. We compared subjects’ performance in the visuomotor task with their performance on matched perceptual tasks that required an observer to estimate the same 3D surface properties needed to control the motor behavior. The relative influence of binocular and monocular cues changed in qualitatively the same way across stimulus conditions in the two types of task; however, subjects gave significantly more weight to binocular cues for controlling hand movements than for making explicit perceptual judgments in these tasks. Thus, the brain changes how it integrates visual cues based not only on the information content of stimuli, but also on the task for which the information is used.
- 1 February 2005
- Published by Association for Research in Vision and Ophthalmology (ARVO) in Journal of Vision
- Vol. 5 (2) , 2
Open AccessReaching for visual cues to depth: The brain combines depth cues differently for motor control and perception
Vision provides a number of cues about the three-dimensional (3D) layout of objects in a scene that could be used for planning and controlling goal-directed behaviors such as pointing, grasping, and placing objects. An emerging consensus from the perceptual work is that the visual brain is a near-optimal Bayesian estimator of object properties, for example, by integrating cues in a way that accounts for differences in their reliability. We measured how the visuomotor system integrates binocular and monocular cues to 3D surface orientation to guide the placement of objects on a slanted surface. Subjects showed qualitatively similar results to those found in perceptual studies-they gave more weight to binocular cues at low slants and more weight to monocular cues like texture at high slants. We compared subjects’ performance in the visuomotor task with their performance on matched perceptual tasks that required an observer to estimate the same 3D surface properties needed to control the motor behavior. The relative influence of binocular and monocular cues changed in qualitatively the same way across stimulus conditions in the two types of task; however, subjects gave significantly more weight to binocular cues for controlling hand movements than for making explicit perceptual judgments in these tasks. Thus, the brain changes how it integrates visual cues based not only on the information content of stimuli, but also on the task for which the information is used.
- Top Cited Papers
- 1 December 2004
- Published by Elsevier in Trends in Neurosciences
- Vol. 27 (12) , 712-719
The Bayesian brain: the role of uncertainty in neural coding and computation
- 1 December 2004
- Published by Elsevier in Trends in Neurosciences
- Vol. 27 (12) , 712-719
Top Cited PapersThe Bayesian brain: the role of uncertainty in neural coding and computation