Vivian C. Paulun

Solid objects typically move as cohesive, bounded bodies; they do not split apart, spread, or divide. Yet, non-solid substances, like sand, do. Adults hold distinct expectations for solid objects and non-solid substances, or things and stuff. But when does this distinction emerge in development? Here, we asked whether infants expect sand to be non-cohesive. We familiarized 4-to-6- and 13-to-15-month-old infants with videos of a solid object or non-solid sand. Subsequent test events showed the object or sand falling onto a wedge and either splitting into two pieces or piles or moving cohesively on one side of the wedge. Infants in both age groups looked longer when the solid object unexpectedly split apart, confirming robust cohesion expectations. Yet, for granular substances, infants showed no equivalent preference—neither expecting stuff to be cohesive nor to be non-cohesive. These findings suggest that cohesion expectations develop along different trajectories for things and stuff.

A key to interacting with the physical world is the ability to visually infer object properties, like elasticity, allowing us to anticipate object behaviour. Such perceptual inferences continue to challenge artificial intelligence systems, highlighting the complexity of the underlying computations. How does the human brain solve this task? Here, we propose a resource-rational model based on learned statistics of object motion to explain how humans judge elasticity. We created 100 000 physics-based simulations of bouncing cubes with different elasticities and found that even tiny changes in initial conditions (e.g. orientation) yield starkly different trajectories. Yet, across these simulations, we identified 23 motion features that capture natural variations in elasticity. Although a weighted combination of these features reliably predicts physical elasticity, surprisingly, humans do not seem to employ cue combination when judging elasticity. Instead, they switch between different cues. A series of experiments designed to carefully tease apart several competing heuristics suggests that observers switch between different computationally efficient yet informative cues depending on the information available in the stimulus.

Core perception: re-imagining precocious reasoning as sophisticated perceiving sets out an impressive agenda that Spelke’s core knowledge theory is more appropriately called core perception. We raise two questions about this argument. First, why impose a distinct boundary between perception and cognition? Second, can investigations of core physics reveal new insights into how the mind bridges perception and cognition?

Imagine pouring a box of granola into a bowl. Are you considering hundreds of individual chunks or the motion of the group as a whole? Human perceptual limits suggest we cannot be representing the individuals, implying we simulate ensembles of objects. If true, we would need to represent group physical properties beyond individual aggregates, similar to perceiving ensemble properties like color, size, or facial expression. Here we investigate whether people do hold ensemble representations of mass, using tasks in which participants watch a video of a single marble or set of marbles falling onto an elastic cloth and judge the individual or average mass. We find first that people better judge average masses than individual masses, then find evidence that the better ensemble judgments are not just due to aggregating information from individual marbles. Together, this supports the concept of ensemble perception in intuitive physics, extending our understanding of how people represent and simulate sets of objects.

Selecting suitable grasps on three-dimensional objects is a challenging visuomotor computation, which involves combining information about an object (e.g., its shape, size, and mass) with information about the actor's body (e.g., the optimal grasp aperture and hand posture for comfortable manipulation). Here, we used functional magnetic resonance imaging to investigate brain networks associated with these distinct aspects during grasp planning and execution. Human participants of either sex viewed and then executed preselected grasps on L-shaped objects made of wood and/or brass. By leveraging a computational approach that accurately predicts human grasp locations, we selected grasp points that disentangled the role of multiple grasp-relevant factors, that is, grasp axis, grasp size, and object mass. Representational Similarity Analysis revealed that grasp axis was encoded along dorsal-stream regions during grasp planning. Grasp size was first encoded in ventral stream areas during grasp planning then in premotor regions during grasp execution. Object mass was encoded in ventral stream and (pre)motor regions only during grasp execution. Premotor regions further encoded visual predictions of grasp comfort, whereas the ventral stream encoded grasp comfort during execution, suggesting its involvement in haptic evaluation. These shifts in neural representations thus capture the sensorimotor transformations that allow humans to grasp objects. SIGNIFICANCE STATEMENT Grasping requires integrating object properties with constraints on hand and arm postures. Using a computational approach that accurately predicts human grasp locations by combining such constraints, we selected grasps on objects that disentangled the relative contributions of object mass, grasp size, and grasp axis during grasp planning and execution in a neuroimaging study. Our findings reveal a greater role of dorsal-stream visuomotor areas during grasp planning, and, surprisingly, increasing ventral stream engagement during execution. We propose that during planning, visuomotor representations initially encode grasp axis and size. Perceptual representations of object material properties become more relevant instead as the hand approaches the object and motor programs are refined with estimates of the grip forces required to successfully lift the object.

A key to interacting with the physical world is the ability to visually infer object properties, like elasticity, allowing us to anticipate object behavior. Such perceptual inferences continue to challenge AI systems, highlighting the complexity of the underlying computations. How does the human brain solve this task? Here, we propose a resource-rational model based on learned statistics of object motion to explain how humans judge elasticity. We created 100,000 physics-based simulations of bouncing cubes with different elasticities and found that even tiny changes in initial conditions (e.g., orientation) yield starkly different trajectories. Yet, across these simulations, we identified 23 motion features that capture natural variations in elasticity. Although a weighted combination of these features reliably predicts physical elasticity, surprisingly, humans do not seem to employ cue combination when judging elasticity. Instead, they switch between different cues. A series of experiments designed to carefully tease apart several competing heuristics, suggests that observers switch between different computationally efficient yet informative cues depending on the information available in the stimulus.

Selecting suitable grasps on three-dimensional objects is a challenging visuomotor computation, which involves combining information about an object (e.g., its shape, size, and mass) with information about the actor’s body (e.g., the optimal grasp aperture and hand posture for comfortable manipulation). Here we used functional magnetic resonance imaging to investigate brain networks associated with these distinct aspects during grasp planning and execution. Human participants viewed and then executed preselected grasps on L-shaped objects made of wood and/or brass. By leveraging a computational approach that accurately predicts human grasp locations, we selected grasp points that disentangled the role of multiple grasp-relevant factors: grasp axis, grasp size, and object mass. Representational Similarity Analysis revealed that grasp axis was encoded along dorsal-stream regions during grasp planning. Grasp size was first encoded in ventral-stream areas during grasp planning, then in premotor regions during grasp execution. Object mass was encoded in ventral-stream and (pre)motor regions only during grasp execution. Premotor regions further encoded visual predictions of grasp comfort, whereas the ventral stream encoded grasp comfort during execution, suggesting its involvement in haptic evaluation. These shifts in neural representations thus capture the sensorimotor transformations that allow humans to grasp objects. Significance Statement Grasping requires integrating object properties with constraints on hand and arm postures. Using a computational approach that accurately predicts human grasp locations by combining such constraints, we selected grasps on objects that disentangled the relative contributions of object mass, grasp size, and grasp axis during grasp planning and execution in a neuroimaging study. Our findings reveal a greater role of dorsal-stream visuomotor areas during grasp planning, and surprisingly, increasing ventral stream engagement during execution. We propose that during planning, visuomotor representations initially encode grasp axis and size. Perceptual representations of object material properties become more relevant instead as the hand approaches the object and motor programs are refined with estimates of the grip forces required to successfully lift the object.

How humans visually select where to grasp objects is determined by the physical object properties (e.g., size, shape, weight), the degrees of freedom of the arm and hand, as well as the task to be performed. We recently demonstrated that human grasps are near-optimal with respect to a weighted combination of different cost functions that make grasps uncomfortable, unstable or impossible e.g., due to unnatural grasp apertures or large torques. Here, we ask whether humans can consciously access these rules. We test if humans can explicitly judge grasp quality derived from rules regarding grasp size, orientation, torque, and visibility. More specifically, we test if grasp quality can be inferred (i) by using motor imagery alone, (ii) from watching grasps executed by others, and (iii) through performing grasps, i.e. receiving visual, proprioceptive and haptic feedback. Stimuli were novel objects made of 10 cubes of brass and wood (side length 2.5 cm) in various configurations. On each object, one near-optimal and one sub-optimal grasp were selected based on one cost function (e.g. torque), while the other constraints (grasp size, orientation, and visibility) were kept approximately constant or counterbalanced. Participants were visually cued to the location of the selected grasps on each object and verbally reported which of the two grasps was best. Across three experiments, participants could either (i) passively view the static objects, (ii) passively view videos of other participants grasping the objects, or (iii) actively grasp the objects themselves. Our results show that participants could already judge grasp optimality from simply viewing the objects, but were significantly better in the video and grasping session. These findings suggest that humans can determine grasp quality even without performing the grasp—perhaps through motor imagery—and can further refine their understanding of how to correctly grasp an object through sensorimotor feedback but also by passively viewing others grasp objects.

Visually inferring the elasticity of a bouncing object poses a challenge to the visual system: The observable behavior of the object depends on its elasticity but also on extrinsic factors, such as its initial position and velocity. Estimating elasticity requires disentangling these different contributions to the observed motion. We created 2-second simulations of a cube bouncing in a room and varied the cube's elasticity in 10 steps. The cube's initial position, orientation, and velocity were varied randomly to gain three random samples for each level of elasticity. We systematically limited the visual information by creating three versions of each stimulus: (a) a full rendering of the scene, (b) the cube in a completely black environment, and (c) a rigid version of the cube following the same trajectories but without rotating or deforming (also in a completely black environment). Thirteen observers rated the apparent elasticity of the cubes and the typicality of their motion. Generally, stimuli were judged as less typical if they showed rigid motion without rotations, highly elastic cubes, or unlikely events. Overall, elasticity judgments correlated strongly with the true elasticity but did not show perfect constancy. Yet, importantly, we found similar results for all three stimulus conditions, despite significant differences in their apparent typicality. This suggests that the trajectory alone contains the information required to make elasticity judgments.