Sripati P. Arun

Print knowledge is known to be important for early reading acquisition, but it has only been measured using letter recognition. However, early readers may possess another kind of print knowledge: they may be familiar with letter shapes even before they learn letter names. Here, we investigated this possibility by testing 300 children at the beginning of formal reading instruction of the Indian language Kannada on four tasks. Each child performed a letter shape familiarity task, a letter recognition task, a visual search task involving letters, and a rapid automatized digit naming task. Our main finding is that, even though many children had low accuracy on naming letters, they were highly familiar with letter shapes, presumably through passive exposure to print in their environment. This familiarity was also associated uniquely with a visual processing advantage: children with high letter familiarity also searched faster for upright compared to inverted letters. Our results suggest that print knowledge must be understood in terms of both letter shape and letter name knowledge, which could have potentially different learning trajectories.

Vision and its neural basis have always been studied in animals that are artificially restrained to obtain stable neural recordings. This approach, while insightful, precludes understanding behaviors like sleep that take place only in natural settings. What happens to neurons in the high-level visual cortex during sleep? Do neural responses to objects seen while awake get reactivated during sleep, just like navigation replay in the hippocampus? To investigate these issues, we performed wireless neural recordings from the high-level visual (IT) and motor regions (PMv, PFC) in two monkeys while they performed a fixation task in which they viewed food images and subsequently slept in their natural setting. Decoders trained on neural activity were able to predict the sleep/wake state as well as transitions between states well in advance. Importantly, decoders trained on neural responses to food images (but not faces/objects) showed higher food probability during REM-like sleep compared to NREM or awake states, suggesting that the monkeys dreamt about food. Thus, neural signatures of food seen while awake are reactivated during sleep.

Our knowledge of the neural basis of sensory and motor information processing comes largely from recording brain activity in controlled settings. This approach, while insightful, blinds us from understanding natural behaviors, which involve far richer visual inputs and far larger range of movements. The prevailing belief is that, during natural behaviors, movement information would modulate visual cortex, and visual inputs would modulate motor cortex. Alternatively, this mixing might only reflect correlations between sensory inputs and movements present during natural behavior. To resolve this issue, we performed wireless recordings from high-level visual (IT) and motor regions (PMv/PFC) in freely moving monkeys while they performed a natural task and a fixation task. In each task, we extracted visual and movement features and asked to what degree these features explained neural activity. In the fixation task, we found a striking double dissociation whereby visual (but not movement) features predicted neural activity in IT, whereas motor (but not visual) features predicted neural activity in PMv & PFC. This dissociation was far weaker in the natural task but present particularly at key events when the visual and motor features were less correlated. Thus, high-level sensory and motor regions in the brain are doubly dissociated under controlled conditions, and remain so at specific times during natural behavior.

Monkeys are widely used as model organisms for human vision and cognition. While their anatomy and physiology strongly correspond to humans, it is unclear to what extent their perception matches with ours. Previous studies have evaluated specific aspects of perception, after extensive training on customized tasks that could have altered their perception. To resolve these issues, we trained monkeys to perform an oddball visual search task on natural images, tested them on carefully controlled images to detect a variety of perceptual phenomena, and compared them with humans. This revealed a number of qualitative similarities and differences. Like humans, monkeys showed similar object relations, Weber’s law and amodal completion. However, unlike humans, monkeys did not show mirror confusion or a global advantage. These findings represent a first comprehensive evaluation of visual perception in monkeys and humans, revealing the limitations of monkeys as a model organism for human vision.

Natural behaviors are challenging to study because they involve large variations in sensory inputs and motor outputs across trials. Even the simple act of eating can involve seeing food from many angles and reaching for it differently each time. The prevailing view is that natural behaviors are more noisy compared to controlled tasks, and involve mixing of sensorimotor information. Alternatively, natural behaviors might elicit reliable neural signatures in high-level visual and motor regions which contain invariant representations, and involve correlated sensory and motor information that leads to apparent mixing. Here, we provide evidence for the latter possibility through wireless recordings in freely moving monkeys performing natural tasks as well as controlled screen tasks. During natural interactions with food, neural activity was decodable with striking fidelity, equivalent to that of food images. Key events during these interactions elicited distinct and reliable neural signatures that reflected the encoding priorities of each region. Visual and motor regions were clearly dissociated in screen tasks, but were dissociated only at specific events in the natural task. Neurons active while seeing food were reactivated during sleep. Taken together, our results reveal that reliable neural signatures in high-level sensory and motor regions form the neural substrates for natural behaviors.

Primates are deeply social, but the understanding the neural basis has been challenging because brain activity has previously been recorded in only one animal interacting with a social partner. Here, we recorded wireless neural activity simultaneously from two macaque monkeys interacting socially in a natural, unconstrained setting. Neural activity in each monkey encoded key social variables such as allogrooming state, partner identity and joint movements. Surprisingly, neurons in the high-level visual cortex of each monkey continuously tracked a social favor signal, i.e. net duration of allogrooming given versus received, providing a neural basis for grooming reciprocity. In each monkey, neural activity was predicted better by his partner’s joints and neural activity, particularly while receiving compared to giving grooming. Thus, the receiver of grooming drove the social interaction, not the groomer. Taken together, our findings elucidate the rich and dynamic neural basis of primate social interactions in an ecologically valid setting.

Reading is an acquired skill that enhances brain activity in human visual cortex and is thought to repurpose preexisting ventral visual circuitry for the fast parallel processing of letter strings. However, the neuronal code underlying position-invariant written word recognition remains elusive. Here, we examined this issue in a macaque monkey model before and after orthographic training. Based on prior simulations of reading acquisition in a convolutional neural network model of the ventral visual pathway, we hypothesized the existence of neurons sensitive to both specific letters and their ordinal position within the word, which should be enhanced by orthographic training. By wirelessly recorded neural activity from the inferior temporal (IT) cortex of macaque monkeys trained on orthographic tasks over five successive days, we indeed discovered IT neurons tuned to letters and sensitive to either ordinal or retinotopic positions. Ordinal units existed prior to training, but their responses were enhanced after training, matching with behavioral improvements in word recognition. These findings support the neuronal recycling hypothesis and demonstrate, at the single-cell level, how reading refines pre-existing neural circuits to facilitate fluent word recognition.

Languages have evolved in part due to cross-modal associations between shape and sound. A famous example is the Bouba-Kiki effect, wherein humans associate words like bouba/kiki to round/angular shapes. How does the Bouba-Kiki effect work for natural words and shapes that contain a mixture of features? If the effect is holistic, the effect for a composite stimulus would not be explainable using the parts. If the effect is compositional, it will be. Here we provide evidence for the latter possibility. In Experiments 1 and 2, we standardized bouba-like and kiki-like shapes and words for use in subsequent experiments. In Experiments 3-5, we created composite shapes/words by combining bouba-like & kiki-like parts. In all experiments, the Bouba-Kiki effect strength for composite shapes/words was predicted remarkably well as a linear sum of the contributions of the constituent parts. Our results greatly simplify our understanding of the Bouba-Kiki effect, leaving little room for holism.

Most visual tasks involve looking for specific object features. But we also often perform property-based tasks where we look for specific property in an image, such as finding an odd item, deciding if two items are same, or if an object has symmetry. How do we solve such tasks? These tasks do not fit into standard models of decision making because their underlying feature space and decision process is unclear. Using well-known principles governing multiple object representations, we show that displays with repeating elements can be distinguished from heterogeneous displays using a property we define as visual homogeneity. In behavior, visual homogeneity predicted response times on visual search, same-different and symmetry tasks. Brain imaging during visual search and symmetry tasks revealed that visual homogeneity was localized to a region in the object-selective cortex. Thus, property-based visual tasks are solved in a localized region in the brain by computing visual homogeneity.

Monkeys are widely used as model organisms for human vision and cognition. While their anatomy and physiology strongly correspond to humans, it is unclear to what extent their perception matches with ours. Previous studies have evaluated specific aspects of perception, after extensive training on customized tasks that could have altered their perception. To resolve these issues, we trained monkeys to perform an oddball visual search task on natural images, tested them on carefully controlled images to detect a variety of perceptual phenomena, and compared them with humans. This revealed a number of qualitative similarities and differences. Like humans, monkeys showed similar object relations, Weber’s law and amodal completion. However, unlike humans, monkeys did not show mirror confusion or a global advantage. These findings represent a first comprehensive evaluation of visual perception in monkeys and humans, revealing the limitations of monkeys as a model organism for human vision. SIGNIFICANCE STATEMENT: Monkeys are widely used to study for vision and cognition, but is their perception like ours? Most previous studies have tested only single aspects of perception in monkeys, often after extensive training which could have altered their perception. Here, we overcame these limitations by training monkeys on a highly general oddball visual search task, testing them on many perceptual phenomena and comparing them with humans on the same tasks. Our results show that monkeys do indeed see the way we do, but with important differences that highlight their limitations as a model for human vision.