Michael J. Tarr
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Michael J. Tarr has a total of 65 co-authors. Here are the top co-authors by number of shared publications:
- Leila Wehbe
Carnegie Mellon University
12 shared publications - Margaret M. Henderson
Carnegie Mellon University
8 shared publications - Aria Y. Wang
Neuroscience Institute, Carnegie Mellon University
4 shared publications - Jacob S. Prince
Harvard University
4 shared publications - Robert E. Kass
Machine Learning Department, Carnegie Mellon University
4 shared publications - Kendrick Kay
Center for Magnetic Resonance Research (CMRR), Department of Radiology, University of Minnesota
3 shared publications
Latest Publications
- Open Access
- 30 October 2026
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 21 (10) , e0361502
BitterBuster: A virtual world for understanding naturalistic patch foraging
The past decade has seen a resurgence of interest in the naturalistic behavior of patch foraging in both the neural and cognitive sciences. Patch foraging is observed throughout the animal kingdom and requires the integration of multiple cognitive processes, including value and perceptual learning, spatial estimation, the evaluation of instantaneous return rates, and anticipating future effort. Here, we present a novel virtual environment, BitterBuster, for studying patch foraging in humans. In the BitterBuster game, players navigate a virtual world with four neighborhoods (patches) populated with houses (harvest points) containing inhabitants known as “Oomplets”. The goal is to search for “bitter” Oomplets, characterized by the union of specific body colors and textures. Each neighborhood has a different overall probability of containing bitter Oomplets and, within each, specific house locations predict the relative rate of bitter Oomplets. BitterBuster can be used for solo or team based foraging, where the tasks of exploring and perceptual learning are implemented by two separate players working together. Our results show how both the solo and team version of BitterBuster can accurately measure multiple aspects of naturalistic patch foraging, with participants learning to identify optimal patches, classify harvestable resources, and make rational decisions about when to leave a patch before overharvesting. BitterBuster is well-suited for researchers interested in the contextual, neural, and cooperative dynamics of foraging.
- 30 October 2026
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 21 (10) , e0361502
Open AccessBitterBuster: A virtual world for understanding naturalistic patch foraging
The past decade has seen a resurgence of interest in the naturalistic behavior of patch foraging in both the neural and cognitive sciences. Patch foraging is observed throughout the animal kingdom and requires the integration of multiple cognitive processes, including value and perceptual learning, spatial estimation, the evaluation of instantaneous return rates, and anticipating future effort. Here, we present a novel virtual environment, BitterBuster, for studying patch foraging in humans. In the BitterBuster game, players navigate a virtual world with four neighborhoods (patches) populated with houses (harvest points) containing inhabitants known as “Oomplets”. The goal is to search for “bitter” Oomplets, characterized by the union of specific body colors and textures. Each neighborhood has a different overall probability of containing bitter Oomplets and, within each, specific house locations predict the relative rate of bitter Oomplets. BitterBuster can be used for solo or team based foraging, where the tasks of exploring and perceptual learning are implemented by two separate players working together. Our results show how both the solo and team version of BitterBuster can accurately measure multiple aspects of naturalistic patch foraging, with participants learning to identify optimal patches, classify harvestable resources, and make rational decisions about when to leave a patch before overharvesting. BitterBuster is well-suited for researchers interested in the contextual, neural, and cooperative dynamics of foraging.
- Open Access
Diffusion-based stimulus optimization reveals functional organization across higher visual cortex
Characterizing the fine-grained functional organization of human higher visual cortex remains a central challenge, as traditional neuroimaging experiments constrain the diversity of stimuli that can be sampled. In prior work we addressed this challenge by developing a novel data-driven tool, termed “BrainDiVE” (Luo et al. 2023), which synthesizes naturalistic images predicted to strongly activate specific brain regions. BrainDiVE leverages pretrained image diffusion models guided by gradients from an image-computable fMRI encoding model. Here, we experimentally validated BrainDiVE by generating images predicted to maximally activate different functional regions of interest and then presenting them to new participants (n=12) in an fMRI study. The model-generated images elicited robust, spatially specific responses in the targeted brain regions, producing significantly greater category selectivity than natural images, validating the method’s ability to capture generalizable neural tuning properties in human ventral visual cortex. We further showed that region-targeted images exaggerate specific sets of low-level and mid-level image statistics, suggesting that category-selective regions are tuned to continuous directions in feature space. Moreover, we demonstrated fine-grained experimental control by differentially activating two face-selective regions, the occipital face area (OFA) and fusiform face area (FFA), providing additional evidence that these regions encode distinct aspects of faces. Finally, we identify a posterior-to-anterior functional gradient within the occipital place area (OPA), suggesting topographic organization based on scene properties such as distance and indoor-outdoor location. These findings enhance our understanding of the representational structure of category-selective regions and introduce a new paradigm for probing neural selectivity in human visual cortex.
Open AccessDiffusion-based stimulus optimization reveals functional organization across higher visual cortex
Characterizing the fine-grained functional organization of human higher visual cortex remains a central challenge, as traditional neuroimaging experiments constrain the diversity of stimuli that can be sampled. In prior work we addressed this challenge by developing a novel data-driven tool, termed “BrainDiVE” (Luo et al. 2023), which synthesizes naturalistic images predicted to strongly activate specific brain regions. BrainDiVE leverages pretrained image diffusion models guided by gradients from an image-computable fMRI encoding model. Here, we experimentally validated BrainDiVE by generating images predicted to maximally activate different functional regions of interest and then presenting them to new participants (n=12) in an fMRI study. The model-generated images elicited robust, spatially specific responses in the targeted brain regions, producing significantly greater category selectivity than natural images, validating the method’s ability to capture generalizable neural tuning properties in human ventral visual cortex. We further showed that region-targeted images exaggerate specific sets of low-level and mid-level image statistics, suggesting that category-selective regions are tuned to continuous directions in feature space. Moreover, we demonstrated fine-grained experimental control by differentially activating two face-selective regions, the occipital face area (OFA) and fusiform face area (FFA), providing additional evidence that these regions encode distinct aspects of faces. Finally, we identify a posterior-to-anterior functional gradient within the occipital place area (OPA), suggesting topographic organization based on scene properties such as distance and indoor-outdoor location. These findings enhance our understanding of the representational structure of category-selective regions and introduce a new paradigm for probing neural selectivity in human visual cortex.
- Open Access
- 18 March 2025
- Published by Springer Nature in Scientific Reports
- Vol. 15 (1) , 1-11
The Oomplet dataset toolkit as a flexible and extensible system for large-scale, multi-category image generation
The modern study of perceptual learning across humans, non-human animals, and artificial agents requires large-scale datasets with flexible, customizable, and controllable features for distinguishing between categories. To support this research, we developed the Oomplet Dataset Toolkit (ODT), an open-source, publicly available toolbox capable of generating 9.1 million unique visual stimuli across ten feature dimensions. Each stimulus is a cartoon-like humanoid character, termed an “Oomplet,” designed to be an instance within clearly defined visual categories that are engaging and suitable for use with diverse groups, including children. Experiments show that adults can use four to five of the ten dimensions as single classification criteria in simple perceptual discrimination tasks, underscoring the toolkit’s flexibility. With the ODT, researchers can dynamically generate large, novel stimulus sets to study perceptual learning across biological and artificial contexts.
- 18 March 2025
- Published by Springer Nature in Scientific Reports
- Vol. 15 (1) , 1-11
Open AccessThe Oomplet dataset toolkit as a flexible and extensible system for large-scale, multi-category image generation
The modern study of perceptual learning across humans, non-human animals, and artificial agents requires large-scale datasets with flexible, customizable, and controllable features for distinguishing between categories. To support this research, we developed the Oomplet Dataset Toolkit (ODT), an open-source, publicly available toolbox capable of generating 9.1 million unique visual stimuli across ten feature dimensions. Each stimulus is a cartoon-like humanoid character, termed an “Oomplet,” designed to be an instance within clearly defined visual categories that are engaging and suitable for use with diverse groups, including children. Experiments show that adults can use four to five of the ten dimensions as single classification criteria in simple perceptual discrimination tasks, underscoring the toolkit’s flexibility. With the ODT, researchers can dynamically generate large, novel stimulus sets to study perceptual learning across biological and artificial contexts.
- Open Access
- 18 January 2024
- Published by Center for Open Science in PsyArXiv
Gatekeeping Without Peer Review
Along with efforts to make peer review more equitable and less biased, journals and their editors should consider ways of improving the editorial "desk review" process.
- 18 January 2024
- Published by Center for Open Science in PsyArXiv
Open AccessGatekeeping Without Peer Review
Along with efforts to make peer review more equitable and less biased, journals and their editors should consider ways of improving the editorial "desk review" process.
- Open Access
- 1 January 2024
- Published by Curran Associates, Inc. in Biophysical Journal
- p. 137999-138031
Divergences between Language Models and Human Brains
Do machines and humans process language in similar ways? Recent research has hinted at the affirmative, showing that human neural activity can be effectively predicted using the internal representations of language models (LMs). Although such results are thought to reflect shared computational principles between LMs and human brains, there are also clear differences in how LMs and humans represent and use language. In this work, we systematically explore the divergences between human and machine language processing by examining the differences between LM representations and human brain responses to language as measured by Magnetoencephalography (MEG) across two datasets in which subjects read and listened to narrative stories. Using an LLM-based data-driven approach, we identify two domains that LMs do not capture well: social/emotional intelligence and physical commonsense. We validate these findings with human behavioral experiments and hypothesize that the gap is due to insufficient representations of social/emotional and physical knowledge in LMs. Our results show that fine-tuning LMs on these domains can improve their alignment with human brain responses.
- 1 January 2024
- Published by Curran Associates, Inc. in Biophysical Journal
- p. 137999-138031
Open AccessDivergences between Language Models and Human Brains
Do machines and humans process language in similar ways? Recent research has hinted at the affirmative, showing that human neural activity can be effectively predicted using the internal representations of language models (LMs). Although such results are thought to reflect shared computational principles between LMs and human brains, there are also clear differences in how LMs and humans represent and use language. In this work, we systematically explore the divergences between human and machine language processing by examining the differences between LM representations and human brain responses to language as measured by Magnetoencephalography (MEG) across two datasets in which subjects read and listened to narrative stories. Using an LLM-based data-driven approach, we identify two domains that LMs do not capture well: social/emotional intelligence and physical commonsense. We validate these findings with human behavioral experiments and hypothesize that the gap is due to insufficient representations of social/emotional and physical knowledge in LMs. Our results show that fine-tuning LMs on these domains can improve their alignment with human brain responses.
- 6 December 2023
- Published by Cambridge University Press (CUP) in Behavioral and Brain Sciences
- Vol. 46, e410
My pet pig won't fly and I want a refund
Pigs can't fly. Any person buying a pig should understand this – it would be absurd to be upset that they can't fly or play poker. But pigs are amazing creatures and can do many interesting and useful things.
- 6 December 2023
- Published by Cambridge University Press (CUP) in Behavioral and Brain Sciences
- Vol. 46, e410
My pet pig won't fly and I want a refund
Pigs can't fly. Any person buying a pig should understand this – it would be absurd to be upset that they can't fly or play poker. But pigs are amazing creatures and can do many interesting and useful things.
- Open Access
Brain Diffusion for Visual Exploration: Cortical Discovery using Large Scale Generative Models
A long standing goal in neuroscience has been to elucidate the functional organization of the brain. Within higher visual cortex, functional accounts have remained relatively coarse, focusing on regions of interest (ROIs) and taking the form of selectivity for broad categories such as faces, places, bodies, food, or words. Because the identification of such ROIs has typically relied on manually assembled stimulus sets consisting of isolated objects in non-ecological contexts, exploring functional organization without robust a priori hypotheses has been challenging. To overcome these limitations, we introduce a data-driven approach in which we synthesize images predicted to activate a given brain region using paired natural images and fMRI recordings, bypassing the need for category-specific stimuli. Our approach -- Brain Diffusion for Visual Exploration ("BrainDiVE") -- builds on recent generative methods by combining large-scale diffusion models with brain-guided image synthesis. Validating our method, we demonstrate the ability to synthesize preferred images with appropriate semantic specificity for well-characterized category-selective ROIs. We then show that BrainDiVE can characterize differences between ROIs selective for the same high-level category. Finally we identify novel functional subdivisions within these ROIs, validated with behavioral data. These results advance our understanding of the fine-grained functional organization of human visual cortex, and provide well-specified constraints for further examination of cortical organization using hypothesis-driven methods.
Open AccessBrain Diffusion for Visual Exploration: Cortical Discovery using Large Scale Generative Models
A long standing goal in neuroscience has been to elucidate the functional organization of the brain. Within higher visual cortex, functional accounts have remained relatively coarse, focusing on regions of interest (ROIs) and taking the form of selectivity for broad categories such as faces, places, bodies, food, or words. Because the identification of such ROIs has typically relied on manually assembled stimulus sets consisting of isolated objects in non-ecological contexts, exploring functional organization without robust a priori hypotheses has been challenging. To overcome these limitations, we introduce a data-driven approach in which we synthesize images predicted to activate a given brain region using paired natural images and fMRI recordings, bypassing the need for category-specific stimuli. Our approach -- Brain Diffusion for Visual Exploration ("BrainDiVE") -- builds on recent generative methods by combining large-scale diffusion models with brain-guided image synthesis. Validating our method, we demonstrate the ability to synthesize preferred images with appropriate semantic specificity for well-characterized category-selective ROIs. We then show that BrainDiVE can characterize differences between ROIs selective for the same high-level category. Finally we identify novel functional subdivisions within these ROIs, validated with behavioral data. These results advance our understanding of the fine-grained functional organization of human visual cortex, and provide well-specified constraints for further examination of cortical organization using hypothesis-driven methods.
- Top Cited Papers
- 13 November 2023
- Published by Springer Nature in Nature Machine Intelligence
- Vol. 5 (12) , 1415-1426
Better models of human high-level visual cortex emerge from natural language supervision with a large and diverse dataset
- 13 November 2023
- Published by Springer Nature in Nature Machine Intelligence
- Vol. 5 (12) , 1415-1426
Top Cited PapersBetter models of human high-level visual cortex emerge from natural language supervision with a large and diverse dataset
- Open Access
AlphaNet: Improving Long-Tail Classification By Combining Classifiers
Methods in long-tail learning focus on improving performance for data-poor (rare) classes; however, performance for such classes remains much lower than performance for more data-rich (frequent) classes. Analyzing the predictions of long-tail methods for rare classes reveals that a large number of errors are due to misclassification of rare items as visually similar frequent classes. To address this problem, we introduce AlphaNet, a method that can be applied to existing models, performing post hoc correction on classifiers of rare classes. Starting with a pre-trained model, we find frequent classes that are closest to rare classes in the model's representation space and learn weights to update rare class classifiers with a linear combination of frequent class classifiers. AlphaNet, applied to several models, greatly improves test accuracy for rare classes in multiple long-tailed datasets, with very little change to overall accuracy. Our method also provides a way to control the trade-off between rare class and overall accuracy, making it practical for long-tail classification in the wild.
Open AccessAlphaNet: Improving Long-Tail Classification By Combining Classifiers
Methods in long-tail learning focus on improving performance for data-poor (rare) classes; however, performance for such classes remains much lower than performance for more data-rich (frequent) classes. Analyzing the predictions of long-tail methods for rare classes reveals that a large number of errors are due to misclassification of rare items as visually similar frequent classes. To address this problem, we introduce AlphaNet, a method that can be applied to existing models, performing post hoc correction on classifiers of rare classes. Starting with a pre-trained model, we find frequent classes that are closest to rare classes in the model's representation space and learn weights to update rare class classifiers with a linear combination of frequent class classifiers. AlphaNet, applied to several models, greatly improves test accuracy for rare classes in multiple long-tailed datasets, with very little change to overall accuracy. Our method also provides a way to control the trade-off between rare class and overall accuracy, making it practical for long-tail classification in the wild.
- Open Access
Thinking Like an Annotator: Generation of Dataset Labeling Instructions
Large-scale datasets are essential to modern day deep learning. Advocates argue that understanding these methods requires dataset transparency (e.g. "dataset curation, motivation, composition, collection process, etc..."). However, almost no one has suggested the release of the detailed definitions and visual category examples provided to annotators - information critical to understanding the structure of the annotations present in each dataset. These labels are at the heart of public datasets, yet few datasets include the instructions that were used to generate them. We introduce a new task, Labeling Instruction Generation, to address missing publicly available labeling instructions. In Labeling Instruction Generation, we take a reasonably annotated dataset and: 1) generate a set of examples that are visually representative of each category in the dataset; 2) provide a text label that corresponds to each of the examples. We introduce a framework that requires no model training to solve this task and includes a newly created rapid retrieval system that leverages a large, pre-trained vision and language model. This framework acts as a proxy to human annotators that can help to both generate a final labeling instruction set and evaluate its quality. Our framework generates multiple diverse visual and text representations of dataset categories. The optimized instruction set outperforms our strongest baseline across 5 folds by 7.06 mAP for NuImages and 12.9 mAP for COCO.
Open AccessThinking Like an Annotator: Generation of Dataset Labeling Instructions
Large-scale datasets are essential to modern day deep learning. Advocates argue that understanding these methods requires dataset transparency (e.g. "dataset curation, motivation, composition, collection process, etc..."). However, almost no one has suggested the release of the detailed definitions and visual category examples provided to annotators - information critical to understanding the structure of the annotations present in each dataset. These labels are at the heart of public datasets, yet few datasets include the instructions that were used to generate them. We introduce a new task, Labeling Instruction Generation, to address missing publicly available labeling instructions. In Labeling Instruction Generation, we take a reasonably annotated dataset and: 1) generate a set of examples that are visually representative of each category in the dataset; 2) provide a text label that corresponds to each of the examples. We introduce a framework that requires no model training to solve this task and includes a newly created rapid retrieval system that leverages a large, pre-trained vision and language model. This framework acts as a proxy to human annotators that can help to both generate a final labeling instruction set and evaluate its quality. Our framework generates multiple diverse visual and text representations of dataset categories. The optimized instruction set outperforms our strongest baseline across 5 folds by 7.06 mAP for NuImages and 12.9 mAP for COCO.