Henning Tiedemann

How do we make sense of something we’ve never seen before? Classifying objects into superordinate classes like ‘animal’ is a key step in interpreting novel experiences, but is challenging because radically different items (e.g., octopus, rabbit) must somehow be grouped together. In general, no single feature is shared by all members. We reasoned that to classify or imagine novel items from outside the distribution of previous experiences, observers parse objects into meaningful component features that they can mentally recombine (‘compositionality’). To test this, we asked participants to draw familiar and novel members of nine superordinate object classes. We then asked other participants to classify the drawings, and mark and label their defining ‘parts’. We find that human classification performance is well predicted by a Bayesian classifier that optimally combines the part labels, suggesting humans can create and classify out-of-distribution experiences through a compositional generative representation of object features.

Many objects and materials in our environment are subject to transformations that alter their shape. For example, branches bend in the wind, ice melts, and paper crumples. Still, we recognize objects and materials across these changes, suggesting we can distinguish an object’s original features from those caused by the transformations (“shape scission”). Yet, if we truly understand transformations, we should not only be able to identify their signatures but also actively apply the transformations to new objects (i.e., through imagination or mental simulation). Here, we investigated this ability using a drawing task. On a tablet computer, participants viewed a sample contour and its transformed version, and were asked to apply the same transformation to a test contour by drawing what the transformed test shape should look like. Thus, they had to (i) infer the transformation from the shape differences, (ii) envisage its application to the test shape, and (iii) draw the result. Our findings show that drawings were more similar to the ground truth transformed test shape than to the original test shape—demonstrating the inference and reproduction of transformations from observation. However, this was only observed for relatively simple shapes. The ability was also modulated by transformation type and magnitude but not by the similarity between sample and test shapes. Together, our findings suggest that we can distinguish between representations of original object shapes and their transformations, and can use visual imagery to mentally apply nonrigid transformations to observed objects, showing how we not only perceive but also ‘understand’ shape.

Shape perception is essential for numerous everyday behaviors from object recognition to grasping and handling objects. Yet how the brain encodes shape remains poorly understood. Here, we probed shape representations using visual aftereffects—perceptual distortions that occur following extended exposure to a stimulus—to resolve a long-standing debate about shape encoding. We implemented contrasting low-level and high-level computational models of neural adaptation, which made precise and distinct predictions about the illusory shape distortions the observers experience following adaptation. Directly pitting the predictions of the two models against one another revealed that the perceptual distortions are driven by high-level shape attributes derived from the statistics of natural shapes. Our findings suggest that the diverse shape attributes thought to underlie shape encoding (e.g., curvature distributions, ‘skeletons’, aspect ratio) are the result of a visual system that learns to encode natural shape geometries based on observing many objects.

Plants and animals are among the most behaviorally significant superordinate categories for humans. Visually assigning objects to such high-level classes is challenging because highly distinct items must be grouped together (e.g., chimpanzees and geckos) while more similar items must sometimes be separated (e.g., stick insects and twigs). As both animals and plants typically possess complex multi-limbed shapes, the perceptual organization of shape into parts likely plays a crucial rule in identifying them. Here, we identify a number of distinctive growth characteristics that affect the spatial arrangement and properties of limbs, yielding useful cues for differentiating plants from animals. We developed a novel algorithm based on shape skeletons to create many novel object pairs that differ in their part structure but are otherwise very similar. We found that particular part organizations cause stimuli to look systematically more like plants or animals. We then generated other 110 sequences of shapes morphing from animal- to plant-like appearance by modifying three aspects of part structure: sprouting parts, curvedness of parts, and symmetry of part pairs. We found that all three parameters correlated strongly with human animal/plant judgments. Together our findings suggest that subtle changes in the properties and organization of parts can provide powerful cues in superordinate categorization.

Humans have the amazing ability to learn new visual concepts from just a single exemplar. How we achieve this remains mysterious. State-of-the-art theories suggest observers rely on internal ‘generative models’, which not only describe observed objects, but can also synthesize novel variations. However, compelling evidence for generative models in human one-shot learning remains sparse. In most studies, participants merely compare candidate objects created by the experimenters, rather than generating their own ideas. Here, we overcame this key limitation by presenting participants with 2D ‘Exemplar’ shapes and asking them to draw their own ‘Variations’ belonging to the same class. The drawings reveal that participants inferred—and synthesized—genuine novel categories that were far more varied than mere copies. Yet, there was striking agreement between participants about which shape features were most distinctive, and these tended to be preserved in the drawn Variations. Indeed, swapping distinctive parts caused objects to swap apparent category. Our findings suggest that internal generative models are key to how humans generalize from single exemplars. When observers see a novel object for the first time, they identify its most distinctive features and infer a generative model of its shape, allowing them to mentally synthesize plausible variants.

Shape is a defining feature of objects, and human observers can effortlessly compare shapes to determine how similar they are. Yet, to date, no image-computable model can predict how visually similar or different shapes appear. Such a model would be an invaluable tool for neuroscientists and could provide insights into computations underlying human shape perception. To address this need, we developed a model (‘ShapeComp’), based on over 100 shape features (e.g., area, compactness, Fourier descriptors). When trained to capture the variance in a database of >25,000 animal silhouettes, ShapeComp accurately predicts human shape similarity judgments between pairs of shapes without fitting any parameters to human data. To test the model, we created carefully selected arrays of complex novel shapes using a Generative Adversarial Network trained on the animal silhouettes, which we presented to observers in a wide range of tasks. Our findings show that incorporating multiple ShapeComp dimensions facilitates the prediction of human shape similarity across a small number of shapes, and also captures much of the variance in the multiple arrangements of many shapes. ShapeComp outperforms both conventional pixel-based metrics and state-of-the-art convolutional neural networks, and can also be used to generate perceptually uniform stimulus sets, making it a powerful tool for investigating shape and object representations in the human brain.

Humans have the striking ability to learn and generalize new visual concepts from just a single exemplar. We suggest that when presented with a novel object, observers identify its significant features and infer a generative model of its shape, allowing them to mentally synthesize plausible variants. To test this, we showed participants abstract 2D shapes (‘Exemplars’) and asked them to draw new objects (‘Variations’) belonging to the same class. We show that this procedure created genuine novel categories. In line with our hypothesis, particular features of each Exemplar were preserved in its Variations and there was striking agreement between participants about which shape features were most distinctive. Also, we show that strategies to create Variations were strongly driven by part structure: new objects typically modified individual parts (e.g., limbs) of the Exemplar, often preserving part order, sometimes altering it. Together, our findings suggest that sophisticated internal generative models are key to how humans analyze and generalize from single exemplars.

When presented with just a single exemplar of an unfamiliar object, we have certain visual intuitions about what other objects from the same class might look like. This is an impressive inference because it is radically under-constrained. Objects can vary along many dimensions, so how do we work out which features of the object are important and which are likely to vary across samples? To investigate this, 17 participants were shown 2D silhouettes of 8 exemplar objects from different classes, which were created to have one or multiple noticeable features (e.g. Sharp corners, distinctive concavities). For each of these, they were asked to draw 12 new objects belonging to the same class, on a tablet computer, resulting in 1632 drawn shapes. The drawings reveal that participants derived very specific inferences about the objects, varying some features substantially, while assiduously preserving others across variants. Despite substantial variations within each class, participants were highly consistent in the features that they chose to include in the variants. Another group of participants viewed the drawings and were asked to rate similarities and assign the variants to the categories. The results reveal high agreement between observers, suggesting robust and consistent inferences. We also analysed the shapes using dozens of image-computable shape metrics (e.g., area, perimeter, curvature statistics, fourier descriptors). Using MDS, the drawn shapes were compared to each other and the exemplars, revealing systematic variations in the features that defined each class. Together, these findings suggest participants infer sophisticated generative models of object appearance from single exemplars. We suggest that when presented with a shape, the visual system parses it, identifies perceptually significant features, and represents the features in a parametric way, providing a means to draw new samples from the internal representation (by varying the parameters of the feature representation).