Unsupervised learning reveals interpretable latent representations for translucency perception
Authors:
Preprint
Abstract
Humans constantly assess the appearance of materials to plan actions, such as stepping on icy roads without slipping. Visual inference of materials is challenging because a given material can appear dramatically different in various scenes. This problem especially stands out for translucent materials (e.g., skin, soap), whose appearance strongly depends on lighting, geometry, and viewpoint. Despite this, humans can still distinguish between different materials, and it remains unsolved how to systematically discover visual features pertinent to material inference. Here, we developed an unsupervised style-based image generation model to identify perceptually relevant dimensions for material appearances from natural photographs. We find the model can synthesize images of convincing material appearances. Importantly, without supervision, human-understandable scene attributes, including object’s shape, material, and body color, spontaneously emerge in the model’s layer-wise latent space in a scale-specific manner. Crucially, the middle-layers of the latent space selectively encode translucency features correlated with perception, suggesting that translucent impressions are established in mid-to-low spatial scale features. Our findings indicate that humans may use the scale-specific statistical structure of natural images to efficiently represent material properties across contexts, and our approach is widely applicable in discovering perceptually relevant features from complex stimuli for many visual inference tasks.Keywords
All Related Versions
- Published version: PLOS Computational Biology, 19 (2), e1010878.
This publication has 106 references indexed in Scilit:
- The Perception and Misperception of Specular Surface ReflectanceCurrent Biology, 2012
- Visual Motion and the Perception of Surface MaterialCurrent Biology, 2011
- Joint Effects of Illumination Geometry and Object Shape in the Perception of Surface ReflectanceI-Perception, 2011
- NWChem: A comprehensive and scalable open-source solution for large scale molecular simulationsComputer Physics Communications, 2010
- The role of color in recognizing material changesOphthalmic and Physiological Optics, 2010
- PsychoPy—Psychophysics software in PythonJournal of Neuroscience Methods, 2007
- How direction of illumination affects visually perceived surface roughnessJournal of Vision, 2006
- Efficient auditory codingNature, 2006
- Low-Level Image Cues in the Perception of Translucent MaterialsACM Transactions on Applied Perception, 2005
- Real-world illumination and the perception of surface reflectance propertiesJournal of Vision, 2003