CLChenxi Liao

Chenxi Liao

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Chenxi Liao has a total of 11 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Material recognition from cross-category morphing

    by Chenxi Liao, Filipp Schmidt, Jacob R. Cheeseman, Masataka Sawayama, Roland W. Fleming, Bei Xiao

    Materials from distinct categories share common image features but also exhibit unique characteristics. How do humans recognize materials despite their enormous appearance variations? Using generative neural networks, we create both prototypical and ambiguous materials that morph between categories. To capture the richness of material representation, we characterize how people judge materials using cross-material morphs through three behavioral tasks: material categorization, material property rating, and visual similarity judgment. We find that morphing smoothly modulates perceptual scales of material appearance. Despite this smooth variation, participants can reliably identify the prototypical materials within given appearance ranges, indicating a strong association between material categories and their visual characteristics. Material properties structure the perceptual space, where the salient dimensions–particularly rigidity–strongly correlate with perceived material categories. In contrast, analyzing image embedding derived from a self-supervised deep learning model reveals that learning visual similarities alone is insufficient to reproduce human perceptual space of materials. Together, our results suggest that material representation is more than visual similarities and may require learning material properties that structure relationships among materials within and across categories. Visual features might be used flexibly and task-dependently to support multiple levels of material perception, ranging from discrimination to material property inference to categorization.

  • Open Access

    From visual appearance to material categories

    by Chenxi Liao, Filipp Schmidt, Jacob Raleigh Cheeseman, Masataka Sawayama, Roland Fleming, Bei Xiao

    Materials from distinct categories share common image features but also exhibit unique characteristics. How do humans recognize materials despite their enormous appearance variations? Using generative neural networks, we create both prototypical and ambiguous materials that morph between categories. To capture the richness of material representation, we characterize how people judge materials using cross-material morphs through three behavioral tasks: material categorization, material property rating, and visual similarity judgment. We find that morphing smoothly modulates perceptual scales of material appearance. Despite this smooth variation, participants can reliably identify the prototypical materials within given appearance ranges, indicating a strong association between material categories and their visual characteristics. Material properties structure the perceptual space, where the salient dimensions--particularly rigidity--strongly correlate with perceived material categories. In contrast, analyzing image embedding derived from a self-supervised deep learning model reveals that learning visual similarities alone is insufficient to reproduce human perceptual space of materials. Together, our results suggest that material representation is more than visual similarities and may require learning material properties that structure relationships among materials within and across categories. Visual features might be used flexibly and task-dependently to support multiple levels of material perception, ranging from discrimination to material property inference to categorization.

  • Open Access

    Probing the link between vision and language in material perception using psychophysics and unsupervised learning

    by Chenxi Liao, Masataka Sawayama, Bei Xiao

    We can visually discriminate and recognize a wide range of materials. Meanwhile, we use language to describe what we see and communicate relevant information about the materials. Here, we investigate the relationship between visual judgment and language expression to understand how visual features relate to semantic representations in human cognition. We use deep generative models to generate images of realistic materials. Interpolating between the generative models enables us to systematically create material appearances in both well-defined and ambiguous categories. Using these stimuli, we compared the representations of materials from two behavioral tasks: visual material similarity judgments and free-form verbal descriptions. Our findings reveal a moderate but significant correlation between vision and language on a categorical level. However, analyzing the representations with an unsupervised alignment method, we discover structural differences that arise at the image-to-image level, especially among ambiguous materials morphed between known categories. Moreover, visual judgments exhibit more individual differences compared to verbal descriptions. Our results show that while verbal descriptions capture material qualities on the coarse level, they may not fully convey the visual nuances of material appearances. Analyzing the image representation of materials obtained from various pre-trained deep neural networks, we find that similarity structures in human visual judgments align more closely with those of the vision-language models than purely vision-based models. Our work illustrates the need to consider the vision-language relationship in building a comprehensive model for material perception. Moreover, we propose a novel framework for evaluating the alignment and misalignment between representations from different modalities, leveraging information from human behaviors and computational models.


  • Open Access

    Probing the Link Between Vision and Language in Material Perception Using Psychophysics and Unsupervised Learning

    by Chenxi Liao, Masataka Sawayama, Bei Xiao

    We can visually discriminate and recognize a wide range of materials. Meanwhile, we use language to express our subjective understanding of visual input and communicate relevant information about the materials. Here, we investigate the relationship between visual judgment and language expression in material perception to understand how visual features relate to semantic representations. We use deep generative networks to construct an expandable image space to systematically create materials of well-defined and ambiguous categories. From such a space, we sampled diverse stimuli and compared the representations of materials from two behavioral tasks: visual material similarity judgments and free-form verbal descriptions. Our findings reveal a moderate but significant correlation between vision and language on a categorical level. However, analyzing the representations with an unsupervised alignment method, we discover structural differences that arise at the image-to-image level, especially among materials morphed between known categories. Moreover, visual judgments exhibit more individual differences compared to verbal descriptions. Our results show that while verbal descriptions capture material qualities on the coarse level, they may not fully convey the visual features that characterize the material’s optical properties. Analyzing the image representation of materials obtained from various pre-trained data-rich deep neural networks, we find that human visual judgments’ similarity structures align more closely with those of the text-guided visual-semantic model than purely vision-based models. Our findings suggest that while semantic representations facilitate material categorization, non-semantic visual features also play a significant role in discriminating materials at a finer level. This work illustrates the need to consider the vision-language relationship in building a comprehensive model for material perception. Moreover, we propose a novel framework for quantitatively evaluating the alignment and misalignment between representations from different modalities, leveraging information from human behaviors and computational models.

  • Open Access

    Unsupervised learning reveals interpretable latent representations for translucency perception

    by Chenxi Liao, Masataka Sawayama, Bei Xiao

    Humans constantly assess the appearance of materials to plan actions, such as stepping on icy roads without slipping. Visual inference of materials is important but challenging because a given material can appear dramatically different in various scenes. This problem especially stands out for translucent materials, whose appearance strongly depends on lighting, geometry, and viewpoint. Despite this, humans can still distinguish between different materials, and it remains unsolved how to systematically discover visual features pertinent to material inference from natural images. Here, we develop an unsupervised style-based image generation model to identify perceptually relevant dimensions for translucent material appearances from photographs. We find our model, with its layer-wise latent representation, can synthesize images of diverse and realistic materials. Importantly, without supervision, human-understandable scene attributes, including the object’s shape, material, and body color, spontaneously emerge in the model’s layer-wise latent space in a scale-specific manner. By embedding an image into the learned latent space, we can manipulate specific layers’ latent code to modify the appearance of the object in the image. Specifically, we find that manipulation on the early-layers (coarse spatial scale) transforms the object’s shape, while manipulation on the later-layers (fine spatial scale) modifies its body color. The middle-layers of the latent space selectively encode translucency features and manipulation of such layers coherently modifies the translucency appearance, without changing the object’s shape or body color. Moreover, we find the middle-layers of the latent space can successfully predict human translucency ratings, suggesting that translucent impressions are established in mid-to-low spatial scale features. This layer-wise latent representation allows us to systematically discover perceptually relevant image features for human translucency perception. Together, our findings reveal that learning the scale-specific statistical structure of natural images might be crucial for humans to efficiently represent material properties across contexts.


  • Open Access

    Unsupervised learning reveals interpretable latent representations for translucency perception

    by Chenxi Liao, Masataka Sawayama, Bei Xiao

    Humans constantly assess the appearance of materials to plan actions, such as stepping on icy roads without slipping. Visual inference of materials is challenging because a given material can appear dramatically different in various scenes. This problem especially stands out for translucent materials (e.g., skin, soap), whose appearance strongly depends on lighting, geometry, and viewpoint. Despite this, humans can still distinguish between different materials, and it remains unsolved how to systematically discover visual features pertinent to material inference. Here, we developed an unsupervised style-based image generation model to identify perceptually relevant dimensions for material appearances from natural photographs. We find the model can synthesize images of convincing material appearances. Importantly, without supervision, human-understandable scene attributes, including object’s shape, material, and body color, spontaneously emerge in the model’s layer-wise latent space in a scale-specific manner. Crucially, the middle-layers of the latent space selectively encode translucency features correlated with perception, suggesting that translucent impressions are established in mid-to-low spatial scale features. Our findings indicate that humans may use the scale-specific statistical structure of natural images to efficiently represent material properties across contexts, and our approach is widely applicable in discovering perceptually relevant features from complex stimuli for many visual inference tasks.


  • Top Cited Papers
    Open Access

    Crystal or jelly? Effect of color on the perception of translucent materials with photographs of real-world objects

    by Chenxi Liao, Masataka Sawayama, Bei Xiao

    Translucent materials are ubiquitous in nature (e.g. teeth, food, and wax), but our understanding of translucency perception is limited. Previous work in translucency perception has mainly used monochromatic rendered images as stimuli, which are restricted by their diversity and realism. Here, we measure translucency perception with photographs of real-world objects. Specifically, we use three behavior tasks: binary classification of “translucent” versus “opaque,” semantic attribute rating of perceptual qualities (see-throughness, glossiness, softness, glow, and density), and material categorization. Two different groups of observers finish the three tasks with color or grayscale images. We find that observers’ agreements depend on the physical material properties of the objects such that translucent materials generate more interobserver disagreements. Further, there are more disagreements among observers in the grayscale condition in comparison to that in the color condition. We also discover that converting images to grayscale substantially affects the distributions of attribute ratings for some images. Furthermore, ratings of see-throughness, glossiness, and glow could predict individual observers’ binary classification of images in both grayscale and color conditions. Last, converting images to grayscale alters the perceived material categories for some images such that observers tend to misjudge images of food as non-food and vice versa. Our result demonstrates that color is informative about material property estimation and recognition. Meanwhile, our analysis shows that mid-level semantic estimation of material attributes might be closely related to high-level material recognition. We also discuss individual differences in our results and highlight the importance of such consideration in material perception.


  • Open Access

    Crystal or Jelly? Effect of Color on the Perception of Translucent Materials with Photographs of Real-world Objects

    by Chenxi Liao, Masataka Sawayama, Bei Xiao

    Translucent materials are ubiquitous in nature (e.g. teeth, food, wax), but our understanding of translucency perception is limited. Previous work in translucency perception has mainly used monochromatic rendered images as stimuli, which are restricted by their diversity and realism. Here, we measure translucency perception with photographs of real-world objects. Specifically, we use three behavior tasks: binary classification of “translucent” versus “opaque”, semantic attribute rating of perceptual qualities (see-throughness, glossiness, softness, glow and density), and material categorization. Two different groups of observers finish the three tasks with color or grayscale images. We find that observers’ agreements depend on the physical material properties of the objects such that translucent materials generate more inter-observer disagreements. Further, there are more disagreements among observers in the grayscale condition in comparison to that in color condition. We also discover that converting images to grayscale substantially affects the distributions of attribute ratings for some images. Furthermore, ratings of see-throughness, glossiness, and glow could predict individual observers’ binary classification of images in both grayscale and color conditions. Lastly, converting images to grayscale alters the perceived material categories for some images such that observers tend to misjudge images of food as non-food and vice versa. Our result demonstrates color is informative about material property estimation and recognition. Meanwhile, our analysis shows mid-level semantic estimation of material attributes might be closely related to high-level material recognition. We also discuss individual differences in our results and highlight the importance of such consideration in material perception.