Computer Vision System for Expressing Texture Using Sound-Symbolic Words
Authors:
Open Access
- 7 October 2021
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Psychology
- Vol. 12
Abstract
The major goals of texture research in computer vision are to understand, model, and process texture and ultimately simulate human visual information processing using computer technologies. The field of computer vision has witnessed remarkable advancements in material recognition using deep convolutional neural networks (DCNNs), which have enabled various computer vision applications, such as self-driving cars, facial and gesture recognition, and automatic number plate recognition. However, for computer vision to “express” texture like human beings is still difficult because texture description has no correct or incorrect answer and is ambiguous. In this paper, we develop a computer vision method using DCNN that expresses texture of materials. To achieve this goal, we focus on Japanese “sound-symbolic” words, which can describe differences in texture sensation at a fine resolution and are known to have strong and systematic sensory-sound associations. Because the phonemes of Japanese sound-symbolic words characterize categories of texture sensations, we develop a computer vision method to generate the phonemes and structure comprising sound-symbolic words that probabilistically correspond to the input images. It was confirmed that the sound-symbolic words output by our system had about 80% accuracy rate in our evaluation.Keywords
This publication has 42 references indexed in Scilit:
- Recognizing Materials Using Perceptually Inspired FeaturesInternational Journal of Computer Vision, 2013
- Tactual perception of material propertiesVision Research, 2010
- Haptic perception: A tutorialAttention, Perception, & Psychophysics, 2009
- Texture from touchScholarpedia, 2009
- Vision and touch: Independent or integrated systems for the perception of texture?Brain Research, 2008
- The Case for Sound SymbolismAnnual Review of Anthropology, 1999
- Reflectance and texture of real-world surfacesACM Transactions on Graphics, 1999
- Perception of texture by vision and touch: Multidimensionality and intersensory integration.Journal of Experimental Psychology: Human Perception and Performance, 1986
- Perception of texture by vision and touch: Multidimensionality and intersensory integration.Journal of Experimental Psychology: Human Perception and Performance, 1986
- A study in phonetic symbolism.Journal of Experimental Psychology, 1929