SAShuntaro C. Aoki

Shuntaro C. Aoki

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Shuntaro C. Aoki has a total of 30 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Cross-cue reconstruction of perceived 3D object structure from human visual cortex

    by Shuntaro C. Aoki, Reo Tsukasa, Shiyun Yang, Misato Tanaka, Eizaburo Doi, Tomohiro Nakamura, Jun-Kai Ho, Yukiyasu Kamitani

    The human brain assembles three-dimensional (3D) percepts from qualitatively different depth cues, yet the perceived 3D structure that the brain builds—a representation shared across cues—has remained difficult to measure directly. Here, we show that this cue-invariant 3D structure can be externalized as explicit 3D objects from human brain activity: fMRI responses are decoded into the latent features of a pretrained 3D point-cloud autoencoder, and a generator then maps these features back to a point cloud. A decoder trained exclusively on responses to 2D rendered objects passed three increasingly stringent tests: (i) it generalized to novel object categories; (ii) it generalized across depth cues to random dot stereograms (RDSs), which evoke 3D percepts through binocular disparity but share no pictorial shape information with the training images; and (iii) it tracked the 3D slant of contour-matched RDSs whose 2D outlines were held identical but whose disparity-defined slants varied, indicating that the reconstruction reflected depth-defined geometry rather than object category or image outline. Cross-cue generalization was strongest in higher visual areas, particularly along the dorsal stream. These results indicate that cross-cue generalization can serve as a criterion for externalizing perceived 3D structure and open a route toward reading out internal 3D representations that go beyond the momentary retinal input and could support predictions of how the world would appear under different viewpoints—a step toward externalizing the brain’s internal world model.

  • Open Access

    Inter-individual and inter-site neural code conversion without shared stimuli

    by Haibao Wang, Jun Kai Ho, Fan L. Cheng, Shuntaro C. Aoki, Yusuke Muraki, Misato Tanaka, Jong-Yun Park, Yukiyasu Kamitani

    Inter-individual variability in fine-grained functional topographies poses challenges for scalable data analysis and modeling. Functional alignment techniques can help mitigate these individual differences but they typically require paired brain data with the same stimuli between individuals, which are often unavailable. Here we present a neural code conversion method that overcomes this constraint by optimizing conversion parameters based on the discrepancy between the stimulus contents represented by original and converted brain activity patterns. This approach, combined with hierarchical features of deep neural networks as latent content representations, achieves conversion accuracies that are comparable with methods using shared stimuli. The converted brain activity from a source subject can be accurately decoded using the target’s pre-trained decoders, producing high-quality visual image reconstructions that rival within-individual decoding, even with data across different sites and limited training samples. Our approach offers a promising framework for scalable neural data analysis and modeling and a foundation for brain-to-brain communication.


  • Top Cited Papers
    Open Access

    Spurious reconstruction from brain activity

    by Ken Shirakawa, Yoshihiro Nagano, Misato Tanaka, Shuntaro C. Aoki, Yusuke Muraki, Kei Majima, Yukiyasu Kamitani


  • Open Access

    Reconstructing visual illusory experiences from human brain activity

    by Fan L. Cheng, Tomoyasu Horikawa, Kei Majima, Misato Tanaka, Mohamed Abdelhack, Shuntaro C. Aoki, Jin Hirano, Yukiyasu Kamitani

    Visual illusions provide valuable insights into the brain’s interpretation of the world given sensory inputs. However, the precise manner in which brain activity translates into illusory experiences remains largely unknown. Here, we leverage a brain decoding technique combined with deep neural network (DNN) representations to reconstruct illusory percepts as images from brain activity. The reconstruction model was trained on natural images to establish a link between brain activity and perceptual features and then tested on two types of illusions: illusory lines and neon color spreading. Reconstructions revealed lines and colors consistent with illusory experiences, which varied across the source visual cortical areas. This framework offers a way to materialize subjective experiences, shedding light on the brain’s internal representations of the world.


  • Open Access

    Reconstructing visual illusory experiences from human brain activity

    by Fan Cheng, Tomoyasu Horikawa, Kei Majima, Misato Tanaka, Mohamed Abdelhack, Shuntaro C. Aoki, Jin Hirano, Yukiyasu Kamitani

    Visual illusions provide significant insights into the brain’s interpretation of the world given sensory inputs. However, the precise manner in which brain activity translates into illusory experiences remains largely unknown. Here we leverage a brain decoding technique combined with deep neural network (DNN) representations to reconstruct illusory percepts as images from brain activity. The reconstruction model was trained on natural images to establish a link between brain activity and perceptual features and then tested on two types of illusions: illusory lines and neon color spreading. Reconstructions revealed lines and colors consistent with illusory experiences, which varied across the source visual cortical areas. This framework offers a way to materialize subjective experiences, shedding new light on the brain’s internal representations of the world.

  • Open Access

    Brain hierarchy score: Which deep neural networks are hierarchically brain-like?

    by Soma Nonaka, Kei Majima, Shuntaro C. Aoki, Yukiyasu Kamitani


  • Open Access

    Brain hierarchy score: Which deep neural networks are hierarchically brain-like?

    by Soma Nonaka, Kei Majima, Shuntaro C. Aoki, Yukiyasu Kamitani

    Summary: Achievement of human-level image recognition by deep neural networks (DNNs) has spurred interest in whether and how DNNs are brain-like. Both DNNs and the visual cortex perform hierarchical processing, and correspondence has been shown between hierarchical visual areas and DNN layers in representing visual features. Here, we propose the brain hierarchy (BH) score as a metric to quantify the degree of hierarchical correspondence based on the decoding of individual DNN unit activations from human brain activity. We find that BH scores for 29 pretrained DNNs with varying architectures are negatively correlated with image recognition performance, indicating that recently developed high-performance DNNs are not necessarily brain-like. Experimental manipulations of DNN models suggest that relatively simple feedforward architecture with broad spatial integration is critical to brain-like hierarchy. Our method provides new ways for designing DNNs and understanding the brain in consideration of their representational homology.


  • Open Access

    Brain Hierarchy Score: Which Deep Neural Networks are Hierarchically Brain-Like?

    by Soma Nonaka, Kei Majima, Shuntaro C. Aoki, Yukiyasu Kamitani

    Achievement of human-level image recognition by deep neural networks (DNNs) has spurred interest in whether and how DNNs are brain-like. Both DNNs and the visual cortex perform hierarchical processing, and correspondence has been shown between hierarchical visual areas and DNN layers in representing visual features. Here, we propose the brain hierarchy (BH) score as a metric to quantify the degree of hierarchical correspondence based on the decoding of individual DNN unit activations from human brain activity. We find that BH scores for 29 pre-trained DNNs with varying architectures are negatively correlated with image recognition performance, indicating that recently developed high-performance DNNs are not necessarily brain-like. Experimental manipulations of DNN models suggest that relatively simple feedforward architecture with broad spatial integration is critical to brain-like hierarchy. Our method provides new ways for designing DNNs and understanding the brain in consideration of their representational homology.


  • Open Access

    Characterization of deep neural network features by decodability from human brain activity

    by Tomoyasu Horikawa, Shuntaro C. Aoki, Mitsuaki Tsukamoto, Yukiyasu Kamitani

    Achievements of near human-level performance in object recognition by deep neural networks (DNNs) have triggered a flood of comparative studies between the brain and DNNs. Using a DNN as a proxy for hierarchical visual representations, our recent study found that human brain activity patterns measured by functional magnetic resonance imaging (fMRI) can be decoded (translated) into DNN feature values given the same inputs. However, not all DNN features are equally decoded, indicating a gap between the DNN and human vision. Here, we present a dataset derived from DNN feature decoding analyses, which includes fMRI signals of five human subjects during image viewing, decoded feature values of DNNs (AlexNet and VGG19), and decoding accuracies of individual DNN features with their rankings. The decoding accuracies of individual features were highly correlated between subjects, suggesting the systematic differences between the brain and DNNs. We hope the present dataset will contribute to revealing the gap between the brain and DNNs and provide an opportunity to make use of the decoded features for further applications.


  • Open Access

    Microstructural properties of the vertical occipital fasciculus explain the variability in human stereoacuity

    by Hiroki Oishi, Hiromasa Takemura, Shuntaro C. Aoki, Ichiro Fujita, Kaoru Amano

    Stereopsis is a fundamental visual function that has been studied extensively. However, it is not clear why depth discrimination (stereoacuity) varies more significantly among people than other modalities. Previous studies have reported the involvement of both dorsal and ventral visual areas in stereopsis, implying that not only neural computations in cortical areas but also the anatomical properties of white matter tracts connecting those areas can impact stereopsis. Here, we studied how human stereoacuity relates to white matter properties by combining psychophysics, diffusion MRI (dMRI), and quantitative MRI (qMRI). We performed a psychophysical experiment to measure stereoacuity and, in the same participants, we analyzed the microstructural properties of visual white matter tracts on the basis of two independent measurements, dMRI (fractional anisotropy, FA) and qMRI (macromolecular tissue volume; MTV). Microstructural properties along the right vertical occipital fasciculus (VOF), a major tract connecting dorsal and ventral visual areas, were highly correlated with measures of stereoacuity. This result was consistent for both FA and MTV, suggesting that the behavioral–structural relationship reflects differences in neural tissue density, rather than differences in the morphological configuration of fibers. fMRI confirmed that binocular disparity stimuli activated the dorsal and ventral visual regions near VOF endpoints. No other occipital tracts explained the variance in stereoacuity. In addition, the VOF properties were not associated with differences in performance on a different psychophysical task (contrast detection). These series of experiments suggest that stereoscopic depth discrimination performance is, at least in part, constrained by dorso-ventral communication through the VOF.