Ryusuke Hayashi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Ryusuke Hayashi has a total of 67 co-authors. Here are the top co-authors by number of shared publications:
- Takuya Igaue
The University of Tokyo
3 shared publications - Hidehiko Takahashi
Department of Psychiatry and Behavioral Sciences, Graduate School of Medical and Dental Sciences Institute of Science Tokyo Tokyo Japan
2 shared publications Makoto Obara
Philips Japan, 2-13-37 Kohnan, Minato-ku 108-8507, Tokyo, Japan
2 shared publications- Noriyuki Higo
Human Informatics and Interaction Research Institute, National Institute of Advanced Industrial Science and Technology, 1-1-1 Higashi, Tsukuba-City, Ibaraki 305-8568, Japan
2 shared publications - Satoshi Nishida
National Institute of Information and Communications Technology
2 shared publications - Tomokazu Tsurugizawa
Human Informatics and Interaction Research Institute National Institute of Advanced Industrial Science and Technology Tsukuba Japan
2 shared publications
Latest Publications
- Open Access
- 1 January 2026
- Published byĀ Institute of Electrical and Electronics Engineers (IEEE)Ā inĀ IEEE Access
- p. 1
Motion Transfer-Enhanced StyleGAN for Generating Diverse Macaque Facial Expressions
- 1 January 2026
- Published byĀ Institute of Electrical and Electronics Engineers (IEEE)Ā inĀ IEEE Access
- p. 1
Open AccessMotion Transfer-Enhanced StyleGAN for Generating Diverse Macaque Facial Expressions
- Open Access
Motion Transfer-Enhanced StyleGAN for Generating Diverse Macaque Facial Expressions
Generating animal faces using generative AI techniques is challenging because the available training images are limited both in quantity and variation, particularly for facial expressions across individuals. In this study, we focus on macaque monkeys, widely studied in systems neuroscience and evolutionary research, and propose a method to generate their facial expressions using a style-based generative image model (i.e., StyleGAN2). To address data limitations, we implemented: 1) data augmentation by synthesizing new facial expression images using a motion transfer to animate still images with computer graphics, 2) sample selection based on the latent representation of macaque faces from an initially trained StyleGAN2 model to ensure the variation and uniform sampling in training dataset, and 3) loss function refinement to ensure the accurate reproduction of subtle movements, such as eye movements. Our results demonstrate that the proposed method enables the generation of diverse facial expressions for multiple macaque individuals, outperforming models trained solely on original still images. Additionally, we show that our model is effective for style-based image editing, where specific style parameters correspond to distinct facial movements. These findings underscore the model's potential for disentangling motion components as style parameters, providing a valuable tool for research on macaque facial expressions.
Open AccessMotion Transfer-Enhanced StyleGAN for Generating Diverse Macaque Facial Expressions
Generating animal faces using generative AI techniques is challenging because the available training images are limited both in quantity and variation, particularly for facial expressions across individuals. In this study, we focus on macaque monkeys, widely studied in systems neuroscience and evolutionary research, and propose a method to generate their facial expressions using a style-based generative image model (i.e., StyleGAN2). To address data limitations, we implemented: 1) data augmentation by synthesizing new facial expression images using a motion transfer to animate still images with computer graphics, 2) sample selection based on the latent representation of macaque faces from an initially trained StyleGAN2 model to ensure the variation and uniform sampling in training dataset, and 3) loss function refinement to ensure the accurate reproduction of subtle movements, such as eye movements. Our results demonstrate that the proposed method enables the generation of diverse facial expressions for multiple macaque individuals, outperforming models trained solely on original still images. Additionally, we show that our model is effective for style-based image editing, where specific style parameters correspond to distinct facial movements. These findings underscore the model's potential for disentangling motion components as style parameters, providing a valuable tool for research on macaque facial expressions.
- Open Access
Monkey Perceptogram: Reconstructing Visual Representation and Presumptive Neural Preference from Monkey Multi-electrode Arrays
Understanding how the primate brain transforms complex visual scenes into coherent perceptual experiences remains a central challenge in neuroscience. Here, we present a comprehensive framework for interpreting monkey visual processing by integrating encoding and decoding approaches applied to two large-scale spiking datasets recorded from macaque using THINGS images (THINGS macaque IT Dataset (TITD) and THINGS Ventral Stream Spiking Dataset (TVSD)). We leverage multi-electrode array recordings from the ventral visual stream--including V1, V4, and inferotemporal (IT) cortex--to investigate how distributed neural populations encode and represent visual information. Our approach employs linear models to decode spiking activity into multiple latent visual spaces (including CLIP and VDVAE embeddings) and reconstruct images using state-of-the-art generative models. We further utilize encoding models to map visual features back to neural activity, enabling visualization of the "preferred stimuli" that drive specific neural ensembles. Analyses of both datasets reveal that it is possible to reconstruct both low-level (e.g., color, texture) and high-level (e.g., semantic category) features of visual stimuli from population activity, with reconstructions preserving key perceptual attributes as quantified by feature-based similarity metrics. The spatiotemporal spike patterns reflect the ventral stream's hierarchical organization with anterior regions representing complex objects and categories. Functional clustering identifies feature-specific neural ensembles, with temporal dynamics show evolving feature selectivity post-stimulus. Our findings demonstrate feasible, generalizable perceptual reconstruction from large-scale monkey neural recordings, linking neural activity to perception.
Open AccessMonkey Perceptogram: Reconstructing Visual Representation and Presumptive Neural Preference from Monkey Multi-electrode Arrays
Understanding how the primate brain transforms complex visual scenes into coherent perceptual experiences remains a central challenge in neuroscience. Here, we present a comprehensive framework for interpreting monkey visual processing by integrating encoding and decoding approaches applied to two large-scale spiking datasets recorded from macaque using THINGS images (THINGS macaque IT Dataset (TITD) and THINGS Ventral Stream Spiking Dataset (TVSD)). We leverage multi-electrode array recordings from the ventral visual stream--including V1, V4, and inferotemporal (IT) cortex--to investigate how distributed neural populations encode and represent visual information. Our approach employs linear models to decode spiking activity into multiple latent visual spaces (including CLIP and VDVAE embeddings) and reconstruct images using state-of-the-art generative models. We further utilize encoding models to map visual features back to neural activity, enabling visualization of the "preferred stimuli" that drive specific neural ensembles. Analyses of both datasets reveal that it is possible to reconstruct both low-level (e.g., color, texture) and high-level (e.g., semantic category) features of visual stimuli from population activity, with reconstructions preserving key perceptual attributes as quantified by feature-based similarity metrics. The spatiotemporal spike patterns reflect the ventral stream's hierarchical organization with anterior regions representing complex objects and categories. Functional clustering identifies feature-specific neural ensembles, with temporal dynamics show evolving feature selectivity post-stimulus. Our findings demonstrate feasible, generalizable perceptual reconstruction from large-scale monkey neural recordings, linking neural activity to perception.
- Open Access
- 28 November 2024
- Published byĀ Frontiers Media SAĀ inĀ Frontiers in Neuroinformatics
- Vol.Ā 18, 1470845
Unsupervised method for representation transfer from one brain to another
Although the anatomical arrangement of brain regions and the functional structures within them are similar across individuals, the representation of neural information, such as recorded brain activity, varies among individuals owing to various factors. Therefore, appropriate conversion and translation of brain information is essential when decoding neural information using a model trained using another personās data or to achieving brain-to-brain communication. We propose a brain representation transfer method that involves transforming a data representation obtained from one personās brain into that obtained from another personās brain, without relying on corresponding label information between the transferred datasets. We defined the requirements to enable such brain representation transfer and developed an algorithm that distills the assumption of common similarity structure across the brain datasets into a rotational and reflectional transformation across low-dimensional hyperspheres using encoders for non-linear dimensional reduction. We first validated our proposed method using data from artificial neural networks as substitute neural activity and examining various experimental factors. We then evaluated the applicability of our method to real brain activity using functional magnetic resonance imaging response data acquired from human participants. The results of these validation experiments showed that our method successfully performed representation transfer and achieved transformations in some cases that were similar to those obtained when using corresponding label information. Additionally, we reconstructed images from individualsā data without training personalized decoders by performing brain representation transfer. The results suggest that our unsupervised transfer method is useful for the reapplication of existing models personalized to specific participants and datasets to decode brain information from other individuals. Our findings also serve as a proof of concept for the methodology, enabling the exchange of the latent properties of neural information representing individualsā sensations.
- 28 November 2024
- Published byĀ Frontiers Media SAĀ inĀ Frontiers in Neuroinformatics
- Vol.Ā 18, 1470845
Open AccessUnsupervised method for representation transfer from one brain to another
Although the anatomical arrangement of brain regions and the functional structures within them are similar across individuals, the representation of neural information, such as recorded brain activity, varies among individuals owing to various factors. Therefore, appropriate conversion and translation of brain information is essential when decoding neural information using a model trained using another personās data or to achieving brain-to-brain communication. We propose a brain representation transfer method that involves transforming a data representation obtained from one personās brain into that obtained from another personās brain, without relying on corresponding label information between the transferred datasets. We defined the requirements to enable such brain representation transfer and developed an algorithm that distills the assumption of common similarity structure across the brain datasets into a rotational and reflectional transformation across low-dimensional hyperspheres using encoders for non-linear dimensional reduction. We first validated our proposed method using data from artificial neural networks as substitute neural activity and examining various experimental factors. We then evaluated the applicability of our method to real brain activity using functional magnetic resonance imaging response data acquired from human participants. The results of these validation experiments showed that our method successfully performed representation transfer and achieved transformations in some cases that were similar to those obtained when using corresponding label information. Additionally, we reconstructed images from individualsā data without training personalized decoders by performing brain representation transfer. The results suggest that our unsupervised transfer method is useful for the reapplication of existing models personalized to specific participants and datasets to decode brain information from other individuals. Our findings also serve as a proof of concept for the methodology, enabling the exchange of the latent properties of neural information representing individualsā sensations.
- Open Access
- 23 October 2024
- Published byĀ ElsevierĀ inĀ NeuroImage
- Vol.Ā 302, 120901
Multi-scale hierarchical brain regions detect individual and interspecies variations of structural connectivity in macaque monkeys and humans
- 23 October 2024
- Published byĀ ElsevierĀ inĀ NeuroImage
- Vol.Ā 302, 120901
Open AccessMulti-scale hierarchical brain regions detect individual and interspecies variations of structural connectivity in macaque monkeys and humans
- Open Access
- 9 September 2024
- Published byĀ Springer NatureĀ inĀ Scientific Reports
- Vol.Ā 14 Ā (1) , 1-13
Commonalities and variations in emotion representation across modalities and brain regions
Humans express emotions through various modalities such as facial expressions and natural language. However, the relationships between emotions expressed through different modalities and their correlations with neural activities remain uncertain. Here, we aimed to unveil some of these uncertainties by investigating the similarity of emotion representations across modalities and brain regions. First, we represented various emotion categories as multi-dimensional vectors derived from visual (face), linguistic, and visio-linguistic data, and used representational similarity analysis to compare these modalities. Second, we examined the linear transferability of emotion representation from other modalities to the visual modality. Third, we compared the representational structure derived in the first step with those from brain activities across 360 regions. Our findings revealed that emotion representations share commonalities across modalities with modality-type dependent variations, and they can be linearly mapped from other modalities to the visual modality. Additionally, emotion representations in uni-modalities showed relatively higher similarity with specific brain regions, while multi-modal emotion representation was most similar to representations across the entire brain region. These findings suggest that emotional experiences are represented differently across various brain regions with varying degrees of similarity to different modality types, and that they may be multi-modally conveyable in visual and linguistic domains.
- 9 September 2024
- Published byĀ Springer NatureĀ inĀ Scientific Reports
- Vol.Ā 14 Ā (1) , 1-13
Open AccessCommonalities and variations in emotion representation across modalities and brain regions
Humans express emotions through various modalities such as facial expressions and natural language. However, the relationships between emotions expressed through different modalities and their correlations with neural activities remain uncertain. Here, we aimed to unveil some of these uncertainties by investigating the similarity of emotion representations across modalities and brain regions. First, we represented various emotion categories as multi-dimensional vectors derived from visual (face), linguistic, and visio-linguistic data, and used representational similarity analysis to compare these modalities. Second, we examined the linear transferability of emotion representation from other modalities to the visual modality. Third, we compared the representational structure derived in the first step with those from brain activities across 360 regions. Our findings revealed that emotion representations share commonalities across modalities with modality-type dependent variations, and they can be linearly mapped from other modalities to the visual modality. Additionally, emotion representations in uni-modalities showed relatively higher similarity with specific brain regions, while multi-modal emotion representation was most similar to representations across the entire brain region. These findings suggest that emotional experiences are represented differently across various brain regions with varying degrees of similarity to different modality types, and that they may be multi-modally conveyable in visual and linguistic domains.
- Open Access
- 28 August 2024
- Published byĀ WileyĀ inĀ Psychiatry and Clinical Neurosciences
- Vol.Ā 78 Ā (11) , 687-695
Homogenization of word relationships in schizophrenia: Topological analysis of cortical semantic representations
Aim: Patients with schizophrenia typically exhibit symptoms of disorganized thought and display concreteness and overāinclusion in verbal reports, depending on the level of abstraction. While concreteness and overāinclusion may appear contradictory, the underlying psychopathology that explains these symptoms remains unclear. In the current study, we used functional magnetic resonance imaging with an encoding modeling approach to examine how concepts of various words, represented as brain activity, are anomalously connected at different levels of abstraction in patients with schizophrenia.Methods: Fourteen individuals diagnosed with schizophrenia and 17 healthy controls underwent functional magnetic resonance imaging to measure brain activity representing concepts of various words. We used a persistent homology (PH) method to analyze the topological structures of word representations in schizophrenia patients, healthy controls, and random data, across different levels of abstraction by varying dissimilarity scales in the representation space.Results: The results revealed that patients with schizophrenia exhibited more homogeneous word relationships across different levels of abstraction compared with healthy controls. Additionally, topological structures exhibited a shift toward a random network structure in patients with schizophrenia compared with controls. The PH method successfully distinguished semantic representations of patients with schizophrenia from those of controls.Conclusions: The current results provide an explanation for the mechanisms underlying the deficits in abstraction ability observed in schizophrenia. The isotopic connection of individual concepts reflects both the reduction of contextual connections at a semantically fineāgrained scale and the absence of clear boundaries between related concepts at a coarse scale, which lead to concreteness and overāinclusion, respectively.
- 28 August 2024
- Published byĀ WileyĀ inĀ Psychiatry and Clinical Neurosciences
- Vol.Ā 78 Ā (11) , 687-695
Open AccessHomogenization of word relationships in schizophrenia: Topological analysis of cortical semantic representations
Aim: Patients with schizophrenia typically exhibit symptoms of disorganized thought and display concreteness and overāinclusion in verbal reports, depending on the level of abstraction. While concreteness and overāinclusion may appear contradictory, the underlying psychopathology that explains these symptoms remains unclear. In the current study, we used functional magnetic resonance imaging with an encoding modeling approach to examine how concepts of various words, represented as brain activity, are anomalously connected at different levels of abstraction in patients with schizophrenia.Methods: Fourteen individuals diagnosed with schizophrenia and 17 healthy controls underwent functional magnetic resonance imaging to measure brain activity representing concepts of various words. We used a persistent homology (PH) method to analyze the topological structures of word representations in schizophrenia patients, healthy controls, and random data, across different levels of abstraction by varying dissimilarity scales in the representation space.Results: The results revealed that patients with schizophrenia exhibited more homogeneous word relationships across different levels of abstraction compared with healthy controls. Additionally, topological structures exhibited a shift toward a random network structure in patients with schizophrenia compared with controls. The PH method successfully distinguished semantic representations of patients with schizophrenia from those of controls.Conclusions: The current results provide an explanation for the mechanisms underlying the deficits in abstraction ability observed in schizophrenia. The isotopic connection of individual concepts reflects both the reduction of contextual connections at a semantically fineāgrained scale and the absence of clear boundaries between related concepts at a coarse scale, which lead to concreteness and overāinclusion, respectively.
- Open Access
- 19 March 2024
- Published byĀ Springer NatureĀ inĀ Research Square
Multi-scale hierarchical brain regions detect individual and inter-species variations of structural connectivity in macaque monkeys and humans
Macaques are representative animal models in translational research. However, the distinct shape and location of the brain regions between macaques and humans prevents us from comparing the brain structure directly. Here, we calculated structural connectivity (SC) with multi-scale hierarchical regions of interest (ROIs) to parcel out human and macaque brain into 4 (level 1 ROIs) or 13 (level 2 ROIs) regions, which consist of anatomically and functionally defined level 3 ROIs (around 100 parcellation of the brain) and similar function. The SC with the level 1 ROIs showed lower individual and inter-species variation in macaques and humans. SC with level 2 ROIs shows that the temporal lobe has high individual and inter-species variation in macaques and humans, while the prefrontal and sensory cortices show low individual and inter-species variation. These results provide insights into the suitability of macaques as animal models for humans.
- 19 March 2024
- Published byĀ Springer NatureĀ inĀ Research Square
Open AccessMulti-scale hierarchical brain regions detect individual and inter-species variations of structural connectivity in macaque monkeys and humans
Macaques are representative animal models in translational research. However, the distinct shape and location of the brain regions between macaques and humans prevents us from comparing the brain structure directly. Here, we calculated structural connectivity (SC) with multi-scale hierarchical regions of interest (ROIs) to parcel out human and macaque brain into 4 (level 1 ROIs) or 13 (level 2 ROIs) regions, which consist of anatomically and functionally defined level 3 ROIs (around 100 parcellation of the brain) and similar function. The SC with the level 1 ROIs showed lower individual and inter-species variation in macaques and humans. SC with level 2 ROIs shows that the temporal lobe has high individual and inter-species variation in macaques and humans, while the prefrontal and sensory cortices show low individual and inter-species variation. These results provide insights into the suitability of macaques as animal models for humans.
- Open Access
- 19 May 2023
- Published byĀ ElsevierĀ inĀ Computers in Human Behavior
- Vol.Ā 146
Signatures of the uncanny valley effect in an artificial neural network
- 19 May 2023
- Published byĀ ElsevierĀ inĀ Computers in Human Behavior
- Vol.Ā 146
Open AccessSignatures of the uncanny valley effect in an artificial neural network
- Open Access
- 21 December 2022
- Published byĀ Oxford University Press (OUP)Ā inĀ Schizophrenia Bulletin
- Vol.Ā 49 Ā (2) , 498-506
Disorganization of Semantic Brain Networks in Schizophrenia Revealed by fMRI
Schizophrenia is a mental illness that presents with thought disorders including delusions and disorganized speech. Thought disorders have been regarded as a consequence of the loosening of associations between semantic concepts since the term āschizophreniaā was first coined by Bleuler. However, a mechanistic account of this cardinal disturbance in terms of functional dysconnection has been lacking. To evaluate how aberrant semantic connections are expressed through brain activity, we characterized large-scale network structures of concept representations using functional magnetic resonance imaging (fMRI). We quantified various concept representations in patientsā brains from fMRI activity evoked by movie scenes using encoding modeling. We then constructed semantic brain networks by evaluating the similarity of these semantic representations and conducted graph theory-based network analyses. Neurotypical networks had small-world properties similar to those of natural languages, suggesting small-worldness as a universal property in semantic knowledge networks. Conversely, small-worldness was significantly reduced in networks of schizophrenia patients and was correlated with psychological measures of delusions. Patientsā semantic networks were partitioned into more distinct categories and had more random within-category structures than those of controls. The differences in conceptual representations manifest altered semantic clustering and associative intrusions that underlie thought disorders. This is the first study to provide pathophysiological evidence for the loosening of associations as reflected in randomization of semantic networks in schizophrenia. Our method provides a promising approach for understanding the neural basis of altered or creative inner experiences of individuals with mental illness or exceptional abilities, respectively.
- 21 December 2022
- Published byĀ Oxford University Press (OUP)Ā inĀ Schizophrenia Bulletin
- Vol.Ā 49 Ā (2) , 498-506
Open AccessDisorganization of Semantic Brain Networks in Schizophrenia Revealed by fMRI
Schizophrenia is a mental illness that presents with thought disorders including delusions and disorganized speech. Thought disorders have been regarded as a consequence of the loosening of associations between semantic concepts since the term āschizophreniaā was first coined by Bleuler. However, a mechanistic account of this cardinal disturbance in terms of functional dysconnection has been lacking. To evaluate how aberrant semantic connections are expressed through brain activity, we characterized large-scale network structures of concept representations using functional magnetic resonance imaging (fMRI). We quantified various concept representations in patientsā brains from fMRI activity evoked by movie scenes using encoding modeling. We then constructed semantic brain networks by evaluating the similarity of these semantic representations and conducted graph theory-based network analyses. Neurotypical networks had small-world properties similar to those of natural languages, suggesting small-worldness as a universal property in semantic knowledge networks. Conversely, small-worldness was significantly reduced in networks of schizophrenia patients and was correlated with psychological measures of delusions. Patientsā semantic networks were partitioned into more distinct categories and had more random within-category structures than those of controls. The differences in conceptual representations manifest altered semantic clustering and associative intrusions that underlie thought disorders. This is the first study to provide pathophysiological evidence for the loosening of associations as reflected in randomization of semantic networks in schizophrenia. Our method provides a promising approach for understanding the neural basis of altered or creative inner experiences of individuals with mental illness or exceptional abilities, respectively.