Cross-Subject EEG Feedback for Implicit Image Generation
Open Access
- 18 June 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 54 (10) , 6105-6117
Abstract
Generative models are powerful tools for producing novel information by learning from example data. However, the current approaches require explicit manual input to steer generative models to match human goals. Furthermore, how these models would integrate implicit, diverse feedback and goals of multiple users remains largely unexplored. Here, we present a first-of-its-kind system that produces novel images of faces by inferring human goals directly from cross-subject brain signals while study subjects are looking at example images. We report on an experiment where brain responses to images of faces were recorded using electroencephalography in 30 subjects, focusing on specific salient visual features (VFs). Preferences toward VFs were decoded from subjects’ brain responses and used as implicit feedback for a generative adversarial network (GAN), which generated new images of faces. The results from a follow-up user study evaluating the presence of the target salient VFs show that the images generated from brain feedback represent the goal of the study subjects and are comparable to images generated with manual feedback. The methodology provides a stepping stone toward humans-in-the-loop image generation.Keywords
Funding Information
- Research Council of Finland (322653, 328875, 336085, 350323, 352915)
- Horizon 2020 FET program of the European Union (CHIST-ERA-20-BCI-001)
- Alfred Kordelinin Säätiö (230099)
- Tekniikan Edistämissäätiö (10168)
This publication has 64 references indexed in Scilit:
- Deep learning modelling techniques: current progress, applications, advantages, and challengesArtificial Intelligence Review, 2023
- Neuroadaptive modelling for generating images matching perceptual categoriesScientific Reports, 2020
- Ethical Issues Posed by Field Research Using Highly Portable and Cloud-Enabled NeuroimagingNeuron, 2020
- Image Generation from Sketch Constraint Using Contextual GAN2018
- Neuroadaptive technology enables implicit cursor control based on medial prefrontal cortex activityProceedings of the National Academy of Sciences, 2016
- P300 speller BCI with a mobile EEG system: comparison to a traditional amplifierJournal of Neural Engineering, 2014
- Single-trial analysis and classification of ERP components — A tutorialNeuroImage, 2011
- Updating P300: An integrative theory of P3a and P3bClinical Neurophysiology, 2007
- Attention web designers: You have 50 milliseconds to make a good first impression!Behaviour & Information Technology, 2006
- Cross-modal visual–auditory–somatosensory integration in a multimodal object recognition task in humansInternational Congress Series, 2005