Image Content Generation with Causal Reasoning
Open Access
- 24 March 2024
- journal article
- research article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 38 (12) , 13646-13654
Abstract
The emergence of ChatGPT has once again sparked research in generative artificial intelligence (GAI). While people have been amazed by the generated results, they have also noticed the reasoning potential reflected in the generated textual content. However, this current ability for causal reasoning is primarily limited to the domain of language generation, such as in models like GPT-3. In visual modality, there is currently no equivalent research. Considering causal reasoning in visual content generation is significant. This is because visual information contains infinite granularity. Particularly, images can provide more intuitive and specific demonstrations for certain reasoning tasks, especially when compared to coarse-grained text. Hence, we propose a new image generation task called visual question answering with image (VQAI) and establish a dataset of the same name based on the classic Tom and Jerry animated series. Additionally, we develop a new paradigm for image generation to tackle the challenges of this task. Finally, we perform extensive experiments and analyses, including visualizations of the generated content and discussions on the potentials and limitations. The code and data are publicly available under the license of CC BY-NC-SA 4.0 for academic and non-commercial usage at: https://github.com/IEIT-AGI/MIX-Shannon/blob/main/projects/VQAI/lgd_vqai.md.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Image Content Generation with Causal ReasoningProceedings of the AAAI Conference on Artificial Intelligence, 2024
- Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models2024
- Imagic: Text-Based Real Image Editing with Diffusion Models2023
- InstructPix2Pix: Learning to Follow Image Editing Instructions2023
- A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity2023
- Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data2023
- NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis2020
- With or without you: predictive coding and Bayesian inference in the brainCurrent Opinion in Neurobiology, 2017
- On the training aspects of Deep Neural Network (DNN) for parametric TTS synthesis2014