CookGAN: Meal Image Synthesis from Ingredients
Authors:
- 1 March 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1439-1447
Abstract
In this work we propose a new computational framework, based on generative deep models, for synthesis of photo-realistic food meal images from textual list of its ingredients. Previous works on synthesis of images from text typically rely on pre-trained text models to extract text features, followed by generative neural networks (GAN) aimed to generate realistic images conditioned on the text features. These works mainly focus on generating spatially compact and well-defined categories of objects, such as birds or flowers, but meal images are significantly more complex, consisting of multiple ingredients whose appearance and spatial qualities are further modified by cooking methods. To generate real-like meal images from ingredients, we propose Cook Generative Adversarial Networks (CookGAN), CookGAN first builds an attention-based ingredients-image association model, which is then used to condition a generative neural network tasked with synthesizing meal images. Furthermore, a cycle-consistent constraint is added to further improve image quality and control appearance. Experiments show our model is able to generate meal images corresponding to the ingredients.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Cross-Modal Retrieval in the Cooking Context2018
- Grab, Pay, and Eat: Semantic Food Detection for Smart RestaurantsIEEE Transactions on Multimedia, 2018
- Wide-Slice Residual Networks for Food Recognition2018
- StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks2017
- Learning Cross-Modal Embeddings for Cooking Recipes and Food Images2017
- Deep Residual Learning for Image Recognition2016
- Health, Food and User's Profile Ontologies for Personalized Information RetrievalProcedia Computer Science, 2015
- Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation2014
- Recipe recommendation using ingredient networks2012
- Flavor network and the principles of food pairingScientific Reports, 2011