CookAnything: A Framework for Flexible and Consistent Multi-Step Recipe Image Generation
Authors:
- Ruoxuan ZhangRZRuoxuan Zhang
- Bin WenBWBin Wen
- Hongxia XieHXHongxia Xie
- Yi YaoYYYi Yao
- Songhan ZuoSZSonghan Zuo
- Jian-Yu Jiang-LinJJJian-Yu Jiang-Lin
- Hong-Han ShuaiHSHong-Han Shuai
- Wen-Huang ChengWCWen-Huang Cheng
Preprint
Abstract
Cooking is a sequential and visually grounded activity, where each step such as chopping, mixing, or frying carries both procedural logic and visual semantics. While recent diffusion models have shown strong capabilities in text-to-image generation, they struggle to handle structured multi-step scenarios like recipe illustration. Additionally, current recipe illustration methods are unable to adjust to the natural variability in recipe length, generating a fixed number of images regardless of the actual instructions structure. To address these limitations, we present CookAnything, a flexible and consistent diffusion-based framework that generates coherent, semantically distinct image sequences from textual cooking instructions of arbitrary length. The framework introduces three key components: (1) Step-wise Regional Control (SRC), which aligns textual steps with corresponding image regions within a single denoising process; (2) Flexible RoPE, a step-aware positional encoding mechanism that enhances both temporal coherence and spatial diversity; and (3) Cross-Step Consistency Control (CSCC), which maintains fine-grained ingredient consistency across steps. Experimental results on recipe illustration benchmarks show that CookAnything performs better than existing methods in training-based and training-free settings. The proposed framework supports scalable, high-quality visual synthesis of complex multi-step instructions and holds significant potential for broad applications in instructional media, and procedural content creation.Keywords
All Related Versions
- Version 1, 2025-12-03, ArXiv
- Version 2, 2025-12-05, ArXiv
- Published version: , 7854.
This publication has 12 references indexed in Scilit:
- FoodLMM: A Versatile Food Assistant Using Large Multi-Modal ModelIEEE Transactions on Multimedia, 2025
- Learning multi-scale features automatically from food and ingredientsMultimedia Systems, 2025
- FoodSky: A food-oriented large language model that can pass the chef and dietetic examinationsPatterns, 2025
- Recognize after early fusion: the Chinese food recognition based on the alignment of image and ingredientsMultimedia Systems, 2024
- FoodSAM: Any Food SegmentationIEEE Transactions on Multimedia, 2023
- ML-CookGAN: Multi-Label Generative Adversarial Network for Food Image GenerationACM Transactions on Multimedia Computing, Communications, and Applications, 2023
- Large Scale Visual Food RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuation2022
- Multi-Scale Multi-View Deep Feature Aggregation for Food RecognitionIEEE Transactions on Image Processing, 2019
- Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food ImagesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2019