Is Multimodal Speculative Decoding Ready for Diffusion-Based Parallel Drafting? A Survey and Empirical Diagnosis
Preprint
- 25 August 2026
- preprint
- Published by MDPI AG in Preprints.org
Abstract
Speculative decoding accelerates autoregressive generation by allowing a lightweight drafter to propose future tokens while a target model verifies them in parallel. Its lossless guarantee has motivated a line of work that pushes the drafter itself into parallel generation—the most recent enorous method is block-parallel generative approaches, including diffusion-based drafting methods such as DFlash and DSpark, achieving a 3.6x speedup on common daily chatting tasks. While this transition is well studied in text-only LLMs, its applicability to multimodal models remains an open question. Existing multimodal speculative decoding efforts focus on input compression, adapter alignment, candidate coverage, or modality-specific verification; however, block-parallel generative drafting remains largely unexplored. To bridge this gap, this paper combines a modality-centered survey with a cross-architecture empirical study to ask: Is multimodal speculative decoding ready for diffusion-based parallel drafting? In this survey, we systematically analyze a wide spectrum of multimodal models—spanning Vision-Language, Video-Language, Audio, and Vision-Language-Action (VLA) architectures—from the dual perspectives of drafting parallelism and cross-modal information interaction. We introduce a unified taxonomy that isolates drafter-side parallelism from orthogonal design choices such as tree construction and verification strategies. Furthermore, we provide a comprehensive empirical comparison of existing methods under varying degrees of parallelism across standardized multimodal benchmarks, including OCR, VQA, visual reasoning, and image captioning. Finally, we summarize the limitations of current approaches, discuss open challenges, and outline promising future directions for this rapidly evolving field.Keywords
All Related Versions
This publication has 46 references indexed in Scilit:
- See the Forest for the Trees: Loosely Speculative Decoding via Visual-Semantic Guidance for Efficient Inference of Video LLMs2026
- Continuous Speculative Decoding for Autoregressive Image GenerationarXiv, 2025
- Simultaneous Masked and Unmasked Decoding with Speculative Decoding Masking for Fast ASR without Accuracy Loss2025
- Fast and High-Quality Auto-Regressive Speech Synthesis via Speculative DecodingarXiv, 2025
- Sparse-to-Dense: A Free Lunch for Lossless Acceleration of Video Understanding in LLMs2025
- Better & Faster Large Language Models via Multi-token PredictionarXiv, 2024
- SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification2024
- EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees2024
- Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding2024
- Accelerating Large Language Model Decoding with Speculative SamplingarXiv, 2023