MultiPaint: A Unified Framework for Multi-Task, Multi-Object, and Multi-Condition Video Inpainting
- 1 July 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 32 (7) , 7031-7045
Abstract
Video inpainting modifies local regions in video while ensuring spatial and temporal coherence. However, existing methods-both traditional and recent diffusion-based ones-face key limitations: they lack unified support for both insertion and completion, and are restricted to single-object inpainting, making it difficult to handle multi-object scenarios involving grounding and interaction. In this article, we propose MultiPaint, a unified framework for multi-task, multi-object, and multi-condition video inpainting. First, we introduce dual-branch adapters to unify the insertion and completion tasks within a single model. Moreover, we propose a test-time scheduled feature composition strategy that enables multi-object inpainting with user-specified locations while better preserving interactions among objects, a setting that has been insufficiently addressed in prior work. Additionally, we introduce a multi-condition inpainting scheme that integrates text-guided, image-guided, and keyframe-guided modes via dynamic frame masking, providing more controllability in appearance customization. Extensive experiments show that MultiPaint achieves state-of-the-art performance on object insertion and scene completion among the recent works. We further demonstrate its versatility in downstream tasks including grounded video generation, object editing, object removal, image-guided inpainting, and long video inpainting.Keywords
Funding Information
- Innovation and Technology Fund
- Innovation and Technology Commission
- Hong Kong Special Administrative Region (ITS/269/24FP)
- Kuaishou Technology
This publication has 53 references indexed in Scilit:
- BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion2024
- Grounding DINO: Marrying DINO with Grounded Pre-training for Open-Set Object Detection2024
- A Task Is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting2024
- DynamiCrafter: Animating Open-Domain Images with Video Diffusion Priors2024
- Error Compensation Framework for Flow-Guided Video Inpainting2022
- Flow-edge Guided Video Completion2020
- Image Inpainting for Irregular Holes Using Partial Convolutions2018
- YouTube-VOS: Sequence-to-Sequence Video Object Segmentation2018
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric2018
- A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation2016