DemoCaricature: Democratising Caricature Generation with a Rough Sketch
Authors:
- Dar-Yen ChenDCDar-Yen Chen
- Subhadeep KoleySKSubhadeep Koley
- Aneeshan SainASAneeshan Sain
- Pinaki Nath ChowdhuryPCPinaki Nath Chowdhury
- Yi-Zhe SongYSYi-Zhe Song
Preprint
Abstract
In this paper, we democratise caricature generation, empowering individuals to effortlessly craft personalised caricatures with just a photo and a conceptual sketch. Our objective is to strike a delicate balance between abstraction and identity, while preserving the creativity and subjectivity inherent in a sketch. To achieve this, we present Explicit Rank-1 Model Editing alongside single-image personalisation, selectively applying nuanced edits to cross-attention layers for a seamless merge of identity and style. Additionally, we propose Random Mask Reconstruction to enhance robustness, directing the model to focus on distinctive identity and style features. Crucially, our aim is not to replace artists but to eliminate accessibility barriers, allowing enthusiasts to engage in the artistry.Keywords
All Related Versions
This publication has 34 references indexed in Scilit:
- FastComposer: Tuning-Free Multi-subject Image Generation with Localized AttentionInternational Journal of Computer Vision, 2024
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches2022
- Masked Autoencoders Are Scalable Vision Learners2022
- HairCLIP: Design Your Hair by Text and Reference Image2022
- Learning to Caricature via Semantic Shape TransformInternational Journal of Computer Vision, 2021
- Inverting Generative Adversarial Renderer for Face Reconstruction2021
- Analyzing and Improving the Image Quality of StyleGAN2020
- WarpGAN: Automatic Caricature Generation2019
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015