Generative Edge Detection with Stable Diffusion
Preprint
Abstract
Edge detection is typically viewed as a pixel-level classification problem mainly addressed by discriminative methods. Recently, generative edge detection methods, especially diffusion model based solutions, are initialized in the edge detection task. Despite great potential, the retraining of task-specific designed modules and multi-step denoising inference limits their broader applications. Upon closer investigation, we speculate that part of the reason is the under-exploration of the rich discriminative information encoded in extensively pre-trained large models (\eg, stable diffusion models). Thus motivated, we propose a novel approach, named Generative Edge Detector (GED), by fully utilizing the potential of the pre-trained stable diffusion model. Our model can be trained and inferred efficiently without specific network design due to the rich high-level and low-level prior knowledge empowered by the pre-trained stable diffusion. Specifically, we propose to finetune the denoising U-Net and predict latent edge maps directly, by taking the latent image feature maps as input. Additionally, due to the subjectivity and ambiguity of the edges, we also incorporate the granularity of the edges into the denoising U-Net model as one of the conditions to achieve controllable and diverse predictions. Furthermore, we devise a granularity regularization to ensure the relative granularity relationship of the multiple predictions. We conduct extensive experiments on multiple datasets and achieve competitive performance (\eg, 0.870 and 0.880 in terms of ODS and OIS on the BSDS test dataset).Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Diffusion Model is Secretly a Training-Free Open Vocabulary Semantic SegmenterIEEE Transactions on Image Processing, 2025
- A Simple Latent Diffusion Approach for Panoptic Segmentation and Mask Inpainting2024
- Text-Image Alignment for Diffusion-Based Perception2024
- Boosting Deep Unsupervised Edge Detection via Segment Anything ModelIEEE Transactions on Industrial Informatics, 2024
- Delving Into Crispness: Guided Label Refinement for Crisp Edge DetectionIEEE Transactions on Image Processing, 2023
- Learning Shape-Invariant Representation for Generalizable Semantic SegmentationIEEE Transactions on Image Processing, 2023
- Learning to Detect Semantic Boundaries with Image-Level Class LabelsInternational Journal of Computer Vision, 2022
- FCL-Net: Towards accurate edge detection via Fine-scale Corrective LearningNeural Networks, 2022
- Contour Detection and Hierarchical Image SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010
- On the accuracy of the Sobel edge detectorImage and Vision Computing, 1983