Generative Edge Detection with Stable Diffusion
Open Access
- 7 September 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 22 (9) , 1-22
Abstract
Edge detection is typically viewed as a pixel-level classification problem mainly addressed by discriminative methods. Recently, generative edge detection methods, especially those based on diffusion models, are initialized in the edge detection task. Despite great potential, the retraining of task-specific designed modules and multi-step denoising inference limits their broader applications. Upon closer investigation, we speculate that part of the reason is the under-exploration of the rich discriminative information encoded in extensively pre-trained large models (e.g., stable diffusion models). Thus motivated, we propose a novel approach, named Generative Edge Detector (GED), by fully utilizing the potential of the pre-trained stable diffusion model. Our model can be trained and inferred efficiently without a specific network design due to the rich high-level and low-level prior knowledge empowered by the pre-trained stable diffusion. Specifically, we propose to finetune the denoising U-Net and predict latent edge maps directly, by taking the latent image feature maps as input. Additionally, due to the subjectivity and ambiguity of the edges, we also incorporate the granularity of the edges into the denoising U-Net model as one of the conditions to achieve controllable and diverse predictions. Furthermore, we devise a granularity regularization to ensure the relative granularity relationship of the multiple predictions. We conduct extensive experiments on multiple datasets and achieve competitive performance (e.g., 0.870 and 0.880 in terms of ODS and OIS on the BSDS test dataset). We will release the source code of GED upon the publication of the work.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62271042)
- Beijing Natural Science Foundation (L259055)
This publication has 26 references indexed in Scilit:
- EdgeMaskFormer: Adapting Mask Transformer for Semantic Edge DetectionIEEE Transactions on Multimedia, 2025
- Implicit and Explicit Language Guidance for Diffusion-Based Visual PerceptionIEEE Transactions on Multimedia, 2024
- Cycle pixel difference network for crisp edge detectionNeurocomputing, 2024
- Edge Perception Camouflaged Object Detection Under Frequency Domain ReconstructionIEEE Transactions on Circuits and Systems for Video Technology, 2024
- Boundary-Guided Lightweight Semantic Segmentation With Multi-Scale Semantic ContextIEEE Transactions on Multimedia, 2024
- Contour-Aware Equipotential Learning for Semantic SegmentationIEEE Transactions on Multimedia, 2022
- FCL-Net: Towards accurate edge detection via Fine-scale Corrective LearningNeural Networks, 2022
- Learning to detect natural image boundaries using local brightness, color, and texture cuesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2004
- A Computational Approach to Edge DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 1986
- On the accuracy of the Sobel edge detectorImage and Vision Computing, 1983