Abstract
Monocular depth estimation within the denoising diffusion paradigm demonstrates impressive generalization ability but suffers from low inference speed. Recent methods adopt a single-step deterministic paradigm to improve inference efficiency. However, uncritically applying the generative features from diffusion models to the perceptual depth estimation task leads to suboptimal results, due to the differences between image generation and dense prediction. In this work, we propose Depth-Master, a single-step diffusion model designed to tame diffusion models for enhanced performance and visual quality. First, to mitigate overfitting to texture details introduced by generative features, we propose a Feature Alignment module, which incorporates high-quality semantic features to enhance the denoising network’s representation capability. Second, to address the lack of fine-grained details in the single-step deterministic framework, we propose a Fourier Enhancement module to adaptively balance low-frequency structure and high-frequency details. We adopt a two-stage training strategy to fully leverage the potential of the two modules. In the first stage, we focus on learning the global scene structure with the Feature Alignment module, while in the second stage, we exploit the Fourier Enhancement module to improve the visual quality. Through these efforts, our model achieves state-of-the-art performance in terms of generalization and detail preservation, outperforming other diffusion-based methods across various datasets. Our project page can be found at https://indu1ge.github.io/DepthMaster page.
Funding Information
  • National Natural Science Foundation of China (62394354)
  • Youth Innovation Promotion Association of the Chinese Academy of Sciences