Depth–Normals Fusion and Photometric Stereo Errors Self-Correction Using Joint Optimization
- 29 May 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 33 (10709908) , 2255-2259
Abstract
We address the problem of depth-normals fusion under conditions where the input depth map contains holes and the photometric stereo (PS) normals are degraded by modeling and calibration imperfections. To improve depth inpainting within these holes-where reconstruction depends entirely on surface normals-we introduce a joint optimization approach that reconstructs the depth while simultaneously computing a parametric correction for the surface gradients derived from the PS normals. This correction is estimated from pixel locations where reliable depth information is available but is applied globally, ensuring the gradients used for inpainting inside the holes are also corrected. We formulate the correction as the sum of a global polynomial in gradient space and a spatially smooth polynomial in image coordinates, targeting only the low-frequency component of the PS gradients. The resulting bi-convex objective function is solved by alternating between a convex depth reconstruction step and a closed-form least squares update for the correction parameters. We demonstrate the performance of our approach both on real data from our own sensor system and on data derived from the DiLiGenT-MV dataset.Keywords
This publication has 19 references indexed in Scilit:
- Perspective‐Aware Fusion of Incomplete Depth Maps and Surface Normals for Accurate 3D ReconstructionElectronics Letters, 2026
- Bilateral Normal Integration2022
- LERPS: Lighting Estimation and Relighting for Photometric Stereo2022
- Experimental Comparison of Optical Inline 3D Measurement and Inspection SystemsIEEE Access, 2021
- Normal Integration: A SurveyJournal of Mathematical Imaging and Vision, 2017
- Filling Kinect depth holes via position-guided matrix completionNeurocomputing, 2016
- Operating Principles of Structured Light Depth Cameras2016
- High Quality Photometric Reconstruction Using a Depth Camera2014
- Shading-Based Shape Refinement of RGB-D Images2013
- Depth from gradient fields and control points: bias correction in photometric stereoImage and Vision Computing, 2004