DepthCloud2Point: Depth Maps and Initial Point for 3D Point Cloud Reconstruction from a Single Image
Open Access
- 12 March 2025
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 14 (6) , 1119
Abstract
Reconstructing 3D objects from single-view images has acquired significant interest due to their wide-ranging applications in robotics, autonomous vehicles, virtual reality, and augmented reality. Current methods, including voxel-based and point cloud-based approaches, face critical challenges such as irregular point distributions and an inability to preserve complex object details, which result in suboptimal reconstructions. To address these limitations, we propose DepthCloud2Point, a framework that combines depth maps, image features, and an initial point cloud to generate detailed and accurate 3D point clouds. Depth maps are employed to provide rich spatial cues, resolving depth ambiguities, while the initial point cloud serves as a geometric prior to ensure uniform point distribution. These components are integrated into a unified pipeline, where the encoder extracts semantic and geometric features, and the generator synthesizes high-fidelity 3D reconstructions. Our approach is trained end-to-end on both synthetic and real-world datasets, achieving state-of-the-art performance. Quantitative results on the ShapeNet dataset show that DepthCloud2Point outperforms 3D-LMNet by 19.07% in CD and 38.86% in EMD, and Pixel2Point by 18.77% in CD and 19.25% in EMD. We also perform a qualitative study that shows that our approach is able to generate reconstructions that closely align with ground truth, capturing intricate object details and maintaining spatial coherence, confirming its outperforming over the 3D-LMNet and Pixel2Point.Keywords
This publication has 29 references indexed in Scilit:
- Advancing 3D point cloud understanding through deep transfer learning: A comprehensive surveyInformation Fusion, 2024
- BinsFormer: Revisiting Adaptive Bins for Monocular Depth EstimationIEEE Transactions on Image Processing, 2024
- Image-based 3D reconstruction for Multi-Scale civil and infrastructure Projects: A review from 2012 to 2022 with new perspective from deep learning methodsAdvanced Engineering Informatics, 2023
- Sparse-to-Dense Matching Network for Large-Scale LiDAR Point Cloud RegistrationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
- A Survey on Active Simultaneous Localization and Mapping: State of the Art and New FrontiersIEEE Transactions on Robotics, 2023
- Deep Learning for 3D Point Clouds: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence, 2021
- Single-View 3D reconstruction: A Survey of deep learning methodsComputers & Graphics, 2021
- Pixel2point: 3D Object Reconstruction From a Single Image Using CNN and Initial SphereIEEE Access, 2020
- Learning Single-Image 3D Reconstruction by Generative Modelling of Shape, Pose and ShadingInternational Journal of Computer Vision, 2019
- Construction of a 3D object recognition and manipulation database from grasp demonstrationsAutonomous Robots, 2015