Joint Learning of 3D Shape Retrieval and Deformation
Authors:
- 1 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 11708-11717
Abstract
We propose a novel technique for producing high-quality 3D models that match a given target object image or scan. Our method is based on retrieving an existing shape from a database of 3D models and then deforming its parts to match the target shape. Unlike previous approaches that independently focus on either shape retrieval or deformation, we propose a joint learning procedure that simultaneously trains the neural deformation module along with the embedding space used by the retrieval module. This enables our network to learn a deformation-aware embedding space, so that retrieved models are more amenable to match the target after an appropriate deformation. In fact, we use the embedding space to guide the shape pairs used to train the deformation module, so that it invests its capacity in learning deformations between meaningful shape pairs. Furthermore, our novel part-aware deformation module can work with inconsistent and diverse part-structures on the source shapes. We demonstrate the benefits of our joint training not only on our novel framework, but also on other state- of-the-art neural deformation modules proposed in recent years. Lastly, we also show that our jointly-trained method outperforms various non-joint baselines.Keywords
All Related Versions
This publication has 35 references indexed in Scilit:
- GRASSACM Transactions on Graphics, 2017
- Volumetric and Multi-view CNNs for Object Classification on 3D Data2016
- Deep Residual Learning for Image Recognition2016
- Joint embeddings of shapes and images via CNN image purificationACM Transactions on Graphics, 2015
- Learning part-based templates from large collections of 3D shapesACM Transactions on Graphics, 2013
- A search-classify approach for cluttered indoor scene understandingACM Transactions on Graphics, 2012
- Photo-inspired model-driven 3D object modelingACM Transactions on Graphics, 2011
- Component‐wise Controllers for Structure‐Preserving Shape ManipulationComputer Graphics Forum, 2011
- Non‐Rigid Registration Under Isometric DeformationsComputer Graphics Forum, 2008
- As-rigid-as-possible shape manipulationACM Transactions on Graphics, 2005