3DCapture: 3D Reconstruction for a Smartphone

Authors:
  • OMOleg Muratov
    Oleg Muratov
  • YSYury Slynko
    Yury Slynko
  • VCVitaly Chernov
    Vitaly Chernov
  • MLMaria Lyubimtseva
    Maria Lyubimtseva
  • ASArtem Shamsuarov
    Artem Shamsuarov
  • VBVictor Bucha
    Victor Bucha

Abstract
We propose a method of reconstruction of 3D representation (a mesh with a texture) of an object on a smartphone with a monocular camera. The reconstruction consists of two parts - real-time scanning around the object and postprocessing. At the scanning stage IMU sensors data are acquired along with tracks of features in video. A special care is taken to comply with 360 scan requirement. All these data are used to build a camera trajectory using bundle adjustment techniques after scanning is completed. This trajectory is used in calculation of depth maps, which then are used to construct a polygonal mesh with overlaid textures. The proposed method ensures tracking at 30 fps on a modern smartphone while the post-processing part is completed within 1 minute using an OpenCL compatible mobile GPU. In addition, we show that with a few modifications this algorithm can be adopted for human face reconstruction.