SENS: Sketch-based Implicit Neural Shape Modeling

Authors:
  • ABAlexandre Binninger
    Alexandre Binninger
  • AHAmir Hertz
    Amir Hertz
  • OSOlga Sorkine-Hornung
    Olga Sorkine-Hornung
  • DCDaniel Cohen-Or
    Daniel Cohen-Or
  • RGRaja Giryes
    Raja Giryes
Preprint
Publisher website
Google Scholar
Abstract
We present SENS, a novel method for generating and editing 3D models from hand-drawn sketches, including those of an abstract nature. Our method allows users to quickly and easily sketch a shape, and then maps the sketch into the latent space of a part-aware neural implicit shape architecture. SENS analyzes the sketch and encodes its parts into ViT patch encoding, then feeds them into a transformer decoder that converts them to shape embeddings, suitable for editing 3D neural implicit shapes. SENS not only provides intuitive sketch-based generation and editing, but also excels in capturing the intent of the user's sketch to generate a variety of novel and expressive 3D shapes, even from abstract sketches. We demonstrate the effectiveness of our model compared to the state-of-the-art using objective metric evaluation criteria and a decisive user study, both indicating strong performance on sketches with a medium level of abstraction. Furthermore, we showcase its intuitive sketch-based shape editing capabilities.

This publication has 44 references indexed in Scilit: