DBDaniele Bonatto

Daniele Bonatto

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Daniele Bonatto has a total of 21 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • $\partial $DIBR: Differentiable Depth Image-based Rendering for Fast Novel View Synthesis

    by Armand Losfeld, Sarah Dury, Gauthier Lafruit, Mehrdad Teratani, Daniele Bonatto

  • Open Access

    Failing to Reflect Reality: Gaussian Splatting’s Collapse on Non-Lambertian Surfaces

    by Eva Dubar, Sarah Dury, Gauthier Lafruit, Daniele Bonatto

    We explore the applicability of Gaussian Splatting (GS) to the view synthesis (VS) of non- Lambertian objects. Materials with anisotropic appearance have always been a failure case for VS, until the recent introduction of spherical harmonics promising satisfying results to handle view-directional aspects. Nevertheless, we demonstrate that highly non-Lambertian objects, such as glass or mirrors, remain impossible to correctly reconstruct and render with Gaussian primitives and spherical harmonics. This paper clarifies the reasons: (1) Gaussian Splatting optimization converges to incorrect geometry, and (2) spherical harmonics fail to represent consistent view dependencies. Benchmarking experiments confirm that GS produces false geometry even with ground truth data. Our findings call into question current material representations in view synthesis and motivate the use of more realistic primitives.

  • Open Access

    Structure-from-motion in micro-image domain for uncalibrated plenoptic 2.0 cameras

    by Sarah Dury, Daniele Bonatto, Jaime Sancho, Eduardo Juarez, Mehrdad Teratani, Gauthier Lafruit

    We introduce a structure-from-motion method specifically designed to process the raw micro-images captured by plenoptic 2.0 cameras. Unlike traditional monocular cameras, plenoptic cameras incorporate a micro-lens array between the sensor and the main lens, capturing depth information at the expense of a more complex set of parameters to evaluate. Instead of simply integrating their projection model into the classical structure-from-motion pipeline, our contribution identifies the pinhole cameras-driven constraints and takes advantage of the inherent disparity information present in plenoptic cameras. This facilitates a robust initialization of the reconstruction. Our method shortcuts two of the limitations of the classical structure-from-motion: the ambiguity found in scenes captured with low angular disparity and the scale ambiguity. It enables the reconstruction of scenes captured by multiple uncalibrated plenoptic cameras, without using any calibration pattern or subaperture view extraction step. Our method undergoes experimental validation on both natural and synthetic datasets, showing a 10% error accuracy for relative pose estimation, which is comparable to calibration-pattern based methods. The results are robust to coarse initialization. Contrary to classical structure-from-motion, it is able to reconstruct scenes with parallel facing cameras. It also shows greater accuracy than reconstruction methods based on pinhole camera conversion.


  • Open Access

    Micro-Image Domain View Synthesizer for Free Navigation With Focused Plenoptic Cameras

    by Sarah Fachada, Daniele Bonatto, Gauthier Lafruit, Mehrdad Teratani

    We present a novel, first-of-its-kind view synthesis method for plenoptic images, which enables the direct manipulation of images in the micro-images array format, thereby bypassing intermediate transformation steps. Current plenoptic imaging approaches typically rely on an initial conversion to dense multiview images, also known as subaperture images extraction. However, the use of subaperture images presents two main limitations that ultimately impact further processing. First, existing subaperture view extraction methods offer limited control over camera parameters, resolutions, and poses of the subaperture views, which are also constrained to a small area around the main lens, thus restricting free navigation. Second, subaperture images are susceptible to artifacts which can propagate to subsequent processes such as calibration, depth estimation and view synthesis. In this paper, we propose a camera model that enables depth image-based rendering with plenoptic cameras, in a way that allows for the direct synthesis of any target viewpoint. In our evaluation, we show that our method expands view synthesis extrapolation to a range that is two to three times greater than that of pipelines requiring a conversion to subaperture images, including generally accepted tools such as depth image-based rendering and learning-based rendering approaches.


  • Synchronization and Calibration of Video Sequences Acquired Using Multiple Plenoptic 2.0 Cameras

    by Daniele Bonatto, Sarah Fachada, Jaime Sancho, Eduardo Juarez, Gauthier Lafruit, Mehrdad Teratani


  • Open Access

    Deep Learning for Reaction-Diffusion Glioma Growth Modeling: Towards a Fully Personalized Model?

    by Corentin Martens, Antonin Rovai, Daniele Bonatto, Thierry Metens, Olivier Debeir, Christine Decaestecker, Serge Goldman, Gaetan Van Simaeys

    Reaction-diffusion models have been proposed for decades to capture the growth of gliomas, the most common primary brain tumors. However, ill-posedness of the initialization at diagnosis time and parameter estimation of such models have restrained their clinical use as a personalized predictive tool. In this work, we investigate the ability of deep convolutional neural networks (DCNNs) to address commonly encountered pitfalls in the field. Based on 1200 synthetic tumors grown over real brain geometries derived from magnetic resonance (MR) data of six healthy subjects, we demonstrate the ability of DCNNs to reconstruct a whole tumor cell-density distribution from only two imaging contours at a single time point. With an additional imaging contour extracted at a prior time point, we also demonstrate the ability of DCNNs to accurately estimate the individual diffusivity and proliferation parameters of the model. From this knowledge, the spatio-temporal evolution of the tumor cell-density distribution at later time points can ultimately be precisely captured using the model. We finally show the applicability of our approach to MR data of a real glioblastoma patient. This approach may open the perspective of a clinical application of reaction-diffusion growth models for tumor prognosis and treatment planning.


  • Open Access

    Real-Time Depth Video-Based Rendering for 6-DoF HMD Navigation and Light Field Displays

    by Daniele Bonatto, Sarah Fachada, Segolene Rogge, Adrian Munteanu, Gauthier Lafruit

    This paper presents a novel approach to provide immersive free navigation with 6 Degrees of Freedom in real-time for natural and virtual scenery, for both static and dynamic content. Stemming from the state-of-the-art in Depth Image-Based Rendering and the OpenGL pipeline, this new View Synthesis method achieves free navigation at up to 90 FPS and can take any number of input views with their corresponding depth maps as priors. Video content can be played thanks to GPU decompression, supporting free navigation with full parallax in real-time. To render a novel viewpoint, each selected input view is warped using the camera pose and associated depth map, using an implicit 3D representation. The warped views are then blended all together to generate the chosen virtual view. Various view blending approaches specifically designed to avoid visual artifacts are compared. Using as few as four input views appears to be an optimal trade-off between computation time and quality, allowing to synthesize high-quality stereoscopic views in real-time, offering a genuine immersive virtual reality experience. Additionally, the proposed approach provides high-quality rendering of a 3D scenery on holographic light field displays. Our results are comparable - objectively and subjectively - to the state of the art view synthesis tools NeRF and LLFF, while maintaining an overall lower complexity and real-time rendering.


  • Open Access

    Consistent Long Sequences Deep Faces

    by Xudong Fan, Daniele Bonatto, Gauthier Lafruit

    Face swapping in videos usually has strong entertainment applications. Deep Fakes (in Faces) are a recent topic in deep learning where the main idea is to substitute the face of a person in a video with the face of another person. But one of the drawbacks of the method is that between two successive frames there are inconsistencies between the faces, such as changing face color, flickering or eyebrows that change. In this paper, we propose a convolutional neural network for swapping faces based on two autoencoders which share the same encoder. In this network, the encoder can distinguish and extract important features of faces, including facial expressions and poses; the decoders will then reconstruct faces according to these features. First, we will generate datasets of faces respectively for person A and person B. Secondly, the local information of two faces is sent to the network to get the model; after the training process, we can use the model to reconstruct the corresponding face of person B when the input is one face of person A. Afterwards, we build a binary mask to select the face area and transfer color from the source face to the target face. Finally, we only need to use a seamless clone to merge the new faces back into the source frames to create a fake video. The experimental results show that the quality of the fake videos is improved significantly.

  • Natural Scenes Datasets for Exploration In 6DOF Navigation

    by Arnaud Schenkel, Daniele Bonatto, Sarah Fachada, Henry-Louis Guillaume, Gauthier Lafruit

    Depth Image-Based Rendering (DIBR) and 6 Degrees-of-Freedom (6DoF) navigation are actually very active fields for immersive video, where the availability of common test conditions is mandatory to compare results. For this purpose, we propose different datasets to evaluate a wide range of tools needed by free navigation: a high-density HD image set, composed of 80k camera positions covering the entire 3D space, necessary to simulate all the head rotations and translations of a common user, and a UHD video set, composed of two sequences acquired with 3 by 5 cameras array. All the processing pipeline to obtain quality results is also detailed.