Matthieu Fradet has a total of 26 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Detection of Removed Objects in 3D Meshes Using Up-to-Date Images for Mixed-Reality Applications

    by Olivier Roupin, Matthieu Fradet, Caroline Baillard, Guillaume Moreau

    Precise knowledge of the real environment is a prerequisite for the integration of the real and virtual worlds in mixed-reality applications. However, real-time updating of a real environment model is a costly and difficult process; therefore, hybrid approaches have been developed: An updated world model can be inferred from an offline acquisition of the 3D world, which is then updated online using live image sequences under the condition of developing fast and robust change detection algorithms. Current algorithms are biased toward object insertion and often fail in object removal detection; in an environment where there is uniformity in the background—in color and intensity—the disappearances of foreground objects between the 3D scan of a scene and the capture of several new pictures of said scene are difficult to detect. The novelty of our approach is that we circumvent this issue by focusing on areas of least change in parts of the scene that should be occluded by the foreground. Through experimentation on realistic datasets, we show that this approach results in better detection and localization of removed objects. This technique can be paired with an insertion detection algorithm to provide a complete change detection framework.


  • Open Access

    Efficient texture mapping via a non-iterative global texture alignment

    by Mohammad Rouhani, Matthieu Fradet, Caroline Baillard

    Texture reconstruction techniques generally suffer from the errors in keyframe poses. We present a non-iterative method for seamless texture reconstruction of a given 3D scene. Our method finds the best texture alignment in a single shot using a global optimisation framework. First, we automatically select the best keyframe to texture each face of the mesh. This leads to a decomposition of the mesh into small groups of connected faces associated to a same keyframe. We call such groups fragments. Then, we propose a geometry-aware matching technique between the 3D keypoints extracted around the fragment borders, where the matching zone is controlled by the margin size. These constraints lead to a least squares (LS) model for finding the optimal alignment. Finally, visual seams are further reduced by applying a fast colour correction. In contrast to pixel-wise methods, we find the optimal alignment by solving a sparse system of linear equations, which is very fast and non-iterative. Experimental results demonstrate low computational complexity and outperformance compared to other alignment methods.

  • Probeless and Realistic Mixed Reality Application in Presence of Dynamic Light Sources

    by Salma Jiddi, Philippe Hobert, Anthony Laurent, Matthieu Fradet, Pierrick Jouet, Caroline Baillard, Eric Marchand

    In this work, we consider the challenge of achieving a coherent blending between real and virtual worlds in the context of a Mixed Reality (MR) scenario. Specifically, we have designed and implemented an interactive demonstrator that shows a realistic MR application without using any light probe. The proposed system takes as input the RGB stream of the real scene, and uses these data to recover both the position and intensity of light sources. The lighting can be static and/or dynamic and the geometry of the scene can be partially altered. Our system is robust in presence of specular effects and handles both uniform and/or textured surfaces.


  • Towards Mobile Diminished Reality

    by Glen Queguiner, Matthieu Fradet, Mohammad Rouhani

    We present a diminished reality application running live on consumer mobile devices. In our pre-observation-based approach, the clean 3D scene, free of undesired objects, is scanned beforehand and reconstructed as a high resolution textured 3D model. At runtime, objects added in a region of interest are efficiently removed by projecting the previously captured background. Differences of illumination conditions between scan time and run-time are compensated to obtain seamless results. The proposed approach requires no segmentation or manual input other than the definition of the 3D region of interest to be diminished, and is not based on any particular assumption on the background geometry. We show the potential of our approach by processing a variety of challenging unknown 3D scenes including textured backgrounds, dynamic illumination conditions and foreground objects partially occluding the diminished region. We provide details on our compute shader implementation to make as easy as possible the reimplementation by the community.


  • A Multi-resolution Approach for Color Correction of Textured Meshes

    by Mohammad Rouhani, Matthieu Fradet, Caroline Baillard

    Mesh texturing is an essential part of 3D scene reconstruction that enables a more realistic perception than the geometry alone and even compensates for inaccurate geometry. In this work we present a flexible formulation for color correction of textured scenes based on color augmentation per face. It can be employed as a post-processing step after selecting the best keyframe per face to compensate for color differences between pairs of neighboring faces. We present a Markov Random Field (MRF) formulation to find the best keyframes as well as the optimal color augmentations. We use a simple model to avoid reflection and camera vignetting during the view selection. Our model for color correction finds the piecewise-linear augmentation to be added to the texture patches of faces. It encourages smoothness inside every fragment while compensating color differences along view transitions. Moreover, we speed up the optimization by breaking down the formulation into multiple binary MRFs that estimate the best augmentations from coarse to fine resolutions. The experimental results prove our method outperforming the state of the art methods.


  • Open Access

    Multi-device mixed reality TV

    by Caroline Baillard, Matthieu Fradet, Vincent Alleaume, Pierrick Jouet, Anthony Laurent


  • [POSTER] MR TV Mozaik: A New Mixed Reality Interactive TV Experience

    by Matthieu Fradet, Caroline Baillard, Anthony Laurent, Tao Luo, Philippe Robert, Vincent Alleaume, Pierrick Jouet, Fabien Servant

    Technicolor has been investigating how Mixed Reality technology could impact the future of home entertainment. We have designed and implemented a system to extend a standard TV experience with AR content, using a consumer tablet or a headset. A virtual TV mosaic is displayed around the TV screen and used as a GUI to control both TV and MR content. Using this interface, the user is able to switch TV content, display meta-data in AR (subtitles, text information or program guide), enhance TV content with interactive 3D objects blended in the environment, or play a game in interaction with the real world. The interactions between the real and the virtual worlds are handled thanks to a scene analysis pre-processing stage, which provides information about both the geometry and the lighting of the real environment. The real-virtual interactions strongly contribute to reinforcement of the immersion feeling. User feedback shows that the concept is very promising.


  • Mixed Reality Extended TV

    by Caroline Baillard, Vincent Alleaume, Matthieu Fradet, Pierrick Jouet, Anthony Laurent, Tao Luo, Philippe Robert, Fabien Servant

    The Extended TV application allows to enhance an audiovisual content displayed on a TV, using Mixed Reality technology. During preprocessing, the close environment of the TV is scanned using a consumer depth camera. The captured RGB-D data are analyzed, providing models for both the 3D geometry and the lighting of the real scene. During runtime, the TV is watched through a tablet, and virtual objects can apparently come out of the screen and start populating the user's environment. Virtual objects can be occluded by real objects, and virtual shadows are consistent with the real ones.


  • Robust Optical Flow Integration

    by Tomas Crivelli, Matthieu Fradet, Pierre-Henri Conze, Philippe Robert, Patrick Perez

    We analyze the problem of how to correctly construct dense point trajectories from optical flow fields. First, we show that simple Euler integration is unavoidably inaccurate, no matter how good is the optical flow estimator. Then, an inverse integration scheme is analyzed which is more robust to bias and input noise and shows better stability properties. Our contribution is threefold: 1) a theoretical analysis that demonstrates why and in what sense inverse integration is more accurate; 2) a rich experimental validation both on synthetic and real (image) data; and 3) an algorithm for approximate online inverse integration. This new technique is precious whether one is trying to propagate information densely available on a reference frame to the other frames in the sequence or, conversely, to assign information densely over each frame by pulling it from the reference.


  • Video Inpainting of Complex Scenes

    by Alasdair Newson, Andrés Almansa, Matthieu Fradet, Yann Gousseau, Patrick Pérez

    We propose an automatic video inpainting algorithm which relies on the optimization of a global, patch-based functional. Our algorithm is able to deal with a variety of challenging situations which naturally arise in video inpainting, such as the correct reconstruction of dynamic textures, multiple moving objects, and moving background. Furthermore, we achieve this in an order of magnitude less execution time with respect to the state-of-the-art. We are also able to achieve good quality results on high-definition videos. Finally, we provide specific algorithmic details to make implementation of our algorithm as easy as possible. The resulting algorithm requires no segmentation or manual input other than the definition of the inpainting mask and can deal with a wider variety of situations than is handled by previous work.