João Madeiras Pereira

Autonomous driving technology is advancing rapidly, driven by integrating advanced intelligent systems. Autonomous vehicles typically follow a modular structure, organized into perception, planning, and control components. Unlike previous surveys, which often focus on specific modular system components or single driving environments, our review uniquely compares both settings, highlighting how deep learning and reinforcement learning methods address the challenges specific to each. We present an in-depth analysis of local and global planning methods, including the integration of benchmarks, simulations, and real-time platforms. Additionally, we compare various evaluation metrics and performance outcomes for current methodologies. Finally, we offer insights into emerging research directions based on the latest advancements, providing a roadmap for future innovation in autonomous driving.

This study investigates the impact of machine learning models on the generation of counterfactual explanations by conducting a benchmark evaluation over three different types of models: a decision tree (fully transparent, interpretable, white-box model), a random forest (semi-interpretable, grey-box model), and a neural network (fully opaque, black-box model). We tested the counterfactual generation process using four algorithms (DiCE, WatcherCF, prototype, and GrowingSpheresCF) in the literature in 25 different datasets. Our findings indicate that: (1) Different machine learning models have little impact on the generation of counterfactual explanations; (2) Counterfactual algorithms based uniquely on proximity loss functions are not actionable and will not provide meaningful explanations; (3) One cannot have meaningful evaluation results without guaranteeing plausibility in the counterfactual generation. Algorithms that do not consider plausibility in their internal mechanisms will lead to biased and unreliable conclusions if evaluated with the current state-of-the-art metrics; (4) A counterfactual inspection analysis is strongly recommended to ensure a robust examination of counterfactual explanations and the potential identification of biases.

Ray tracing is a computer graphics technique used to produce realistic visuals by physically simulating the behavior of light. Although this technique can be described straightforwardly, fully comprehending it might be challenging. It is typically taught in the classroom using the 2-D formats, such as paper or a blackboard. We propose using immersive environments for incrementing the understanding of ray tracing. We focus on improving the knowledge of the technique in experienced users, particularly Master of Computer Science students minoring in a computer-graphics-related area. We argue that exploring the ray tracing process in an immersive visualization environment can further improve the understanding of ray tracing acquired using conventional means. With that objective, this study starts by presenting RayTracerVR , a virtual reality prototype tool for learning the mechanisms of ray tracing. This tool can be used to visually explore and interact with the different aspects of the technique. It allows users to observe the progression of the rays throughout the sequential stages of the ray tracing process and analyze its corresponding computer pseudocode. The study includes user evaluation where RayTracerVR is employed to assess improvements in ray tracing understanding. The prototype's usability is also assessed. The findings indicate that using the ray tracing immersive learning environment results in a supplemental increase in understanding in users who have previously learned ray tracing using conventional means.

This paper introduces a contribution made to one of the newest methods for simulating indirect lighting in dynamic scenes , the cascaded light propagation volumes . Our contribution consists on using Spherical Radial Basis Functions instead of Spherical Harmonic, since the first achieves much better results when many coefficients are used. We explain how to integrate the Spherical Radial Basis Functions with the cascaded light propagation volumes, and evaluate our technique against the same implementation, but with Spherical harmonics.

Depth maps produced by consumer-grade sensors suffer from inaccurate measurements and missing data from either system or scene-specific sources. Data-driven denoising algorithms can mitigate such problems; however, they require vast amounts of ground truth depth data. Recent research has tackled this limitation using self-supervised learning techniques, but it requires multiple RGB-D sensors. Moreover, most existing approaches focus on denoising single isolated depth maps or specific subjects of interest highlighting a need for methods that can effectively denoise depth maps in real-time dynamic environments. This paper extends state-of-the-art approaches for depth-denoising commodity depth devices, proposing SelfReDepth, a self-supervised deep learning technique for depth restoration, via denoising and hole-filling by inpainting of full-depth maps captured with RGB-D sensors. The algorithm targets depth data in video streams, utilizing multiple sequential depth frames coupled with color data to achieve high-quality depth videos with temporal coherence. Finally, SelfReDepth is designed to be compatible with various RGB-D sensors and usable in real-time scenarios as a pre-processing step before applying other depth-dependent algorithms. Our results demonstrate our approach’s real-time performance on real-world datasets shows that it outperforms state-of-the-art methods in denoising and restoration performance at over 30 fps on Commercial Depth Cameras, with potential benefits for augmented and mixed-reality applications.

One of the current challenges in physically-based simulations, and, more specifically, fluid simulations, is to produce visually appealing results at interactive rates, capable of being used in multiple forms of media. In recent times, a lot of effort has been made with regards to this with the use of multi-core architectures, as many of the computations involved in the algorithms for these simulations are very well suited for these architectures. Although there is a considerable amount of works regarding acceleration techniques in this field, there is yet room to further explore and analyze some of them. To investigate this problem, we surveyed the topic of fluid simulations and some of the recent contributions towards this field. Additionally, we implemented two versions of a fluid simulation algorithm, one on the CPU and the other on the GPU using NVIDIA's CUDA framework, with the intent of gaining a better understanding of the effort needed to move these simulations to a multi-core architecture and the performance gains that we get with it.

One of the main performance-heavy areas in raytracing is direct lighting. Direct lighting is usually solved in a process called next-event estimation which samples light sources at each ray intersection. Since real-time rendering is an objective, instead of sampling all the luminaries, just a set of lights deemed more important are sampled by using a technique called Monte Carlo Importance Sampling. Aiming further acceleration, some importance sampling-based approaches build hierarchical data structures over all light sources, which results in high maintenance costs for dynamic scenes. By using a previously introduced algorithm, this paper describes a two-level light bounding volume hierarchy (BVH) to accelerate direct lighting while minimising the quality loss in dynamic scenes common in similar algorithms. The main advantage of it is the ability to have dynamic lights without losing excessive accuracy or performance to maintenance operations. Our approach was developed as an extension to the Mogwai"s PathTracer application from NVIDIA to support direct lighting. The algorithm rebuilds the top-level structure in the CPU allowing it to retain its accuracy and refits only the bottom-level structures in need of updating on the GPU. The CPU rebuild is a relatively costly operation, to avoid excessive performance loss it is done asynchronously only being used in the next frame. We tested several quality metrics as well as frame times, our implementation achieved an up to 36% increase in quality metrics and an average of 7% slowdown when compared with the single-level BVH.

Ray tracing has long been the holy grail of real time rendering. This technique, commonly used for photo realism, simulates the physical behavior of light, at the cost of being computationally heavy. With the introduction of Nvidia RTX graphic card family, which provides hardware support for ray tracing, this technique started to look like a reality for real time. However, the same problems that afflicted the usage of this technique remain, and even with specialized hardware it is still extremely expensive. To account for these drawbacks, researchers and developers pair this technique with rasterization and denoising. This results in a hybrid system that tries to join the best of both worlds, having both photo realistic quality and real time performance. In this work we intend on further exploring hybrid render systems, offering a review of the state of the art with a special focus on real time ray tracing and our own hybrid implementation with photo realistic quality and real time performance (>30 fps), implemented using the Vulkan API. In this project, we highlight the detailed analysis of the impacts of History Rectification (Variance Color Clamping) on the temporal filter component of the denoising system and how to overcome the introduced artifacts. Additionally, we also highlight the analysis of the introduction of a separable blur on the spatial filter and the introduction of Reinhard Tone Mapping prior to denoising, consequently improving this procedure.

Three-dimensional reconstruction of events recorded on images has been a common challenge between computer vision and computer graphics for a long time. Estimating the real position of objects and surfaces using vision as an input is no trivial task and has been approached in several different ways. Although huge progress has been made so far, there are several open issues to which an answer is needed. The use of videos as an input for a rendering process (video-based rendering, VBR) is something that recently has been started to be looked upon and has added many other challenges and also solutions to the classical image-based rendering issue (IBR). This article presents the state of art on video-based rendering and image-based techniques that can be applied on this scenario, evaluating the open issues yet to be solved, indicating where future work should be focused.