Edgars Lielāmurs

Mechanically scanning LiDAR imaging sensors are abundantly used in applications ranging from basic safety assistance to high-level automated driving, offering excellent spatial resolution and full surround-view coverage in most scenarios. However, their complex optomechanical structure introduces limitations, namely limited mounting options and blind zones, especially in elongated vehicles. To mitigate these challenges, we propose a distributed Time-of-Flight (ToF) sensor system with a flexible hardware–software architecture designed for multi-sensor synchronous triggering and fusion. We formalize the sensor triggering, interference mitigation scheme, data aggregation and fusion procedures and highlight challenges in achieving accurate global registration with current state-of-the-art methods. The resulting surround view visual information is then applied to Spiking Neural Network (SNN)-based object detection and probabilistic occupancy grid mapping (OGM) for enhanced environmental awareness. The proposed system is demonstrated on a test vehicle, achieving coverage of blind zones in a range of 0.5–6 m with a scalable and reconfigurable sensor mounting setup. Using seven ToF sensors, we can achieve a 10 Hz synchronized frame rate, with a 360° point cloud registration and fusion latency below 40 ms. We collected real-world driving data to evaluate the system, achieving 65% mean Average Precision (mAP) in object detection with our SNN. Overall, this work presents a replacement or addition to LiDAR in future high-level automation tasks, offering improved coverage and system integration.

Spatial image transformation is a commonly used component in many image processing pipelines. It enables the correction of optical distortions, image registration onto a common reference plane, electronic image stabilisation, digital zoom, video mosaicking, etc. With the growing tendency to embed image processing in low-power devices, attaining an efficient transformation solution becomes increasingly decisive. Furthermore, interpolation is the key operation in achieving the high quality of the transformed data from the original data. Fortunately, different implementations have already seen several efficiency improvements in recent years. However, interpolation relies on sampling a set of neighbouring points from memory, which has yet to be addressed efficiently for smaller computational platforms with limited memory resources. In this work, we derive a generic mathematical model and circuit design principles for the spatial transformation accelerator design for N-dimensional data. Furthermore, we present an efficient simultaneous access scheme for high-quality signal reconstruction. Finally, the introduced ideas are verified in field programmable gate arrays using one-dimensional and two-dimensional data transformation use cases. The presented solution is able to transform images with sizes ranging from 256 × 256 to 8192 × 8192 and achieves a transfer rate of 275 frames per second with 512 × 512 images.

Infrared imaging sensors are frequently used in thermal signature detection applications in industrial, automotive, military and many other areas. However, advanced infrared detectors are generally associated with high costs and complexity. Infrared detectors usually necessitate a thermoelectric heater–cooler for temperature stabilization and various computationally complex preprocessing algorithms for fixed pattern noise (FPN) correction. In this paper, we leverage the benefits of uncooled focal plane arrays and describe a complete digital circuit design for Field Programmable Gate Array (FPGA)-based infrared image acquisition and pre-processing. The proposed design comprises temperature compensation, non-uniformity correction, defective pixel correction cores, spatial image transformation and registration with RGB images. When implemented on Xilinx Ultrascale+ FPGA, the system achieves a throughput of 30 frames per second using the Fraunhofer IMS Digital 17 μm QVGA-IRFPA with a microbolometer array size of 320 × 240 pixels and an RGB camera with a 1024 × 720 resolution. The maximum ratio of the standard deviation to the mean of 0.35% was achieved after FPN correction.