Shaokun Han

A high-precision extrinsic calibration is the underlying premise of the accurate perception of light detection and ranging (LiDAR) and camera systems commonly used in the autonomous driving industry. We propose a coarse-to-fine strategy to get rigid-body transformation between solid-state LiDAR with non-repetitive scanning and a RGB camera system using a chessboard as the calibration target. This method exploits the reflectance intensity characteristics of the LiDAR point cloud, which exhibit the distinct distribution in white and black blocks of chessboard. In the coarse calibration step, a reflectance intensity Gaussian mixture model was used to extract the unicolor block point cloud from the chessboard point cloud. Therefore, the initial estimate of the extrinsic parameter was obtained by aligning the corners in the point cloud and calculating the centroid of the unicolor block point cloud and corners in the image. In the refinement step, we extracted points on the border of each block as LiDAR features and designed an iterative optimization algorithm to align the intensity of LiDAR features with grayscale features in the image. This method utilizes the intensity information and compensates for corner errors in the point cloud due to reflectance intensity binarization. The results of the comparative experiment revealed that the proposed method outperformed existing methods in terms of accuracy. Experiments based on simulations and real-world conditions revealed that the proposed algorithm demonstrated a high accuracy, robustness, and consistency.

Point cloud completion task aims to predict the missing part of incomplete point clouds and generate complete point clouds with details. In this paper, we propose a novel point cloud completion network, namely CompleteDT. Specifically, features are learned from point clouds with different resolutions, which is sampled from the incomplete input, and are converted to a series of \textit{spots} based on the geometrical structure. Then, the Dense Relation Augment Module (DRA) based on the transformer is proposed to learn features within \textit{spots} and consider the correlation among these \textit{spots}. The DRA consists of Point Local-Attention Module (PLA) and Point Dense Multi-Scale Attention Module (PDMA), where the PLA captures the local information within the local \textit{spots} by adaptively measuring weights of neighbors and the PDMA exploits the global relationship between these \textit{spots} in a multi-scale densely connected manner. Lastly, the complete shape is predicted from \textit{spots} by the Multi-resolution Point Fusion Module (MPF), which gradually generates complete point clouds from \textit{spots}, and updates \textit{spots} based on these generated point clouds. Experimental results show that, because the DRA based on the transformer can learn the expressive features from the incomplete input and the MPF can fully explore these feature to predict the complete input, our method largely outperforms the state-of-the-art methods.

Non-repetitive scanning Light Detection And Ranging(LiDAR)-Camera systems are commonly used in autonomous navigation industries, benefiting from their low-cost and high-perception characteristics. However, due to the irregular scanning pattern of LiDAR, feature extraction on point cloud encounters the problem of non-uniformity distribution of density and reflectance intensity, accurate extrinsic calibration remains a challenging task. To solve this problem, this paper presented an open-source calibration method using only a printed chessboard. We designed a two-stage coarse-to-fine pipeline for 3D corner extraction. Firstly, a Gaussian Mixture Model(GMM)-based intensity cluster approach is proposed to adaptively identify point segments in different color blocks of the chessboard. Secondly, a novel Iterative Lowest-cost Pose(ILP) algorithm is designed to fit the chessboard grid and refine the 3D corner iteratively. This scheme is unique for turning the corner feature extraction problem into a grid align problem. After the corresponding 3D-2D points are solved, by applying the PnP(Perspective-n-Point) method, along with nonlinear-optimization refinement, the extrinsic parameters are obtained. Extensive simulation and real-world experimental results show that our method achieved subpixel-level precision in terms of reprojection error. The comparison demonstrated that the effectiveness and accuracy of the proposed method outperformed existing methods.

Vision-related state estimation usually extracts multiple feature points from images captured by the camera. In this paper, we propose a robust feature homogenization method for resolving the problem of feature clustering. The proposed method deduced the depth of feature points from optical flow magnitude, and the homogenization of feature points was acquired by adaptively enforcing the minimum distance between neighboring feature points. With the assistance of optical flow, the proposed method develops a preference for feature points with smaller depths in feature homogenization. Experimental results show that the proposed method improves the system’s global consistency and tracking stability by using optical flow information.

Combined with the characteristic of the multi-slit streak tube imaging LiDAR (MS-STIL) system where the imaging areas corresponding to each slit do not interfere with each other, we denoised the streak images by an improved fast non-local mean filtering algorithm. Experiments were performed to investigate the effectiveness of the method. The experimental results show that the spatial resolution of the system is improved from 22 to 16 mm; the relative distance error is reduced by an average of 22.76%; and the intensity accuracy improved significantly when the distance is 10 m. Additionally, the overall denoising effect is comprehensively verified by long-range target imaging. The mean square error of the reconstructed depth image and intensity image are reduced from 0.0836, 0.0067 to 0.0433, 0.0037, respectively. The applicability of the proposed method was verified through comparative experiments in different environments.

To reduce the impact of the image reconstruction process and improve the identification efficiency of the multislit streak tube imaging lidar (MS-STIL) system, an object classification method based on the echo of the MS-STIL system is proposed. A streak image data set is constructed that contains a total of 240 common outdoor targets in 6 categories. Additionally, the deep-learning network model based on ResNet is chosen to implement streak image classification. The effects of two classification methods based on streak images and reconstructed depth images are compared. To verify the maximum classification capability of the proposed method, the recognition effects are investigated under 6 and 20 classes. The results show that the classification accuracy decreases from 99.42% to 67.64%. After the data set is expanded, the classification accuracy improved to 85.35% when the class number of the target is 20.

Using unmanned aerial vehicles (UAVs) for remote sensing has the advantages of high flexibility, convenient operation, low cost, and wide application range. It fills the need for rapid acquisition of high-resolution aerial images in modern photogrammetry applications. Due to the insufficient parallaxes and the computation-intensive process, dense real-time reconstruction for large terrain scenes is a considerable challenge. To address these problems, we proposed a novel SLAM-based MVS (Multi-View-Stereo) approach, which can incrementally generate a dense 3D (three-dimensional) model of the terrain by using the continuous image stream during the flight. The pipeline of the proposed methodology starts with pose estimation based on SLAM algorithm. The tracked frames were then selected by a novel scene-adaptive keyframe selection method to construct a sliding window frame-set. This was followed by depth estimation using a flexible search domain approach, which can improve accuracy without increasing the iterate time or memory consumption. The whole system proposed in this study was implemented on the embedded GPU based on an UAV platform. We proposed a highly parallel and memory-efficient CUDA-based depth computing architecture, enabling the system to achieve good real-time performance. The evaluation experiments were carried out in both simulation and real-world environments. A virtual large terrain scene was built using the Gazebo simulator. The simulated UAV equipped with an RGB-D camera was used to obtain synthetic evaluation datasets, which were divided by flight altitudes (800-, 1000-, 1200 m) and terrain height difference (100-, 200-, 300 m). In addition, the system has been extensively tested on various types of real scenes. Comparison with commercial 3D reconstruction software is carried out to evaluate the precision in real-world data. According to the results on the synthetic datasets, over 93.462% of the estimation with absolute error distance of less then 0.9%. In the real-world dataset captured at 800 m flight height, more than 81.27% of our estimated point cloud are less then 5 m difference with the results of Photoscan. All evaluation experiments show that the proposed approach outperforms the state-of-the-art ones in terms of accuracy and efficiency.

This work introduces a super-resolution (SR) algorithm for range images on the basis of self-guided joint filtering (SGJF), adding the range information of the range image as a coefficient of the filter to reduce the influence of the intensity image texture on the super-resolved image. A range image SR recognition system is constructed to study the effect of four SR algorithms including the SGJF algorithm on the recognition of the laser radar (ladar) range image. The effects of different model library sizes, SR algorithms, SR factors and noise conditions on the recognition are tested via experiments. Results demonstrate that all tested SR algorithms can improve the recognition rate of low-resolution (low-res) range images to varying degrees and the proposed SGJF algorithm has a very good comprehensive recognition performance. Finally, suggestions for the use of SR algorithms in actual scene recognition are proposed on the basis of the experimental results.

The complementary advantages of the multi-spectral technique and streak tube detection imaging technique can enrich target information with high detection accuracy. In this paper, a new multi-spectral streak tube imaging method is proposed. Instead of using a set of receiving systems for each wavelength, i.e., the traditional method, we use different lengths of fiber arrays in our imaging system to produce the time difference between the echo signals of different wavelengths arriving at the detectors. The imaging process is completed by just one set of the receiving system. The contradiction between multiple wavelength information sub-area processing and large depth of field is alleviated, and the high time resolution is ensured. The mathematical model of this simplified system is established, and two simulation experiments are organized. First, the influence of the time difference on the judgment of echo signals is discussed. Second, we simulate the process of detecting and imaging a target using the traditional multi-spectral streak tube detection method and the proposed method. Finally, we compare the three-dimensional images obtained via the two methods. By observation and calculation, we find that the imaging depth of field and time resolution of the proposed method are better than those of the traditional method.