Yihan Zhong

The great development of smart urban transportation has led to the booming of the autonomous vehicle industry. Society places great expectations on autonomous driving to enhance safety and efficiency in transportation systems, particularly in challenging environments like urban canyons where precise positioning is crucial. Global Navigation Satellite System (GNSS) positioning modules are the crucial element of autonomous vehicles for absolute positioning. However, the GNSS pseudorange suffers significant biases and degradation due to the multipath and non-line-of-sight (NLOS) errors caused by tall buildings that widely exist in urban canyons. To improve GNSS positioning accuracy in harsh urban canyons, this paper introduces a deep multimodal learning-based approach to correct distorted pseudorange measurements. The vision features of the environment captured by the camera are integrated to form the multimodal network with GNSS measurements. Meanwhile, positional encoding (PE) is proposed to fuse the image features and the satellite position information in a higher-dimensional space. The biases are outputted by the network and are used for improved positioning. Compared to existing solutions, extensive test results show that the proposed method achieves improved positioning accuracy ranging from 12% to 20%. Additionally, the method proposed in this paper takes only 0.5 seconds to perform epoch-by-epoch pseudorange correction on Jetson Orin Nano, which meets the real-time requirements and shows extremely high practical value. This paper also opensources a dataset under varying time, satellite distributions, and lighting conditions with accurate NLOS labels. We aim to contribute this dataset to the GNSS and deep learning communities, aspiring to establish it as the benchmark for NLOS classification and pseudorange bias correction. The datasets can be accessed at: https://github.com/ebhrz/KLTDataset.

Remote Sighted Assistance (RSA) systems provide visually impaired people (VIPs) with real-time guidance by connecting them with remote sighted agents to facilitate daily travel. However, unfamiliar environments often complicate decision-making for agents and can induce anxiety in VIPs, thereby reducing the effectiveness of the assistance provided. To address this challenge, this paper proposes a video-based map assistance method. By pre-recording pedestrian path videos and aligning them with geographic locations, the system enables route preview and enhances navigation guidance. This study introduces a factor graph optimization (FGO) algorithm that integrates Global Navigation Satellite System (GNSS) and pedestrian dead reckoning (PDR) data for pedestrian positioning. It incorporates road-anchor constraints, a turning-point-based anchor-matching method, and a coarse-to-fine optimization strategy to improve the positioning accuracy. GNSS provides global reference positions, PDR offers precise relative motion constraints through accurate heading estimation, and anchor factors further enhance localization accuracy by leveraging known geometric features. We collected data using a smartphone equipped with a four-camera module and conducted tests in representative urban environments. Experimental results demonstrate that the proposed anchor-aided FGO-GNSS/PDR algorithm achieves robust and accurate positioning, effectively supporting video-based map construction in complex urban settings. With anchor constraints, the mean horizontal positioning error was reduced by 42% to 65% and the maximum error by 38% to 76% across all datasets. In this study, the mean horizontal positioning error was 1.36 m.

Global Navigation Satellite Systems (GNSS) are crucial for intelligent transportation systems (ITS), providing essential positioning capabilities globally. However, in urban canyons, the GNSS performance could significantly degraded due to the blockage of direct GNSS signals. The pseudorange measurements are largely affected and the conventional model of weighting observations is not suitable in urban canyons. This paper addresses these challenges by integrating Artificial Intelligence (AI), specifically deep learning, into GNSS positioning process to enhance positioning accuracy. Traditional methods have primarily focused on pseudorange correction due to the absence of ground truth for weight estimation. In response, we propose an innovative indirect training approach using deep learning to optimize both pseudorange bias and weight estimation, aiming to minimize the positioning errors. To support this integration, we developed pyrtklib, a Python binding for the open-source RTKLIB tool, bridging the gap between traditional GNSS algorithms, typically developed in Fortran or C, and modern Python-based AI frameworks. Comparative analyses demonstrate that our method surpasses established tools like goGPS and RTKLIB in positioning accuracy, marking a significant advancement in the field. The source code of tightly coupled deep learning and GNSS integration, along with pyrtklib, is available on GitHub at https://github.com/ebhrz/TDL-GNSS and https://github.com/IPNL-POLYU/pyrtklib.

The recent surge in the development of autonomous vehicles has increased the need for reliable dynamic positioning of road agents in urban areas. Doppler frequency measurement of the global navigation satellite system (GNSS) can provide dynamic information and be used to estimate velocity. However, similar to pseudorange, the accuracy of Doppler frequency is degraded in dense urban areas, due to signal reflections from obstacles, resulting in substantial velocity errors. Existing methods tend to directly exclude non-line-of-sight (NLOS) Doppler frequency, which in turn leads to insufficient measurement numbers. 3D mapping aided (3DMA) GNSS ray-tracing method is commonly used to estimate extra delay from NLOS. The angle of arrival (AOA) of the reflected signal is obtained while tracing the propagation path, which can also be used to simulate NLOS Doppler frequency. Thus, this paper investigates the potential of using NLOS Doppler frequency as a feature for estimating velocity. A novel candidate-based 3DMA GNSS velocity estimation framework is proposed using Doppler frequency to examine the consistency between simulation on each candidate and measurement. Experimental results show NLOS Doppler frequency feature can enhance velocity estimation accuracy, reducing the root-mean-square error of velocity estimation by 46% from 1.06 m/s to 0.57 m/s in dense urban areas.

A sharp yet conservative overbound for heavy-tailed error distributions is essential in integrity monitoring applications due to availability and continuity constraints. This paper proposes the Principal Gaussian overbound (PGO) for heavy-tailed error distributions by leveraging the characteristics of the Gaussian mixture model. The overbounding property of the PGO is proved to be preserved through convolution, which makes it possible to derive pseudorange-level requirements from the position domain integrity requirements. Experimental results on two datasets show that the PGO provides the most competitive bounding performance for heavy-tailed differential global navigation satellite system (DGNSS) pseudorange errors when compared to the two-step Gaussian overbound and Gaussian-Pareto overbound, yielding a sharp bound in both the core and tail parts of the error distribution. The proposed method reduces the mean vertical protection level (VPL) by more than 78% compared to the two-step Gaussian overbounding method on the urban dataset. In addition, the mean computation time of VPL is only 0.08 _S with fifteen measurements by employing fast Fourier transforms, suggesting the substantial potential of the PGO in GNSS applications with strict integrity and real-time requirements. Furthermore, the feasibility of the PGO in fault detection is discussed.

Smooth and accurate global navigation satellite system (GNSS) positioning for pedestrians in urban canyons is still a challenge due to the multipath effects and the non-line-of-sight (NLOS) receptions caused by the reflections from surrounding buildings. Factor graph optimization (FGO) attracts more and more attention in GNSS society for improving urban GNSS positioning by effectively exploiting the measurement redundancy from historical information to resist the outlier measurements. Unfortunately, the FGO-based GNSS standalone positioning is still challenged in highly urbanized areas. As an extension of the previous FGO-based GNSS positioning method, the potential of the pedestrian dead reckoning (PDR) model in FGO to improve the GNSS standalone positioning performance in urban canyons is exploited in this paper. Specifically, the relative motion of the pedestrian is estimated based on the raw acceleration measurements from the onboard smartphone inertial measurement unit (IMU) via the PDR algorithm. Then the raw GNSS pseudorange, Doppler measurements, and relative motion from PDR are integrated using the FGO. Given the context of pedestrian navigation with a small acceleration most of the time, a novel soft motion model is proposed to smooth the states involved in the factor graph model. This paper verified the effectiveness of employing the PDR model in FGO step-by-step through two datasets collected in dense urban canyons of Hong Kong using smartphone-level GNSS receivers. The comparison between the conventional extended Kalman filter, several existing methods, and FGO-based integration is presented. The proposed method shows better results than the conventional FGO method in all test datasets, with at least a 22% decrease in the mean value of positioning error. The proposed method reduces the average localization error from 31.64 m to 18.51 m in a deep urban area.

Positioning in urban areas is still a challenge due to non-line-of-sight (NLOS) and multipath reception. This paper explores the geometrical characteristics of the GNSS ranging measurement by a graphical representation to better indicate the pseudorange consistency, which can be used to mitigate the NLOS and multipath receptions. The graphical representation is created by the grid-based method combined with the single differenced technique, which is called the single differenced residual map (SDRes Map). With the graphical properties of the SDRes Map, four main focuses of the NLOS/multipath problems, including positioning, signal status prediction, satellite weighting calculation, and NLOS/multipath error calculation, are able to be tackled simultaneously and demonstrated to have superior performance against the conventional or even state-of-the-art method methods.

The Global Navigation Satellite System (GNSS) is one of the most popular solutions to localize potential road cracks. Unfortunately, the accurate positioning of the GNSS in urban environments presents a significant challenge due to complex signal blockage and reflection phenomena. To tackle this, we propose a method that enhances GNSS positioning accuracy, particularly suited for the intricate layouts of urban canyons. Our approach integrates the prior map with lane line information into the Factor Graph Optimization (FGO) algorithm, effectively mitigating the impacts of multi-path effects and NLOS. However, the poor accuracy of the initial guess from the GNSS positioning can easily lead to incorrect lane matching which is one of the main challenges of lane matching in highly urbanized areas. To fill this gap, this paper proposes to use the switchable factor to model the potential incorrect lane matching by leveraging the redundancy of lane information across multi-epochs. This paper verified the effectiveness of this methodology using two datasets from the dense urban environment of Hong Kong, collected using the low-cost automobile level receiver, and compared the results with conventional methods. Our findings affirm that integrating the FGO-based GNSS positioning system with map information significantly boosts positioning accuracy, demonstrating the robustness of our approach.

Leveraging multiple sensors enhances complex environmental perception and increases resilience to varying luminance conditions and high-speed motion patterns, achieving precise localization and mapping. This paper proposes, ECMD, an event-centric multisensory dataset containing 81 sequences and covering over 200 km of various challenging driving scenarios including high-speed motion, repetitive scenarios, dynamic objects, etc. ECMD provides data from two sets of stereo event cameras with different resolutions (640×480, 346×260), stereo industrial cameras, an infrared camera, a top-installed mechanical LiDAR with two slanted LiDARs, two consumer-level GNSS receivers, and an onboard IMU. Meanwhile, the ground-truth of the vehicle was obtained using a centimeter-level high-accuracy GNSS-RTK/INS navigation system. All sensors are well-calibrated and temporally synchronized at the hardware level, with recording data simultaneously. We additionally evaluate several state-of-the-art SLAM algorithms for benchmarking visual and LiDAR SLAM and identifying their limitations. The dataset is available at https://arclab-hku.github.io/ecmd/ .