Xuan Xiao

Accurately tracking the location of vehicles in real-time is crucial for mobile navigation services. However, detecting that the vehicle is off-route can be challenging when drivers stray from the pre-planned route. Traditional GNSS-based solutions are often unreliable due to noisy signals, leading to erratic location errors and delayed alarms when the vehicle enters an off-route area. This paper presents a novel off-route detection framework, ForkTracker, which combines image, Inertial Measurement Unit (IMU), and Global Navigation Satellite System (GNSS) information to track vehicles and predict whether they will deviate. Specifically, a lightweight deep learning model is used to extract driving direction from images, and inertial readings are utilized for event detection, such as lane shifts. Finally, the multi-source data is fused through a Sequential Monte Carlo framework to produce the vehicle's real-time position. We have implemented a prototype and conducted experiments on real-world data. The results show that our system can effectively predict the driving direction before the off-route occurs, with a proportion of 97.3% in advance and 14.1 m on average.

While location awareness is common outdoors due to GNSS systems and devices, pedestrians are back into darkness indoors such as in underground parking lots. We often forget where we parked the car and get confused by such maze-like structure. In order to help drivers find their cars without any additional equipment and map support, we propose a car-finding navigation system that only relies on smartphones. It automatically identifies the user’s drop-off point, records the walking trajectory from car’s location to the exit, and provides fine-grained return navigation to help user back to the car. To address the accuracy and diversity of pedestrian tracking, we propose an inertial sequence learning framework based on outdoor crowdsourced trajectories, which avoids the dedicated efforts on groundtruth collection for model training. We also explore a smartphone posture detection method that supports multiple placements of the smartphone, and a SVM-based trajectory refinement algorithm with only inertial readings. Besides, we explore a particle filter framework to track and navigate the pedestrian in real time. We have developed a prototype and conducted a series of experiments in multiple underground parking lots, and the results have demonstrated our effectiveness compared with the state-of-the-art.

The lack of indoor floor plans is one of the major obstacles to ubiquitous indoor location-based services. Dedicated mobile robots with high-precision sensors can measure and produce accurate indoor maps, but the deployment remains low for the public. Computer vision techniques are adopted by some existing smartphone-based methods to build the 3D point cloud, which have the cost of a quantity of the efforts of image collection and the risk of privacy issues. In this paper, we propose BatMapper-Plus which adopt acoustic ranging and inertial tracking to construct precise and complete indoor floor plans on smartphones. It emits acoustic signals to measure the distance from the smartphone to a neighbouring wall segment, and produces accessible areas by surrounding the building during walking. It also refines the constructed indoor floor plan to eliminate scattered segments, and identifies connection areas, including stairs and elevators among different floors. In addition, we propose an LSTM-based dead-reckoning model which is trained by outdoor IMU readings and GPS records, and use it to infer the step length during indoor walking, thereby improving the floor plan quality. We also elaborate how to use the constructed map for indoor navigation, i.e., a Dynamic Time Warping algorithm which automatically matches current inertial readings and historical sensory data during map construction to produce fine-grained walking guidance. To show our effectiveness compared with the state-of-the-art, we carry out extensive experiments in a teaching building and a residential building. It proves that our method is efficient without any privacy concerns and texture/illumination limitations.

Visual-based unsupervised learning [1] – [3] has emerged as a promising approach in estimating monocular depth and ego-motion, avoiding intensive efforts on collecting and labeling the ground truth. However, they are still restrained by the brightness constancy assumption among video sequences, especially susceptible with frequent illumination variations or nearby textureless surroundings in indoor environments. In this article, we selectively combine the complementary strength of visual and inertial measurements, i.e., videos extract static and distinct features while inertial readings depict scale-consistent and environment-agnostic movements, and propose a novel unsupervised learning framework to predict both monocular depth and ego-motion trajectory simultaneously. This challenging task is solved by learning both forward and backward inertial sequences to eliminate inevitable noises, and reweighting visual and inertial features via gated neural networks in various environments or with user-specific moving dynamics. In addition, we also employ structure cues to produce scene depths from a single image and explore structure consistency constraints to calibrate the depth estimates in indoor buildings. Experiments on the outdoor KITTI data set and our dedicated indoor prototype reveal that our approach consistently outperforms the state of the art on both depth and ego-motion estimates. To the best of our knowledge, this is the first work to fuse visual and inertial data without any supervision signals for monocular depth and ego-motion estimation, and our solution remain effective and robust even in textureless indoor scenarios.

Decorating speed bumps is a common and effective measure to compulsively reduce vehicular velocity in dangerous regions, especially at tunnels, slopes, and parking lots. Despite a decade of deployment, the bump information is still sporadic in map-based applications, for example, driving alert ahead of bumps. One major obstacle is the lack of an up-to-date bump database, thus service providers have to hire dedicated personnel to gather and periodically calibrate such data, which is effort-intensive and time-consuming to large-scale coverage. In this article, we propose a personal bump seeker (PBS), a novel mobile crowdsensing application to automatically identify and update speed bumps in urban cities. Specifically, we formulate bump detection as a regression model, thus improving the robustness of inertial patterns on temporal domain. We also explore a leader-follower mechanism that automatically extracts the bump signal for training and inference in individual smartphones, regardless of different smartphones, vehicles, and driving habits during crowdsensing. Finally, we implement a prototype and conduct extensive experiments on large-scale real-world traffic datasets collected by the DiDi platform, and the results demonstrate our effectiveness in producing a city-level bump database.

Although vehicle location-based services are prevalent outdoors, we are back into darkness in many GPS blocked environments, such as tunnels, indoor parking garages, and multilevel flyovers. Existing smartphone-based solutions usually adopt inertial dead reckoning to infer the trajectory, but low-quality inertial sensors in phones are plagued by heavy noises, causing unbounded localization errors through double integrations for movements. In this article, we propose VeTorch, a smartphone inertial odometry that devises an inertial sequence learning framework to track vehicles in real time when GPS signal is not available. Specifically, we transform the inertial dynamics from the phone to the vehicle regardless of the arbitrary phone's placement in the car and explore a temporal convolutional network to learn the vehicle's moving dependencies directly from the inertial data. To tackle the heterogeneous smartphone properties and driving habits, we propose a federated learning-based active model training mechanism to produce customized models for individual smartphones, without incurring user privacy issues. We implement a highly efficient prototype and conduct extensive experiments on two large-scale real-world traffic data sets collected by a modern ride-hailing platform. Our results outperform the state-of-the-art vehicular inertial dead-reckoning solutions on both accuracy and efficiency.