A wearable obstacle avoidance device for visually impaired individuals with cross-modal learning
Top Cited Papers
Open Access
- 24 March 2025
- journal article
- research article
- Published by Springer Nature in Nature Communications
- Vol. 16 (1) , 1-17
Abstract
It is challenging for wearable obstacle avoidance devices to simultaneously meet practical demands of high reliability, rapid response, long-lasting duration, and usable design. Here we report a wearable obstacle avoidance device, comprising a set of self-developed glasses (weighing ~400 grams, including an ~80 grams battery) and a common smartphone. Specifically, the glasses collect the multi-modal data for comprehensive environmental perception, including video and depth modalities, and implement a depth-aided video compression module. This module not only adaptively compresses video data to reduce transmission delay to the smartphone, but also operates on a customized FPGA board featuring a multi float-point vector unit streaming processing architecture, thereby facilitating responsive and energy-efficient obstacle detection. Additionally, we design a cross-modal obstacle detection module on the smartphone, which ensures reliable detection and provides user-friendly auditory and tactile alerts by utilizing cross-modal learning based on modal correlations. Multiple indoor and outdoor experimental results demonstrate 100% collision avoidance rates, delay of less than 320 ms, and duration of approximately 11 hours.Keywords
Funding Information
- National Natural Science Foundation of China (62427802, 62231017, 62301269, 62122094)
This publication has 41 references indexed in Scilit:
- Low vision rehabilitation for better quality of life in visually impaired adultsCochrane Database of Systematic Reviews, 2020
- Flexible Electronics toward Wearable SensingAccounts of Chemical Research, 2019
- Safe Local Navigation for Visually Impaired Users With a Time-of-Flight and Haptic Feedback DeviceIEEE Transactions on Neural Systems and Rehabilitation Engineering, 2018
- Lights from the Dark: Neural Responses from a Blind Visual HemifieldFrontiers in Neuroscience, 2017
- Sensor-Based Assistive Devices for Visually-Impaired People: Current Status, Challenges, and Future DirectionsSensors, 2017
- When Ultrasonic Sensors and Computer Vision Join Forces for Efficient Obstacle Detection and RecognitionSensors, 2016
- Influence of Delay Period Duration on Inhibitory Processes for Response PreparationCerebral Cortex, 2015
- Dynamic Vibrotactile Signals for Forward Collision Avoidance Warning SystemsHuman Factors: The Journal of the Human Factors and Ergonomics Society, 2014
- Quantifying Information Transfer Through a Head-Attached Vibrotactile Display: Principles for Design and ControlIEEE Transactions on Biomedical Engineering, 2012
- The efficiency and walking speed of visually impaired peopleErgonomics, 1986