Looking for a better fit? An Incremental Learning Multimodal Object Referencing Framework adapting to Individual Drivers
Preprint
Abstract
The rapid advancement of the automotive industry towards automated and semi-automated vehicles has rendered traditional methods of vehicle interaction, such as touch-based and voice command systems, inadequate for a widening range of non-driving related tasks, such as referencing objects outside of the vehicle. Consequently, research has shifted toward gestural input (e.g., hand, gaze, and head pose gestures) as a more suitable mode of interaction during driving. However, due to the dynamic nature of driving and individual variation, there are significant differences in drivers' gestural input performance. While, in theory, this inherent variability could be moderated by substantial data-driven machine learning models, prevalent methodologies lean towards constrained, single-instance trained models for object referencing. These models show a limited capacity to continuously adapt to the divergent behaviors of individual drivers and the variety of driving scenarios. To address this, we propose \textit{IcRegress}, a novel regression-based incremental learning approach that adapts to changing behavior and the unique characteristics of drivers engaged in the dual task of driving and referencing objects. We suggest a more personalized and adaptable solution for multimodal gestural interfaces, employing continuous lifelong learning to enhance driver experience, safety, and convenience. Our approach was evaluated using an outside-the-vehicle object referencing use case, highlighting the superiority of the incremental learning models adapted over a single trained model across various driver traits such as handedness, driving experience, and numerous driving conditions. Finally, to facilitate reproducibility, ease deployment, and promote further research, we offer our approach as an open-source framework at \url{https://github.com/amrgomaaelhady/IcRegress}.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Towards Adaptive User-centered Neuro-symbolic Learning for Multimodal Interaction with Autonomous Systems2023
- It’s all about you: Personalized in-Vehicle Gesture Recognition with a Time-of-Flight Camera2023
- Intelligent In‐Vehicle Interaction TechnologiesAdvanced Intelligent Systems, 2021
- A continual learning survey: Defying forgetting in classification tasksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2021
- How to Increase Automated Vehicles’ Acceptance through In-Vehicle Interaction Design: A ReviewInternational Journal of Human–Computer Interaction, 2021
- Deep 3D Object Detection Networks Using LiDAR Data: A ReviewIEEE Sensors Journal, 2020
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences of the United States of America, 2017
- ModDrop: Adaptive Multi-Modal Gesture RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Driver Gaze Tracking and Eyes Off the Road Detection SystemIEEE Transactions on Intelligent Transportation Systems, 2015
- Adaptive Generation in Dialogue Systems Using Dynamic User ModelingComputational Linguistics, 2014