AHMF: Adaptive Hybrid-Memory-Fusion Model for Driver Attention Prediction
Preprint
Abstract
Accurate driver attention prediction can serve as a critical reference for intelligent vehicles in understanding traffic scenes and making informed driving decisions. Though existing studies on driver attention prediction improved performance by incorporating advanced saliency detection techniques, they overlooked the opportunity to achieve human-inspired prediction by analyzing driving tasks from a cognitive science perspective. During driving, drivers' working memory and long-term memory play crucial roles in scene comprehension and experience retrieval, respectively. Together, they form situational awareness, facilitating drivers to quickly understand the current traffic situation and make optimal decisions based on past driving experiences. To explicitly integrate these two types of memory, this paper proposes an Adaptive Hybrid-Memory-Fusion (AHMF) driver attention prediction model to achieve more human-like predictions. Specifically, the model first encodes information about specific hazardous stimuli in the current scene to form working memories. Then, it adaptively retrieves similar situational experiences from the long-term memory for final prediction. Utilizing domain adaptation techniques, the model performs parallel training across multiple datasets, thereby enriching the accumulated driving experience within the long-term memory module. Compared to existing models, our model demonstrates significant improvements across various metrics on multiple public datasets, proving the effectiveness of integrating hybrid memories in driver attention prediction.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- On the importance of working memory in the driving safety field: A systematic reviewAccident Analysis & Prevention, 2023
- A multimodal deep neural network for prediction of the driver’s focus of attention based on anthropomorphic attention mechanism and prior knowledgeExpert Systems with Applications, 2022
- Driving as well as on a Sunny Day? Predicting Driver's Fixation in Rainy Weather Conditions via a Dual-Branch Visual ModelIEEE/CAA Journal of Automatica Sinica, 2022
- Multisource Adaption for Driver Attention Prediction in Arbitrary Driving ScenesIEEE Transactions on Intelligent Transportation Systems, 2022
- DS-TransUNet: Dual Swin Transformer U-Net for Medical Image SegmentationIEEE Transactions on Instrumentation and Measurement, 2022
- A Dynamic Spatial-Temporal Attention Network for Early Anticipation of Traffic AccidentsIEEE Transactions on Intelligent Transportation Systems, 2022
- Working Memory Connections for LSTMNeural Networks, 2021
- U2-Net: Going deeper with nested U-structure for salient object detectionPattern Recognition, 2020
- How Do Drivers Allocate Their Potential Attention? Driving Fixation Prediction via Convolutional Neural NetworksIEEE Transactions on Intelligent Transportation Systems, 2019
- Where Does the Driver Look? Top-Down-Based Saliency Detection in a Traffic Driving EnvironmentIEEE Transactions on Intelligent Transportation Systems, 2016