Tao Huang

Accurate driver intention prediction is essential for intelligent vehicle decision-making. Existing methods typically model in-cabin behavior or external traffic context independently, overlooking their dynamic interaction. This study introduces InOut-IntentNet, a bidirectional cross-modal fusion framework that jointly models driver behaviors and environmental interactions. The framework includes: (1) an Inside Behavior Capture Module extracting driving-operation and head-pose features via 3D ResNet and Transformer encoders; (2) an Outside Environment Encoding Module modeling spatiotemporal interactions and lane semantics based on stacked graph convolutions; (3) a Bidirectional Cross-Modal Fusion Module aligning in-cabin and external cues using cross-attention and gated fusion; and (4) an Intention Prediction Module trained with class-balanced loss. Experiments on the Brain4Cars show that InOut-IntentNet achieves 89.77% accuracy and 89.20% F1-score, outperforming state-of-the-art methods across early prediction intervals. In-cabin cues provide stronger intention signals, while fusion with external context improves robustness and class-wise reliability. Cross-dataset evaluation on DAAD further confirms strong generalization and real-time efficiency.

The human-vehicle collaboration in automated vehicles is an effective transitional means to overcome the difficulty of rapidly transitioning to a highly automated level of intelligence. Furthermore, it can fully leverage the strengths of both drivers and autonomous driving systems, embodying a design philosophy of human-centered. Therefore, this paper provides a review and perspectives of the human-vehicle collaboration for automated vehicles. First, the concept, forms and methods of human-vehicle collaboration are reviewed. Then, a human-vehicle mutual trust collaboration framework based on complementary advantages of humans and vehicles and brain-like intelligence is proposed. Specifically, the framework focuses on driver behavior understanding and brain-like cognitive decision planning. After that, the methods of driver behavior understanding and brain-like cognitive decision planning are summarized. Finally, challenges and future works are analyzed to contribute the develop of understandable, trustable, and acceptable human-vehicle collaboration systems.

The task of making lane change decisions for autonomous vehicles in mixed traffic is intricate and challenging due to the uncertainty of surrounding vehicles. The uncertainty exists in terms of the diverse social driving preferences and unpredictable driving behavior of human drivers. To address these challenges, the decision-making process for changing lanes is represented as an incomplete information game, where the driver characteristics of surrounding vehicles are unknown during the interaction. To eliminate the uncertainty of the driving environment, the concept of driver aggressiveness is proposed to quantify the social driving preferences based on the Risk-Response (R-R) diagram in an explainable manner. Then the predicted trajectory is utilized to calculate the driving risks using Gaussian Mixture Model (GMM) that is trained by the naturalistic driving data in the interactive lane change scenarios extracted from the highD dataset. To make the simulation environment more diverse and realistic, the data-driven motion model social Intelligent Driver Model (SIDM) is constructed based on car-following data obtained from cut-in scenarios in the highD dataset. The simulations are conducted by setting up the environment vehicles equipped with SIDM model with diverse social driving preferences. The findings indicate that the proposed decision-making model can recognize the category of surrounding vehicles, and in realistic interactive driving scenarios, it can produce adaptive and human-like driving decisions.

Real-time driver behavior detection (DBD) is essential for developing driver-centered human-vehicle co-driving systems. This paper proposes an accurate and easily implementable DBD method based on driver behavior images and deep learning. Convolutional neural networks have difficulty dealing with changes in driver behavior images (rotation, scale, and translation). They suffer from a tradeoff between accuracy and number of trainable parameters, noise interference in images of driver behavior, and the black-box problem of neural networks, limiting the effectiveness of current DBD methods. Therefore, we design a novel deep learning-based DBD method consisting of a deep deformable inverted residual network with an attention mechanism. Deformable convolution is used to deal with image rotation and translation. An inverted residual block and linear bottleneck approach based on depthwise separable convolution is used to reduce the number of trainable parameters while maintaining high accuracy. An attention mechanism based on soft thresholding is incorporated into the nonlinear transformation layers to extract driver behavior-relevant features. We also innovatively propose a visualization method to improve the interpretability of the proposed method. The experimental results show that the proposed method can accurately detect driver behavior (95.17% mAP on the Kaggle driving test dataset), significantly outperforming state-of-the-art methods regarding accuracy, real-time performance, and reliability.

Effective driver distraction detection (DDD) can significantly improve driving safety. Inspired by the definition of driver distraction, this work aims to detect driver distraction based on the true driver’s focus of attention (TDFoA). This is accomplished based on two stages, the prediction of TDFoA and DDD based on the driver’s focus of attention (DFoA) and the TDFoA. However, this process is challenging due to the complex and dynamic traffic environment, momentary glimpses not related to the TDFoA in normal driving, the interference of noise (information unrelated to the TDFoA), the ignoring of multi-scale information from driving scenarios, and the variability of thresholds used to distinguish between driver distraction and non-distraction. To solve these problems, we design a deep 3D residual network with attention mechanism and encoder-decoder (D3DRN-AMED) based on successive frames with convolution LSTM for the prediction of TDFoA, where successive frames are used to eliminate the impact of momentary glimpses. The convolution LSTM achieves the transition of features in successive frames, considering the historical variation of driving scenarios. Attention mechanism based on soft thresholding is inserted into the D3DRN-AMED as nonlinear transformation layers to eliminate the noise-related features, and the encoder-decoder module is introduced to extract multi-scale features. A method based on a neural network is then proposed to detect driver distraction according to the difference or similarity between DFoA and TDFoA, which can accurately detect driver distraction without determining the threshold. Experimental results show that the proposed method can effectively detect driver distraction.