Hui Liu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Hui Liu has a total of 12 co-authors. Here are the top co-authors by number of shared publications:
- Chunsheng Liu
School of Control Science and Engineering, Shandong University, Jinan 250061, China
8 shared publications - Faliang Chang
School of Control Science and Engineering, Shandong University, Jinan 250061, China
8 shared publications - Yansha Lu
School of Control Science and Engineering, Shandong University, Jinan, China
5 shared publications - Minhang Liu
Shandong University, Jinan, China
2 shared publications - Yansha Lu
School of Control Science and Engineering, Shandong University, Jinan 250061, China
2 shared publications - Hengqiang Huan
School of Control Science and Engineering, Shandong University, Jinan, China
1 shared publication
Latest Publications
- Open Access
- 17 July 2026
- Published by MDPI AG in Biomimetics
- Vol. 11 (7) , 503
LDR-Net: Landmark-Guided Diverse Regional Representation Learning for Facial Expression Recognition
Reliable facial expression recognition (FER) is essential for human–computer interaction across scenarios. However, existing methods often overlook spatial structure when seeking region-specific features due to fixed or limited local regions, limiting representation capability under real-world variations. Inspired by the multi-location attention of the human visual system, we propose a Landmark-guided Diverse Regional Representation Network (LDR-Net), using dynamic landmarks as patch centers to preserve structural details while locating expression-critical areas, facilitating effective regional representations for FER. First, a novel Diverse Regional Feature Extraction (DRFE) module operates via complementary operations: landmark-guided cropping for local details and cross-level integration with feature reorganization for holistic aggregation. Second, a novel Diverse Representation Learning (DRL) module is proposed with a collaborative dual-stream mechanism that captures fine-grained local dependencies via Transformers while reinforcing global features through attention-based enhancement, enabling comprehensive feature learning. Finally, a new Hybrid Feature Fusion (HFF) module is proposed for joint decision optimization via a hierarchical hybrid strategy, which aggregates intra-branch predictions followed by weighted branch-level fusion. Experiments on three FER benchmarks (RAF-DB, AffectNet, SFEW) and five occlusion/pose test sets demonstrate that LDR-Net outperforms state-of-the-art methods, while cross-scene validation on KMU-FED confirms its effectiveness in real-world driving.
- 17 July 2026
- Published by MDPI AG in Biomimetics
- Vol. 11 (7) , 503
Open AccessLDR-Net: Landmark-Guided Diverse Regional Representation Learning for Facial Expression Recognition
Reliable facial expression recognition (FER) is essential for human–computer interaction across scenarios. However, existing methods often overlook spatial structure when seeking region-specific features due to fixed or limited local regions, limiting representation capability under real-world variations. Inspired by the multi-location attention of the human visual system, we propose a Landmark-guided Diverse Regional Representation Network (LDR-Net), using dynamic landmarks as patch centers to preserve structural details while locating expression-critical areas, facilitating effective regional representations for FER. First, a novel Diverse Regional Feature Extraction (DRFE) module operates via complementary operations: landmark-guided cropping for local details and cross-level integration with feature reorganization for holistic aggregation. Second, a novel Diverse Representation Learning (DRL) module is proposed with a collaborative dual-stream mechanism that captures fine-grained local dependencies via Transformers while reinforcing global features through attention-based enhancement, enabling comprehensive feature learning. Finally, a new Hybrid Feature Fusion (HFF) module is proposed for joint decision optimization via a hierarchical hybrid strategy, which aggregates intra-branch predictions followed by weighted branch-level fusion. Experiments on three FER benchmarks (RAF-DB, AffectNet, SFEW) and five occlusion/pose test sets demonstrate that LDR-Net outperforms state-of-the-art methods, while cross-scene validation on KMU-FED confirms its effectiveness in real-world driving.
- 3 March 2026
- Published by Springer Nature in Signal, Image and Video Processing
- Vol. 20 (3) , 129
SSAH-Net: Semantic-and-Skeleton Attention Hybrid Network for driver action recognition across illumination
- 3 March 2026
- Published by Springer Nature in Signal, Image and Video Processing
- Vol. 20 (3) , 129
SSAH-Net: Semantic-and-Skeleton Attention Hybrid Network for driver action recognition across illumination
- 21 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Intelligent Systems
- Vol. 40 (5) , 25-33
Semantic Map Construction Under Complex Weather Scenarios
Semantic maps provide information on road elements, which is crucial for ensuring driving safety. However, previous methods mostly focus on normal weather conditions, neglecting the challenges of scene feature extraction caused by image degradation in complex weather scenarios. To address these challenges, we propose a Normal weather scene-guided Complex weather scene Map Construction Network (NCMC-Net). Firstly, we propose a scene Feature Sharing Module (FSM) based on the complement learning mechanism. This module alternately updates the shared feature pool with features from normal and complex weather scenes during two training stages. Secondly, we design a Feature Refinement Module (FRM) based on the channel attention mechanism to adjust the intensity of multi-channel features selectively. The two modules facilitate the extraction of optimal complex weather scene features guided by features from normal weather scenes. Experimental results on the complex weather scene dataset nuScenes-W validate our model’s performance.
- 21 April 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Intelligent Systems
- Vol. 40 (5) , 25-33
Semantic Map Construction Under Complex Weather Scenarios
Semantic maps provide information on road elements, which is crucial for ensuring driving safety. However, previous methods mostly focus on normal weather conditions, neglecting the challenges of scene feature extraction caused by image degradation in complex weather scenarios. To address these challenges, we propose a Normal weather scene-guided Complex weather scene Map Construction Network (NCMC-Net). Firstly, we propose a scene Feature Sharing Module (FSM) based on the complement learning mechanism. This module alternately updates the shared feature pool with features from normal and complex weather scenes during two training stages. Secondly, we design a Feature Refinement Module (FRM) based on the channel attention mechanism to adjust the intensity of multi-channel features selectively. The two modules facilitate the extraction of optimal complex weather scene features guided by features from normal weather scenes. Experimental results on the complex weather scene dataset nuScenes-W validate our model’s performance.
- 30 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (7) , 6456-6463
Camera-Based Online Vectorized HD Map Construction With Incomplete Observation
Camera-based online map construction focuses on learning map elements from surround-view images. Distinguished with previous methods that rely on complete observations, we explore a new map construction problem under incomplete observations where one or more perspectives of the surround-view are missing due to camera damage or occlusion. Incomplete observations lead to inferior performance and may even result in failure. Map construction based on incomplete observations faces two challenges: supplementing missing perspective features and reducing the complexity of high-dimensional feature learning. To address these issues, we propose a novel Panoramic Observation Prior Network (POP-Net). Firstly, based on the observation switch training mechanism, we propose a Panoramic Learning Module (PL-Module). It establishes a learnable panoramic feature space, facilitating the extraction of panoramic features from incomplete observations, thus supplementing missing perspective features. Secondly, based on the feature decomposition mechanism, we design a Panoramic Decomposition-Aggregation Operation (PDA-Operation), which decomposes high-dimensional panoramic features into low-dimensional local scene features. This allows limited local scene features to represent diverse panoramic features, alleviating computational and memory burdens of high-dimensional feature learning. Experimental results demonstrate that our method surpasses existing approaches under incomplete observation scenarios.
- 30 May 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (7) , 6456-6463
Camera-Based Online Vectorized HD Map Construction With Incomplete Observation
Camera-based online map construction focuses on learning map elements from surround-view images. Distinguished with previous methods that rely on complete observations, we explore a new map construction problem under incomplete observations where one or more perspectives of the surround-view are missing due to camera damage or occlusion. Incomplete observations lead to inferior performance and may even result in failure. Map construction based on incomplete observations faces two challenges: supplementing missing perspective features and reducing the complexity of high-dimensional feature learning. To address these issues, we propose a novel Panoramic Observation Prior Network (POP-Net). Firstly, based on the observation switch training mechanism, we propose a Panoramic Learning Module (PL-Module). It establishes a learnable panoramic feature space, facilitating the extraction of panoramic features from incomplete observations, thus supplementing missing perspective features. Secondly, based on the feature decomposition mechanism, we design a Panoramic Decomposition-Aggregation Operation (PDA-Operation), which decomposes high-dimensional panoramic features into low-dimensional local scene features. This allows limited local scene features to represent diverse panoramic features, alleviating computational and memory burdens of high-dimensional feature learning. Experimental results demonstrate that our method surpasses existing approaches under incomplete observation scenarios.
- 25 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 25 (10) , 13694-13705
Egocentric Vulnerable Road Users Trajectory Prediction With Incomplete Observation
Vulnerable Road Users (VRUs) trajectory prediction aims to analyze the future movements of pedestrians and cyclists for intelligent driving. Most previous methods just focus on VRUs trajectory prediction using idealized complete observations, and rare consider occlusions and tracking losses. Focus on the incomplete observation problem, we propose a novel Observation Store and Query Fusion Network (OSQF-Net), for VRUs trajectory prediction with incomplete observation. Firstly, based on the external memory bank mechanism and complete-incomplete observation joint training strategy, a Memory Bank-based Feature Store and Query Module (MSQ-Module) is proposed to extract complete motion features, from disrupted motion patterns caused by incomplete observation. Subsequently, based on temporal extraction and attention mechanism, a Spatio-Temporal Fusion Module (STF-Module) is proposed to effectively fuse the pseudo-complete motion features and incomplete motion features in both spatial and temporal dimensions. Finally, with these two modules and a CVAE network, the OSQF-Net can generate a latent space with complete motion patterns, which guides future trajectory prediction. Experimental results demonstrate that OSQF-Net achieves superior prediction performance and real-time inference capability for egocentric VRUs trajectory prediction, under both complete and incomplete observation scenarios.
- 25 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 25 (10) , 13694-13705
Egocentric Vulnerable Road Users Trajectory Prediction With Incomplete Observation
Vulnerable Road Users (VRUs) trajectory prediction aims to analyze the future movements of pedestrians and cyclists for intelligent driving. Most previous methods just focus on VRUs trajectory prediction using idealized complete observations, and rare consider occlusions and tracking losses. Focus on the incomplete observation problem, we propose a novel Observation Store and Query Fusion Network (OSQF-Net), for VRUs trajectory prediction with incomplete observation. Firstly, based on the external memory bank mechanism and complete-incomplete observation joint training strategy, a Memory Bank-based Feature Store and Query Module (MSQ-Module) is proposed to extract complete motion features, from disrupted motion patterns caused by incomplete observation. Subsequently, based on temporal extraction and attention mechanism, a Spatio-Temporal Fusion Module (STF-Module) is proposed to effectively fuse the pseudo-complete motion features and incomplete motion features in both spatial and temporal dimensions. Finally, with these two modules and a CVAE network, the OSQF-Net can generate a latent space with complete motion patterns, which guides future trajectory prediction. Experimental results demonstrate that OSQF-Net achieves superior prediction performance and real-time inference capability for egocentric VRUs trajectory prediction, under both complete and incomplete observation scenarios.
- 22 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 24 (11) , 11850-11863
JHPFA-Net: Joint Head Pose and Facial Action Network for Driver Yawning Detection Across Arbitrary Poses in Videos
Yawning detection is a key means in driver fatigue detection, which suffers difficulties including head poses, facial expressions, illumination variations, occlusions, etc. Yet, most previous methods mainly focus on frontal faces, and are deficient to deal with different facial actions under arbitrary poses in the actual driving environment. In this study, we propose a novel Joint Head Pose and Facial Action Network (JHPFA-Net) for driver yawning detection across arbitrary poses in videos, with three main parts including a Geometric-based Key-frame Selection Module (GK-Module), a Face Frontalization with Warp Attention Module (FF-Module) and a dual-channel classifier for Head Pose $\&$ Facial Action Fusion Module (HF-Module). Firstly, the GK-Module is proposed to extract geometric vectors and to construct a two-stage judgment mechanism, with the purpose of dealing with frame redundancy and improving the efficiency of JHPFA-Net structure. Secondly, distinguished with existing methods, the FF-Module is proposed to synthesize photo-realistic frontal faces, which can be used for capturing the facial actions under arbitrary poses. Finally, the HF-Module is proposed to fuse head pose attributes and facial modalities together, for the purpose of achieving pose-invariant detection and improving accuracy. Extensive experiments show that the proposed JHPFA-Net achieves state-of-the-art results comparing with some representative methods on the public YawDD benchmark, and it performs well in real-time application.
- 22 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 24 (11) , 11850-11863
JHPFA-Net: Joint Head Pose and Facial Action Network for Driver Yawning Detection Across Arbitrary Poses in Videos
Yawning detection is a key means in driver fatigue detection, which suffers difficulties including head poses, facial expressions, illumination variations, occlusions, etc. Yet, most previous methods mainly focus on frontal faces, and are deficient to deal with different facial actions under arbitrary poses in the actual driving environment. In this study, we propose a novel Joint Head Pose and Facial Action Network (JHPFA-Net) for driver yawning detection across arbitrary poses in videos, with three main parts including a Geometric-based Key-frame Selection Module (GK-Module), a Face Frontalization with Warp Attention Module (FF-Module) and a dual-channel classifier for Head Pose $\&$ Facial Action Fusion Module (HF-Module). Firstly, the GK-Module is proposed to extract geometric vectors and to construct a two-stage judgment mechanism, with the purpose of dealing with frame redundancy and improving the efficiency of JHPFA-Net structure. Secondly, distinguished with existing methods, the FF-Module is proposed to synthesize photo-realistic frontal faces, which can be used for capturing the facial actions under arbitrary poses. Finally, the HF-Module is proposed to fuse head pose attributes and facial modalities together, for the purpose of achieving pose-invariant detection and improving accuracy. Extensive experiments show that the proposed JHPFA-Net achieves state-of-the-art results comparing with some representative methods on the public YawDD benchmark, and it performs well in real-time application.
- 14 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 23 (10) , 18695-18706
Adaptive Short-Temporal Induced Aware Fusion Network for Predicting Attention Regions Like a Driver
Driver attention prediction can solve the problem of ‘Where should the driver pay attention?’, Most previous methods are designed to predict regional attention with redundant regions. Furthermore, popular spatial-temporal feature extraction networks such as ConvLSTM and 3D-CNN are difficult to achieve real-time. To overcome these difficulties, we propose an Adaptive Short-temporal Induced Aware Fusion Network (ASIAF-Net) for region-level and object-level driver attention prediction. 1 In ASIAF-Net, we design an Attention Related Spatial Feature Encoder (AF-Encoder) and an Induced Aware Fusion Network (IAF-Net) as the main network; with an Association Analysis Cell (AAC), the AF-Encoder makes it possible to effectively capture the relationship information of different objects. Considering most vital visual cues from moving objects, we propose a Self-adaptive Short-temporal Feature Extraction Module (SSFE-Module) to obtain inter-frame motion features. In IAF-Net, a Multi-scale Driver Attention Region Prediction Branch is designed to predict the regional attention, and an Object Saliency Estimation Branch is proposed to fuse the perception results and the regional attention map to estimate the object-level attention. Experiments show that the proposed ASIAF-Net can predict driver’s attention on regions and objects more robustly and precisely than state-of-the-art methods on three datasets, and that it achieves real-time on our ADAS platform.
- 14 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 23 (10) , 18695-18706
Adaptive Short-Temporal Induced Aware Fusion Network for Predicting Attention Regions Like a Driver
Driver attention prediction can solve the problem of ‘Where should the driver pay attention?’, Most previous methods are designed to predict regional attention with redundant regions. Furthermore, popular spatial-temporal feature extraction networks such as ConvLSTM and 3D-CNN are difficult to achieve real-time. To overcome these difficulties, we propose an Adaptive Short-temporal Induced Aware Fusion Network (ASIAF-Net) for region-level and object-level driver attention prediction. 1 In ASIAF-Net, we design an Attention Related Spatial Feature Encoder (AF-Encoder) and an Induced Aware Fusion Network (IAF-Net) as the main network; with an Association Analysis Cell (AAC), the AF-Encoder makes it possible to effectively capture the relationship information of different objects. Considering most vital visual cues from moving objects, we propose a Self-adaptive Short-temporal Feature Extraction Module (SSFE-Module) to obtain inter-frame motion features. In IAF-Net, a Multi-scale Driver Attention Region Prediction Branch is designed to predict the regional attention, and an Object Saliency Estimation Branch is proposed to fuse the perception results and the regional attention map to estimate the object-level attention. Experiments show that the proposed ASIAF-Net can predict driver’s attention on regions and objects more robustly and precisely than state-of-the-art methods on three datasets, and that it achieves real-time on our ADAS platform.
- 1 November 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Consumer Electronics
- Vol. 67 (4) , 275-284
Driver Gaze Zone Estimation via Head Pose Fusion Assisted Supervision and Eye Region Weighted Encoding
Driver gaze zone estimation is an important task in Advanced Driver Assistance Systems (ADAS), which suffers difficulties including head pose, capture direction, glass occlusion, and real-time requirement, etc. Most previous methods combine face modalities and head pose using concat process, which may result in over-fitting due to the unbalanced dimension. Focusing on gaze zone estimation problems, we propose the Head Pose Fusion Assisted supervision & Eye Region Weighted Encoding ( HP-ERW ) structure, which fuses head pose attribute and face modalities together through spatial attention and Kronecker product mechanisms. Firstly, we introduce a pre-processing module dealing with head pose and face information, with the purpose of extracting input vectors and improving the fusion speed of the HP-ERW structure. Secondly, an Eye Region Weighted Encoding Network (ERW-Net) based on spatial attention is proposed to strengthen the networks perception ability for encoding features. Finally, we propose a dual-channel Head Pose Fusion Network ( HP-Net ) based on the Kronecker product mechanism, with the purpose of fusing head pose and improving the estimation accuracy. Experiments show that the HP-ERW outperforms compared existing methods on several public datasets. The designed ADAS using the proposed method achieves 23.5 fps real-time application with small memory requirement of 4,884 KB.
- 1 November 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Consumer Electronics
- Vol. 67 (4) , 275-284
Driver Gaze Zone Estimation via Head Pose Fusion Assisted Supervision and Eye Region Weighted Encoding
Driver gaze zone estimation is an important task in Advanced Driver Assistance Systems (ADAS), which suffers difficulties including head pose, capture direction, glass occlusion, and real-time requirement, etc. Most previous methods combine face modalities and head pose using concat process, which may result in over-fitting due to the unbalanced dimension. Focusing on gaze zone estimation problems, we propose the Head Pose Fusion Assisted supervision & Eye Region Weighted Encoding ( HP-ERW ) structure, which fuses head pose attribute and face modalities together through spatial attention and Kronecker product mechanisms. Firstly, we introduce a pre-processing module dealing with head pose and face information, with the purpose of extracting input vectors and improving the fusion speed of the HP-ERW structure. Secondly, an Eye Region Weighted Encoding Network (ERW-Net) based on spatial attention is proposed to strengthen the networks perception ability for encoding features. Finally, we propose a dual-channel Head Pose Fusion Network ( HP-Net ) based on the Kronecker product mechanism, with the purpose of fusing head pose and improving the estimation accuracy. Experiments show that the HP-ERW outperforms compared existing methods on several public datasets. The designed ADAS using the proposed method achieves 23.5 fps real-time application with small memory requirement of 4,884 KB.