Lunfeng Guo

Driver intention recognition is essential for optimizing driving decisions by dynamically adjusting speed and trajectory to enhance system performance. However, in the underground coal mine environment, traditional vision-based methods face significant limitations in accuracy and adaptability. To effectively improve the accuracy of vision-based driver intention recognition, this study introduces a novel approach leveraging cross-modal knowledge distillation (CMKD) to integrate electroencephalography (EEG) signals with video data to identify driver intentions in coal mining operations. By combining these modalities, the method capitalizes on their complementary strengths to achieve a more comprehensive understanding of driver intent. Experimental analysis across various models evaluates the performance of the proposed CMKD method, which integrates EEG signals with video data. Results reveal a substantial improvement in recognition accuracy over traditional machine vision-based approaches, with a maximum accuracy of 84.38%. This advancement enhances the reliability of driver intention detection and offers more robust support for decision making in automated mine transport systems.

In open-pit mining, autonomous trucks are essential for enhancing both safety and productivity. Object detection technology is critical to their smooth and secure operation, but training these models requires large amounts of high-quality annotated data representing various conditions. It is expensive and time-consuming to collect these data during open-pit mining due to the harsh environmental conditions. Simulation engines have emerged as an effective alternative, generating diverse labeled data to augment real-world datasets. However, discrepancies between simulated and real-world environments, often referred to as the Sim2Real domain shift, reduce model performance. This study addresses these challenges by presenting a novel semi-supervised domain adaptation for object detection (SSDA-OD) framework named Adamix, which is designed to reduce domain shift, enhance object detection, and minimize labeling costs. Adamix builds on a mean teacher architecture and introduces two key modules: progressive intermediate domain construction (PIDC) and warm-start adaptive pseudo-label (WSAPL). PIDC builds intermediate domains using a mixup strategy to reduce source domain bias and prevent overfitting, while WSAPL provides adaptive thresholds for pseudo-labeling, mitigating false and missed detections during training. When evaluated in a Sim2Real scenario, Adamix shows superior domain adaptation performance, achieving a higher mean average precision (mAP) compared with state-of-the-art methods, with 50% less labeled data required, achieved through active learning. The results demonstrate that Adamix significantly reduces dependence on costly real-world data collection, offering a more efficient solution for object detection in challenging open-pit mining environments.

Unsupervised Domain Adaptation for Object Detection (UDA-OD) aims to adapt a model trained on a labeled source domain to an unlabeled target domain, addressing challenges posed by domain shifts. However, existing methods often face significant challenges, particularly in detecting small objects and over-relying on classification confidence for pseudo-label selection, which often leads to inaccurate bounding box localization. To address these issues, we propose a novel UDA-OD framework that leverages scale consistency (SC) and Temporal Ensemble Pseudo-Label Selection (TEPLS) to enhance cross-domain robustness and detection performance. Specifically, we introduce Cross-Scale Prediction Consistency (CSPC) to enforce consistent detection across multiple resolutions, improving detection robustness for objects of varying scales. Additionally, we integrate Intra-Class Feature Consistency (ICFC), which employs contrastive learning to align feature representations within each class, further enhancing adaptation. To ensure high-quality pseudo-labels, TEPLS combines temporal localization stability with classification confidence, mitigating the impact of noisy predictions and improving both classification and localization accuracy. Extensive experiments on challenging benchmarks, including Cityscapes to Foggy Cityscapes, Sim10k to Cityscapes, and Virtual Mine to Actual Mine, demonstrate that our method achieves state-of-the-art performance, with notable improvements in small object detection and overall cross-domain robustness. These results highlight the effectiveness of our framework in addressing key limitations of existing UDA-OD approaches.

In auxiliary transportation within mines, accurately assessing the cognitive and response states of drivers is vital for ensuring safety and operational efficiency. This study investigates the effects of various vehicle interaction stimuli on the electroencephalography (EEG) signals of mine transport vehicle drivers, analyzing the cognitive and response states of drivers under different conditions to evaluate their impact on safety performance. Through experimental design, we simulate multiple scenarios encountered in real operations, including interactions with dynamic and static vehicles, personnel, and warning signs. EEG technology records brain signals during these scenarios, and data analysis reveals changes in the cognitive states and responses of drivers to different stimuli. The results indicate significant variations in EEG signals with interactions involving dynamic and static vehicles, personnel, and warning signs, reflecting shifts in the cognitive and response states of drivers. Additionally, the study examines the overall impact of different interaction objects and environments. The detailed analysis of EEG signals in different scenarios sheds light on changes in perception, attention, and responses related to drivers, which is critical for advancing safety and sustainability in mining operations.

Object detection for autonomous haulage trucks in open-pit mines is challenging due to unstructured environments. Collecting and annotating large-scale real-world images in these harsh conditions is extremely expensive. Inspired by parallel vision, virtual images have been used to supplement real-world data. However, models trained on virtual data exhibit poor performance in real mines due to the simulation-to-real (sim2real) domain shift. To address this issue, a human-in-the-loop algorithm called Active Parallel Teacher (APT) is introduced to select and annotate real images for sim2real object detection, in which the gradient information of a mean-teacher architecture is formulated as an important indicator for sample selection. Based on the gradient embeddings, three active sampling metrics are designed to measure sample importance from different perspectives including uncertainty, inter-domain difference, and diversity of target domain. Following that, an adaptive budget allocation strategy is presented to achieve the trade-off between uncertainty and difference along with different model training stages, with the purpose of maximizing the performance gain of human annotations. Finally, a Mean Teacher based domain adaptation process is developed to leverage fine-tuning and mix-training techniques to full utilize both labeled and unlabeled real data. Experimental results on the open-pit mine dataset demonstrate that our method can outperform other domain adaptation methods and attains 91% of the performance achieved under fully supervised conditions with only 7.5% of the real data labeled.