Zachary David Asher

The accurate identification of look-alike medical vials is essential for patient safety, particularly when similar vials contain different substances, volumes, or concentrations. Traditional methods, such as manual selection or barcode-based identification, are prone to human error or face reliability issues under varying lighting conditions. This study addresses these challenges by introducing a real-time deep learning-based vial identification system, leveraging a Lightweight YOLOv4 model optimized for edge devices. The system is integrated into a Mixed Reality (MR) environment, enabling the real-time detection and annotation of vials with immediate operator feedback. Compared to standard barcode-based methods and the baseline YOLOv4-Tiny model, the proposed approach improves identification accuracy while maintaining low computational overhead. The experimental evaluations demonstrate a mean average precision (mAP) of 98.76 percent, with an inference speed of 68 milliseconds per frame on HoloLens 2, achieving real-time performance. The results highlight the model’s robustness in diverse lighting conditions and its ability to mitigate misclassifications of visually similar vials. By combining deep learning with MR, this system offers a more reliable and efficient alternative for pharmaceutical and medical applications, paving the way for AI-driven MR-assisted workflows in critical healthcare environments.

The accurate identification of medicine vials is crucial for emergency medical services, especially for vials that resemble one another but have different labels, volumes, and concentrations. This study introduces a method to detect vials in real-time using mixed reality technology through Microsoft HoloLens 2. The system is also equipped with an SQL server to manage barcode and vial information. We conducted a comparative analysis of the barcode detection capabilities of the HoloLens 2 camera and an external scanner. The HoloLens 2 effectively identified larger barcodes when they were 20–25 cm away in normal lighting conditions. However, it faced difficulties in detecting smaller barcodes that were consistently detected by the external scanner. The frame rate investigation revealed performance fluctuations: an average of 10.54 frames per second (fps) under standard lighting conditions, decreasing to 10.10 fps in low light and further reducing to 10.05 fps when faced with high barcode density. Resolution tests demonstrated that a screen resolution of 1920 × 1080 yielded the best level of accuracy, with a precision rate of 98%. On the other hand, a resolution of 1280 × 720 achieved a good balance between accuracy 93% and speed. The HoloLens 2 demonstrates satisfactory performance under ideal circumstances; however, enhancements in detecting algorithms and camera resolution are required to accommodate diverse surroundings. This approach seeks to help paramedics make quick and accurate decisions during critical situations and tackle common obstacles such as reliance on networks and human mistakes. Our new approach of a hybrid method that integrates an external Bluetooth scanner with the MR device gives optimal results compared to the scanner-only approach.

Current state-of-the-art (SOTA) LiDAR-only detectors perform well for 3D object detection tasks, but point cloud data are typically sparse and lacks semantic information. Detailed semantic information obtained from camera images can be added with existing LiDAR-based detectors to create a robust 3D detection pipeline. With two different data types, a major challenge in developing multi-modal sensor fusion networks is to achieve effective data fusion while managing computational resources. With separate 2D and 3D feature extraction backbones, feature fusion can become more challenging as these modes generate different gradients, leading to gradient conflicts and suboptimal convergence during network optimization. To this end, we propose a 3D object detection method, Attention-Enabled Point Fusion (AEPF). AEPF uses images and voxelized point cloud data as inputs and estimates the 3D bounding boxes of object locations as outputs. An attention mechanism is introduced to an existing feature fusion strategy to improve 3D detection accuracy and two variants are proposed. These two variants, AEPF-Small and AEPF-Large, address different needs. AEPF-Small, with a lightweight attention module and fewer parameters, offers fast inference. AEPF-Large, with a more complex attention module and increased parameters, provides higher accuracy than baseline models. Experimental results on the KITTI validation set show that AEPF-Small maintains SOTA 3D detection accuracy while inferencing at higher speeds. AEPF-Large achieves mean average precision scores of 91.13, 79.06, and 76.15 for the car class’s easy, medium, and hard targets, respectively, in the KITTI validation set. Results from ablation experiments are also presented to support the choice of model architecture.

External human–machine interfaces (eHMIs) serve as communication bridges between autonomous vehicles (AVs) and road users, ensuring that vehicles convey information clearly to those around them. While their potential has been explored in one-to-one contexts, the effectiveness of eHMIs in complex, real-world scenarios with multiple pedestrians remains relatively unexplored. Addressing this gap, our study provides an in-depth evaluation of how various eHMI displays affect pedestrian behavior. The research aimed to identify eHMI configurations that most effectively convey an AV’s information, thereby enhancing pedestrian safety. Incorporating a mixed-methods approach, our study combined controlled outdoor experiments, involving 31 participants initially and 14 in a follow-up session, supplemented by an intercept survey involving 171 additional individuals. The participants were exposed to various eHMI displays in crossing scenarios to measure their impact on pedestrian perception and crossing behavior. Our findings reveal that the integration of a flashing green LED, robotic sign, and countdown timer constitutes the most effective eHMI display. This configuration notably increased pedestrians’ willingness to cross and decreased their response times, indicating a strong preference and enhanced concept understanding. These findings lay the groundwork for future developments in AV technology and traffic safety, potentially guiding policymakers and manufacturers in creating safer urban environments.

Advancements in driving automation systems such as Lane Keeping Assist has shown improvements in safety by reducing traffic accidents. Nonetheless, despite the promise of these technologies, some studies have shown statistical data that such systems have a poor resilient operation. Therefore, there is a need to design more robust and resilient Advanced Driver Assistance Systems (ADAS) while broadening their operational design domain. This study proposes a set of non-functional system level requirements informed by applying Resilience Engineering (RE) principles to a Lane Centering (LC) system and comparing it to a conventional LC system when subjected to disruptive events. The LC system was integrated with a monitoring system based on infrastructure sensor technology as a potential solution to address the proposed non-functional system-level requirements. These two LC systems were developed in the CARLA simulator and tested in a lane line occlusion and adaptive speed scenario. RE metrics such as the resistance and recovery were measured to assess the ability of the system to sustain and recover from disruptive events. Simulation results indicate that the LC system with the proposed system requirements shows a higher resistance and recovery score in the lane line occlusion scenario compared to the conventional LC system. In contrast, the LC system with the proposed system requirements achieved a higher resistance score but a lower recovery score in the adaptive speed scenario compared to the conventional LC system. These results demonstrate that the proposed safety non-functional system requirements can be used cope with more unknown events and that RE principles can provide the stepping stones to design and enable resilient automated driving operations.

Connected autonomous vehicle (CAV) technology has the potential to enable significant gains in energy economy (EE). Much research attention has been focused on autonomous eco-driving control enabled by various methods. In this study, the state of the literature on autonomous eco-driving control is reviewed, an overall systems’ description of eco-driving control for a CAV is provided, and representative methods are evaluated comparatively against each other in simulation. Simulations are conducted using real-world traffic signal data and a validated future automotive systems technology simulator (FASTSim) model. Results indicate that an EE improvement in the range of 5%–15% is attainable depending on the method and cost function used. In this article it is shown that dynamic programming (DP) methods are most effective in improving EE but are significantly more computationally expensive than other methods. The genetic algorithm (GA) methods are shown to present the most potential in terms of EE improvement and run-time. Results also indicate that velocity-sensitive cost functions allow all the methods to perform better than pure acceleration minimization.