Maria-Magdalena Wolf

As automated vehicles continue to evolve, teleoperation is emerging as a fallback solution in edge-case scenarios where human intervention is required. To ensure effective and safe remote support, the design of human-machine interfaces (HMIs) must be centered around the needs and capabilities of the operator. In recent years, various graphical user interfaces (GUIs) for teleoperation have been developed and predominantly evaluated in simulation environments. An integrated investigation of display and interaction concepts in combination with real vehicle teleoperation remains lacking. This work addresses this gap by investigating a GUI in three different layout options for two teleoperation concepts: Direct Control using Steering Wheel and Pedals, and Trajectory Guidance through separate path and velocity input via Mouse and Keyboard or Touchscreen. The conducted user study (N = 45) evaluates these approaches using a 1:10 scaled vehicle in a controlled environment to enable the collection of metrics, such as collisions, in challenging scenarios without the intervention of a safety driver or incurring high consequential costs. The evaluation shows that different interaction concepts favor different GUI layouts. For Steering Wheel and Pedals, a Picture-in-Picture layout is preferred, whereas for sequential input via Touchscreen or Mouse and Keyboard, a Horizontal split layout proves more suitable. Additionally, it emphasizes the advantage of Direct Control via Steering Wheel and Pedals as being significantly faster than Trajectory Guidance using a Touchscreen or Mouse and Keyboard. Overall, participants consider the user interface acceptable in terms of usability and workload. The participants’ feedback provides valuable insights and design suggestions for further improvements, serving as a foundation for future research.

Teleoperation emerged as a promising fallback for situations beyond the capabilities of automated vehicles. Nevertheless, teleoperation still faces challenges, such as reduced situational awareness. Since situational awareness is primarily built through the remote operator’s visual perception, the graphical user interface (GUI) design is critical. In addition to video feed, supplemental informational elements are crucial—not only for the predominantly studied remote driving, but also for emerging desk-based remote assistance concepts. This work develops a GUI for different teleoperation concepts by identifying key informational elements during the teleoperation process through expert interviews (N = 9). Following this, a static and dynamic GUI prototype was developed and evaluated in a click dummy study (N = 36). Thereby, the dynamic GUI adapts the number of displayed elements according to the teleoperation phase. Results show that both GUIs achieve good system usability scale (SUS) ratings, with the dynamic GUI significantly outperforming the static version in both usability and task completion time. However, the results might be attributable to a learning effect due to the lack of randomization. The user experience questionnaire (UEQ) score shows potential for improvement. To enhance the user experience, the GUI should be evaluated in a follow-up study that includes interaction with a real vehicle.

Teleoperation emerged as a promising fallback for situations beyond the capabilities of automated vehicles. Nevertheless, teleoperation still faces challenges, such as reduced situational awareness. Since situational awareness is primarily built through the remote operator's visual perception, the Graphical User Interface (GUI) design is critical. In addition to video feeds, supplemental informational elements are crucial - not only for the predominantly studied Remote Driving but also for the arising desk-based Remote Assistance concepts. This work develops a GUI for different teleoperation concepts by identifying key informational elements during the teleoperation process through expert interviews (N = 9). Following this, a static and dynamic GUI prototype is developed and evaluated in a click-dummy study (N = 36). Thereby, the dynamic GUI adapts the number of displayed elements according to the teleoperation phase. Results show that both GUIs achieve good System Usability Scale (SUS) ratings, with the dynamic GUI significantly outperforming the static version in both usability and task completion time. The User Experience Questionnaire (UEQ) score shows potential for improvement. To enhance the user experience, the GUI should be evaluated in a follow-up study that includes interaction with a real vehicle.

Teleoperation is a popular solution to remotely support highly automated vehicles through a human remote operator whenever a disengagement of the automated driving system is present. The remote operator wirelessly connects to the vehicle and solves the disengagement through support or substitution of automated driving functions and therefore enables the vehicle to resume automation. There are different approaches to support automated driving functions on various levels, commonly known as teleoperation concepts. A variety of teleoperation concepts is described in the literature, yet there has been no comprehensive and structured comparison of these concepts, and it is not clear what subset of teleoperation concepts is suitable to enable safe and efficient remote support of highly automated vehicles in a broad spectrum of disengagements. The following work establishes a basis for comparing teleoperation concepts through a literature overview on automated vehicle disengagements and on already conducted studies on the comparison of teleoperation concepts and metrics used to evaluate teleoperation performance. An evaluation of the teleoperation concepts is carried out in an expert workshop, comparing different teleoperation concepts using a selection of automated vehicle disengagement scenarios and metrics. Based on the workshop results, a set of three teleoperation concepts is derived that can be used to address a wide variety of automated vehicle disengagements in a safe and efficient way. This set includes the Remote Driving concept Shared Control as well as Collaborative Planning and Perception Modification from the Remote Assistance category.

Since Automated Driving Systems are not expected to operate flawlessly, Automated Vehicles will require human assistance in certain situations. For this reason, teleoperation offers the opportunity for a human to be remotely connected to the vehicle and assist it. The Remote Operator can provide extensive support by directly controlling the vehicle, eliminating the need for Automated Driving functions. However, due to the physical disconnection to the vehicle, monitoring and controlling is challenging compared to driving in the vehicle. Therefore, this work follows the approach of simplifying the task for the Remote Operator by separating the path and velocity input. In a study using a miniature vehicle, different operator-vehicle interactions and input devices were compared based on collisions, task completion time, usability and workload. The evaluation revealed significant differences between the three implemented prototypes using a steering wheel, mouse and keyboard or a touchscreen. The separate input of path and velocity via mouse and keyboard or touchscreen is preferred but is slower compared to parallel input via steering wheel.