Shahab Eslamian

This paper describes the development of a new method to control the camera arm of a surgical robot and create a better sense of remote presence for the surgeon. The current surgical systems are entirely controlled by the surgeon, using hand controllers and foot pedals to manipulate either the instrument or the camera arms. The surgeon must pause the operation to move the camera arm to obtain a desired view and then resume the operation. The camera and tools cannot be moved simultaneously, leading to interrupted and unnatural movements. These interruptions can lead to medical errors and extended operation times. In our system, the surgeon controls the camera arm by his natural head movements while being immersed in a 3D-stereo view of the scene with a head-mounted display (HMD). The novel approach enables the camera arm to be maneuvered based on sensors of the HMD. We implemented this method on a da Vinci Standard Surgical System using the HTC Vive headset along with the Unity engine and the Robot Operating System framework. This paper includes the result of a subjective six-participant usability study that compares the workload of the traditional clutched camera control method against the HMD-based control. Initial results indicate that the system is usable, stable, and has a lower physical and mental workload when using the HMD control method.

This paper describes a recording and playback system developed using a da Vinci Standard Surgical System and research kit. The system records stereo laparoscopic videos, robot arm joint angles, and surgeon–console interactions in a synchronized manner. A user can then, on-demand and at adjustable speeds, watch stereo videos and feel recorded movements on the hand controllers of entire procedures or sub procedures. Currently, there is no reported comprehensive ability to capture expert surgeon movements and insights and reproduce them on hardware directly. This system has important applications in several areas: (1) training of surgeons, (2) collection of learning data for the development of advanced control algorithms and intelligent autonomous behaviors, and (3) use as a “black box” for retrospective error analysis. We show a prototype of such an immersive system on a clinically-relevant platform along with its recording and playback fidelity. Lastly, we convey possible research avenues to create better systems for training and assisting robotic surgeons.