Giancarlo Ferrigno

Endoscopic minimally invasive surgery relies on precise tissue video segmentation to avoid complications such as vascular bleeding or nerve injury. However, existing video segmentation methods often fail to maintain long-term robustness due to target loss and challenging conditions (e.g., occlusion, motion blur), limiting their applicability in prolonged surgical procedures. To address these limitations, we proposed the Unified Framework for Joint Pixel-Level Segmentation and Tracking (STF), it integrates a synergistic segmentation-guided tracking pipeline with an adaptive re-detection mechanism. First, a deep learning-based segmentation network precisely localizes the target tissue. A cost-efficient Hough Voting Network then tracks the segmented region, while a Bayesian refinement module improves compatibility between segmentation and tracking. If tracking reliability drops, an evaluation module triggers re-segmentation, ensuring continuous and stable long-term performance. Extensive experiments confirm that STF achieves superior accuracy and temporal consistency over segmentation networks in long-term surgical video segmentation, particularly under extreme conditions. This automated methodology significantly improves the robustness and re-detection capability for sustained tissue analysis, markedly reducing the dependency on manual intervention prevalent in many model-based tracking solutions.

Parkinson's disease is defined by dopaminergic neuron loss in the substantia nigra, yet its hallmark, exaggerated beta-band synchrony, pervades motor cortex, thalamus, and cerebellum, implicating network dynamics far beyond any single circuit. How focal subcortical dopamine depletion translates into brain-wide oscillatory pathology remains unresolved. We use a connectome-constrained multiscale model of the mouse brain, embedding biophysically detailed spiking networks of basal ganglia and cerebellum within whole-brain corticothalamic dynamics grounded in the Allen Mouse Brain Connectivity Atlas. We show that confining dopamine depletion exclusively to subcortical circuits is sufficient to produce widespread beta hypersynchrony (10-30 Hz), accompanied by heterogeneous theta and gamma dysregulation. Virtual loop ablations reveal that cortical and cerebellar beta amplification strictly requires intact cortico-basal ganglia-thalamic feedback; severing this loop confines beta to subcortical generators. These results support resonance within closed large-scale loops, rather than local rhythmogenesis, as the mechanism underlying distributed Parkinsonian beta pathology.

Ultrasound (US)-guided needle-based interventions have been extensively employed in clinical practice for biopsies, regional anesthesia, and tumor ablation, considering the advantages of real-time imaging, portability, and non-ionizing radiation. Precise localization and tracking of the needle tip are critical for ensuring the successful needle placement while minimizing the risk of injury to surrounding anatomical structures. However, achieving real-time and reliable needle tip tracking remains challenging due to the inherent properties of US images, such as low spatial resolution, speckle noise, and imaging artifacts that obscure the needle tip. This study proposes a hybrid learning-based framework for accurately and robustly localizing the needle tip within the US image plane during robot-assisted interventional procedures. The framework comprises a dual-branch template matching-based detection module, a Convolutional Neural Network (CNN)-based detection module, and a “dynamic template selection and updating" mechanism to enhance overall tracking performance. Additionally, to recover the needle tip invisibility caused by unanticipated movements, an active exploration strategy based on a Gaussian Process (GP) model and Bayesian Optimization (BO) is proposed for efficient and accurate robotic US probe pose adjustment. The effectiveness of the proposed method for needle tip tracking performance has been evaluated through a series of experiments conducted on ex vivo porcine liver samples under varying insertion angles and velocities. Experimental results demonstrate that the proposed framework achieves a maximum median localization error of 1.17 mm and an Interquartile Range (IQR) of 1.28 mm across all test conditions. Furthermore, in scenarios requiring needle tip visibility recovery, the proposed BO-based strategy achieved a median localization error of 1.52 mm and an IQR of 0.95 mm.

Objective: Surgical workflow understanding, especially surgical phase and step recognition, plays a crucial role for improving context-aware computer-assisted system in modern operating room. However, many state-of-the-art methods are limited by the scarcity of diverse and well-annotated datasets and have only been experimented with laparoscopic and non-robotic datasets, which makes the advancement of intelligent systems and model generalization validation in varied and intricate real-word surgical situation extremely challenging. Methods: To address this challenge, we introduce a video dataset with expert-annotated workflow information for Robot-assisted Radical Prostatectomy with Lymphadenectomy. We record surgical videos performed on 10 patients using the da Vinci Xi robot at the European Institute of Oncology in Milan, Italy. The videos are annotated under the guidance of experienced surgeons, with the entire dataset exceeding 25.3 hours. Subsequently, we perform a comprehensive evaluation using CNN-based and Transformer-based methods to establish potential benchmarks for surgical phase and step recognition on the dataset. Each model is rigorously assessed using five-fold cross-validation to ensure reliability and generalizability. Results: Eight representative models, including six CNN-based (SAHC, PhaseNet, SV-RCNet, TMR-Net, TeCNO, and Trans-SVNet) and two Transformer-based approaches (Timesformer and Surgformer), are evaluated across both tasks. For surgical phase recognition, Surgformer achieves the highest performance with the accuracy of 90.3%. For surgical step recognition, Surgformer_16_4 yields comparable results to that of Timesformer_16_4, around 78%. Significance: We present a complete procedural dataset in the data-scarce domain of RARPL, enabling robust benchmarking for surgical workflow recognition and bridging the gap between algorithm development and clinical practice. The dataset is accessible at the link:

Spinal intervention can benefit from advancements in robotic systems, particularly in the field of Human-Robot Interaction (HRI). Despite their potential, robotic technologies remain limited to selected spine procedures. Meanwhile, complex and time-consuming procedures, such as osteotomy, are still performed manually, impacting surgeon workload and stress. This paper introduces a robotic system to assist surgeons during the drilling of the vertebral body, enhancing precision and improving patient outcomes. The system integrates a pre-planning tool for establishing the desired correction angle, a virtual fixture-based impedance control algorithm to assist the user in following cutting trajectories while safeguarding surrounding structures, and a navigation tool for enhanced procedural guidance. The system consists of a 7 Degree-of-Freedom robotic manipulator equipped with a drilling system attached to its end-effector. System accuracy was validated on 5 vertebral phantoms, achieving tracking errors of 0.39 $\pm $ 0.033 mm and 0.74 $\pm $ 0.07 mm, margin errors of 0.84 $\pm $ 0.17 mm and 0.19 $\pm $ 0.5 mm, and angular alignment errors of 0.34 $\pm $ 0.16 and 0.13 $\pm $ 0.27$^{\circ } $ for the sagittal and coronal planes, respectively. Additionally, the system's performance was evaluated with real porcine bone, with results demonstrating similar precision and robustness under realistic mechanical properties. A user Study was conducted to evaluate whether the proposed system effectively enhanced user performance compared to a manual control strategy. The results demonstrated reduced deviation from the desired path and fewer instances of the system exceeding predetermined restricted areas when using the proposed system. Additionally, participants rated the proposed system as less mentally and physically demanding in comparison to the traditional control strategy.

This work presents a compliant and passive shared control framework for teleoperated robot-assisted tasks. Inspired by the human operator's capability of continuously regulating the arm impedance to perform contact-rich tasks, a novel control schema, exploiting the variable impedance control framework for force tracking is proposed. Moreover, bilateral teleoperation and shared control strategies are implemented to alleviate the human operator's workload. Furthermore, a global energy tank-based approach is integrated to enforce the system's passivity. The proposed framework is first evaluated to assess the force-tracking capability when the robot autonomously performs contact-rich tasks, e.g., in an ultrasound scanning scenario. Then, a validation experiment is conducted utilizing the proposed shared control framework. Finally, the system's usability is investigated with 12 users. The experiment results in system assessment revealed a maximum median error of 0.25 N across all the force-tracking experiment setups, i.e., constant and time-varying ones. Then, the validation experiment demonstrated significant improvements regarding the force tracking tasks compared to conventional control methods, and the system passivity was preserved during the task execution. Finally, the usability experiment shows that the human operator workload is significantly reduced by 54.6% compared to the other two control modalities. The proposed framework holds significant potential for the execution of remote robot-assisted medical procedures, such as palpation and ultrasound scanning, particularly in addressing deformation challenges while ensuring safety, compliance, and system passivity.

Advancements in robotic systems hold significant promise for enhancing spinal interventions. Despite this potential, the integration of robotic platforms in spine surgeries remains limited to only a few procedures. This paper presents a variable impedance control scheme within a shared-control framework to enhance haptic feedback during spinal surgeries. The system allows surgeons to guide the robot while dynamically adjusting stiffness based on contact forces and human intent, using electromyography signals. This adaptive control offers real-time guidance during interactions with different materials, serving as a safety measure to safeguard delicate structures encountered during surgical maneuvers. The system comprises a 7-DoF robotic manipulator with a 6-axis force/torque sensor and an 8-channel EMG sensor. Technical validation and a user study assessed performance compared to constant parameter (CIC) and linear variable (LVIC) impedance control methods. Results showed reduced contact force ( $ -5.065 \pm 1.45{\kern2.84526pt}N$ vs CIC's $ -13.72 \pm 6.52 {\kern2.84526pt}N$ and LVIC's $ -8.73 \pm 2.41 {\kern2.84526pt}N$ ) and in-contact displacement ( $ 0.0074 \pm 0.0018 {\kern2.84526pt} m$ vs CIC's $ 0.019 \pm 0.0068 {\kern2.84526pt} m$ and LVIC's $ 0.0125 \pm 0.0028 {\kern2.84526pt} m$ ), when interacting with delicate materials, minimizing the risk to critical anatomical structures. Additionally, a user survey confirmed that the proposed system improved haptic perception and control while preventing undesired movements during interactions with various tissues and structures.

The integration of robotics and Electrical Stimulation (ES) in neurorehabilitation combines the precise execution of complex tasks enabled by robotics with motor learning, muscle conditioning, and cardiovascular fitness induced by ES. We propose a hybrid lower-limb exoskeleton for gait training to maximize these benefits. Two neuromuscular electrical stimulators and a 4-degrees-of-freedom motorized exoskeleton were combined with different modalities: ES-motor cooperative integration for the swinging knee, synchronized but independently operating ES and motors for the hip both in swing and in stance phases, and for the knee during stance, and ES-only for the non-actuated ankle. Two conditions - exoskeleton only and hybrid - were compared on twelve non-disabled subjects and eleven participants with neurological disorders. Electrical stimulation reduced the output motor torque at the knee by approximately 50%, while maintaining comparable movement performance (tracking errors <8° in both conditions and groups). Overall, the hybrid system was perceived as more usable by participants with neurological disorders compared to the exoskeleton alone, with an improvement of around 16%. These results are promising for developing lighter exoskeletons with potentially enhanced therapeutic benefits for motor learning.

3D pose reconstruction of surgical instruments from images stands as a critical component in environment perception within robotic minimally invasive surgery (RMIS). The current deep learning methods rely on complex networks to enhance accuracy, making real-time implementation difficult. Moreover, diverging from a singular rigid body, surgical instruments exhibit an articulation structure, making the annotation of 3D poses more challenging. In this paper, we present a novel approach to formulate the 3D pose reconstruction of articulated surgical instruments as a Markov Decision Process (MDP). A Reinforcement Learning (RL) agent employs 2D image labels to control a virtual articulated skeleton to reproduce the 3D pose of the real surgical instrument. Firstly, a convolutional neural network is used to estimate the 2D pixel positions of joint nodes of the surgical instrument skeleton. Subsequently, the agent controls the 3D virtual articulated skeleton to align its joint nodes’ projections on the image plane with those in the real image. Validation of our proposed method is conducted using a semi-synthetic dataset with precise 3D pose labels and two real datasets, demonstrating the accuracy and efficacy of our approach. The results indicate the potential of our method in achieving real-time 3D pose reconstruction for articulated surgical instruments in the context of RMIS, addressing the challenges posed by low-texture surfaces and articulated structures.