Le Zhang

In robotic manipulation, robot-assisted surgery, and other contact-rich interactions with deformable bodies, reliable prediction of contact force and deformation requires rapid determination of the effective mechanical parameters governing the soft-tissue model. However, these model-equivalent properties are difficult to obtain directly from physical interaction, particularly for human soft tissue with substantial variability in geometry and mechanical response. This work presents a force–vision multimodal framework for rapidly identifying effective elastic and contact-model parameters from measured force and deformation. The human breast is selected as a representative validation object because of its high compliance, pronounced geometric variability, and sensitivity of its deformation response to mechanical properties and contact conditions. A six-axis force/torque sensor and an eye-in-hand RGB-D camera provide time-synchronized measurements of three-dimensional contact force and surface-marker displacement. A finite-element contact model maps the probe pose and a parameter vector to the measured responses, while a multiobjective Bayesian optimization procedure identifies the parameter set under a limited simulation budget. Experiments using three breast-tissue-mimicking phantoms with different geometries and stiffnesses yield normalized force and displacement residuals of 4.93%–6.00% and 7.18%–12.33%, respectively, within 55 model evaluations. Force and displacement provide complementary constraints, and the Bayesian solver achieves residuals comparable to particle swarm optimization with a 5.5-fold reduction in the number of finite-element evaluations. Held-out indentation and controlled posture-change tests further support the predictive consistency of the identified model-equivalent parameters within the experimental protocol. The proposed framework provides an approach for rapid parameter identification of deformable-tissue contact models, with potential to support more consistent reproduction of contact force and deformation in soft-tissue simulation and future sim2real robotic manipulation with force–vision feedback.

The robotic breast ultrasound scanning (RBUS) system can help sonographer in the early screening of breast cancer. However, it still faces rigorous safety concerns, including non-smooth scanning paths and improper contact force regulation. In this letter, we propose a new path optimization method to improve the scanning smoothness, as well as, a dynamic contact regulation strategy considering both the breast deformation and ultrasound images. For the path planning, the anti-radial scanning approach is first adopted to generate some initial path points together with their normal vectors; then, a dual-objective optimization function is formulated, which considers both the acceleration continuity and deviation of the path, to filter out local path anomalies. For the force control, a contact force-strain regression model is first proposed and used to predict the desired force between the breast and probe at each path point. While within the adjacent path points, an updating algorithm is introduced to dynamically adjust the contact force in real-time by surveying the confidence of the ultrasound image. Our experimental results show that the proposed methods can make the scanning path smoother, and obtain intimate probe-tissue contact with less desired force.

Traditional breast cancer surgeries require collaboration between ultrasound (US) doctors and surgeons, making the procedure complex and treating physicians prone to fatigue. In leader–follower robotic surgery, a surgeon controls an US robotic arm and an instrument robotic arm with their left and right hands, enabling independent surgical performance. However, the lack of US scanning skills among surgeons, as well as the physical separation in leader–follower operations, can negatively impact both the scanning and surgical outcomes. This paper proposes a robot-assisted scheme based on dynamic arm impedance compensation (IC) that references expert arm stiffness to compensate for novice arm stiffness. The impedance compensator adjusts the compensation strategy according to the scanning area and scanning stage. The impedance force generator estimates the scanning direction via Kalman filtering and applies stiffness and damping forces in the vertical direction to suppress tremors and other involuntary movements. The experimental results revealed that during the coarse and fine scanning phases, the probe position variance decreased by 57.9% and 73.6%, the contact force variance decreased by 55.2% and 42.5%, and the US image confidence increased by 22.0% and 23.8%, respectively. Compared with traditional filtering compensation (FC) schemes, this approach reduces the average position variance and contact force variance by 32.0% and 25.3%, respectively, and increases confidence by 7.3%. In a no-compensation test, the IC training group outperformed the FC group. This scheme can assist leader–follower US scanning and rapidly improve surgical skills.

Robotic ultrasound-guided system improves surgical flexibility and accuracy. Ultrasound (US) calibration is the key to obtain spatial relationship and localization of US images. However, existing calibration methods mainly rely on an external tracker, which leads to complicated workflow and prone to error accumulation. In this paper, we proposed a robot-assisted automatic US calibration method. This method can automatically complete the US calibration without any external tracker, thus improving the calibration accuracy and simplifying the workflow. In addition, we propose an automatic image segmentation method incorporating prior knowledge. It automatically segments the features of the calibration model in the US image during the US scanning process, and ensures the smooth execution of the automatic robot calibration process. Specifically, we transformed the calibration problem into a point-to-point registration problem and conducted experiments using three different registration algorithms. The results show that the calibration accuracy of our method is 0.65±0.20 mm. Our method significantly reduces calibration time compared to manual calibration. We also compare the proposed method with state-of-the-art methods, and the results demonstrated that the proposed method performs better.

Experienced sonographers can adjust their arm impedance in real-time to obtain high-quality ultrasound (US) images during US scanning. These operational skills can be captured through robot systems with multimodal data collection capabilities (position, force, and impedance). However, low operational transparency between the system (generally, a serial robot with admittance control) and its users will result in significant delays and errors, interfering with the skill acquisition process. The paper proposes a new system that adopts the parallel mechanism (Omega.7) to improve the transparency of the operation. The scanning probe and a 6-axis force sensor are attached to the end of Omega.7. When operating the probe, a zero-force drag effect can be realized through gravity and torque compensations. The arm impedance during the scanning can be measured through the force disturbance method by analyzing external forces on the device. Ultrasonic scans were conducted on phantoms of different hardness, and arm impedance was measured. Statistical analysis reveals that when scanning softer phantoms, arms exhibit higher stiffness. The transparency analysis results show that the equipment designed in this paper has a higher level of transparency than the scheme of serial robot with admittance control. The high operation transparency of the system makes it an ideal skill-acquisition device with broad applications.

Experienced sonographers can improve ultrasound (US) image quality by adjusting their hand impedance. However, little research exists on how the hand’s impedance is adjusted, especially at different detection stages. The arm impedance of novices and experts at different scanning stages was measured and compared in depth using a developed ultrasonic scanning skill acquisition system with high operational transparency. The end-of-hand impedance was measured by introducing an external disturbance and fitting a mass–damper–spring model to the hand. US scanning quality was measured by filtering the variance of position signals and the variance of contact force through high-pass filters. Multivariate variance analysis indicated that experts exhibit lower position and contact force variances, as well as larger stiffness values than novices. In the normal direction, which is considered the most critical for US scanning, experts had less position variance and significantly greater hand stiffness than novices. When transitioning from coarse scanning to fine scanning, experts improved the scanning effect by increasing hand stiffness in this direction. The impedance measurement results in this paper provide a theoretical basis for robot remote or autonomous ultrasonic scanning and can help robots learn the impedance adjustment skills of experts.