Longhui Qin

Shape recognition is essential in multifarious manipulations, such as object grasping, block building, and parts assembly, which most frequently relies on visual perception. However, it inevitably suffers from adverse impact of illumination intensity, visual distortion from different views, and frequent occlusions in dynamic scenes. To tackle this problem, the paper proposes recognizing different shapes via tactile exploration, mimicking the human manner of touch in dark environments. Based on our designed tactile fingertip, which comprises four sensing elements and a fingerprint structure, we employed the signal processing framework consisting of data preprocessing, feature pool establishment, selection of the most discriminative feature combination, and recognition model using a shallow neural network. Particularly, the feature pool was extended to comprise 386 multi-perspective features spanning different domains and scales. When the nine features with the highest weights were selected using Neighborhood Component Analysis, our extreme learning machine-based recognition model achieved an accuracy as high as 99.72%. To contextualize this performance, a comprehensive benchmark was conducted against several state-of-the-art deep learning paradigms, including end-to-end and transfer learning models. The analysis reveals that for data-constrained and resource-sensitive tactile sensing applications, our proposed and interpretable feature engineering approach not only achieves superior accuracy but also offers significant advantages in computational efficiency. This work validates the practical value of tailored feature engineering and sheds light on more efficient utilization of tactile signals in real-world robotic recognition tasks.

Robotic assembly of electrical connectors enables the automation of high-efficiency production of electronic products. A rigid gripper is adopted as the end-effector by the majority of existing works with a force–torque sensor installed at the wrist, which suffers from very limited perception capability of the manipulated objects. Moreover, the grasping and movement actions, as well as the inconsistency between the robot base and the end-effector frame, tend to result in angular misalignment, usually leading to assembly failure. Bio-inspired by the human finger, we designed a tactile finger in this paper with three characteristics: (1) Compliance: A soft ‘skin’ layer provides passive compliance for plenty of manipulation actions, thus increasing the tolerance for alignment errors. (2) Tactile Perception: Two types of sensing elements are embedded into the soft skin to tactilely sense the involved contact status. (3) Enhanced manipulation force: A rigid fingernail is designed to enhance the manipulation force and enable potential delicate operations. Moreover, a tactile-based alignment algorithm is proposed to search for the optimal orientation angle about the z axis. In the application of U-disk insertion, the three characteristics are validated and a success rate of 100% is achieved, whose generalization capability is also validated through the assembly of three types of electrical connectors.

In robot-assisted minimally invasive surgery (RMIS), robotic palpation is vital for enhancing tissue assessment accuracy, especially for tumor depth detection, which is crucial for precise resections and improved treatment outcomes. By complementing and leveraging the strengths of data from multiple tactile sensors, the tumor detection task can realize significant improvements in accuracy and reliability, thereby enhancing overall performance in robotic palpation in RMIS. However, challenges arise due to differences in sensor modalities and the lack of unified data representation. To address this, we fabricate silicone-based phantom tissue to simulate soft tissue and embedded simulated tumors with hard inclusions at depths ranging from 0–11 mm, treating it as a 12-class classification problem. We conduct two robotic palpation experiments: one with the BarrettHand capacitive sensor and the other with the Digit sensor, collecting two tactile datasets. To explore the joint learnability of datasets collected from different tactile sensors, we design a dual autoencoder-based joint learning framework that integrates two recurrent autoencoders to process the two different tactile datasets. By applying a joint loss mechanism to connect their latent spaces, the autoencoders are trained jointly together, with the latent representation used for supervised classification. Extensive experiments show that joint learning enables sharing of features learned from different tactile datasets, thereby enhancing learning efficiency and classification accuracy for tactile datasets with different modalities and improving the performance of both autoencoders compared to independent training.

As an important natural medium for human interaction with the external environment, the hand has a high degree of dexterity and complexity, and accurate human hand digital twin models play an important role in the field of medical rehabilitation. In this paper, magnetic-inertial sensing fusion information is used as driving data to build a personalized digital twin model of the human hand. A parametric hand model based on the individual phalange bio-constructive characteristics is established. To derive the personalized individual hand parametric model, the fingertip position detection problem is transformed into a magnetic field inverse problem by using a passive magnetic source as a marker with s tructurized spatial physical field. The magnetic-inertial sensing fusion information helps dimensionality reduction of the higher-order full-posture inversion problem without individual calibration. The derived personalized hand geometry parameters were compared with medical digital photographic markers with an error of [0.38, 2.87] mm. A personalized human hand digital twin platform driven by magnetic-inertial sensing fusion information was developed based on the Matlab Simscape Multibody module. Dynamic twin model driver based on real-time human hand motion detection information was implemented. The sphere grasp experiments show that the relative fitting error (%) according to derived fingertip position for the radius of 50 mm, 60 mm, and 70 mm is 0.14 %, 0.25 %, and 1.08 %, respectively, demonstrating the good performance of the proposed system.

To date, numerous tactile sensors and algorithms have been developed to tackle various perception problems. However, excessive endeavors are devoted to the multifarious definition, extraction, and analysis of hand-crafted features in order to improve the perception accuracy. To address this problem, in this paper, we designed a tactile finger containing four sensing elements to perceive both dynamic and static stimuli, and meanwhile proposed a novel signal processing pipeline. The pipeline mainly consisted of time-series signals conversion, automatic deep features extractor, and a shallow recognition model. When the tactile finger was applied to explore 16 surfaces based on a robotic platform, the four-channel signals were converted and concatenated into a time-frequency image via continuous wavelet transform. A deep feature extraction network was constructed based on a pretrained deep learning (DL) model, Resnet101, to extract the required features, which acted as high-level representations of the most discriminative components from the tactile images. Finally, these features were fed into a shallow machine learning model, i.e., extreme learning machine, achieving an accuracy as high as 92.38%. In such a manner, the powerful learning capability of DL models was transferred to the new recognition model directly while the tedious feature extraction procedures were alleviated. Besides, several relevant issues, such as the layer depth, the DL model type, and the shallow recognition model, are addressed and discussed to reveal their influences on the performance.

In this paper, the nonlinear mechanical response of elastic cable structures under mechanical load is studied based on the discrete catenary theory. A cable net is discretized into multiple nodes and edges in our numerical approach, which is followed by an analytical formulation of the elastic energy and the associated Hessian matrix to realize the dynamic simulation. A fully implicit framework is proposed based on the discrete differential geometry (DDG) theory. The equilibrium configuration of a target object is derived by adding damping force into the system, known as the dynamic relaxation method. The mechanical response of a single suspended cable is investigated and compared with the analytical solution for cross-validation. A more intricate scenario is further discussed in detail, where a structure consisting of multiple slender cables is connected through joints. Utilizing the robustness and efficiency of our discrete numerical framework, a systematic parameter sweep is performed to quantify the force displacement relationships of nets with the different number of cables and different directions of fibers. Finally, an empirical scaling law is provided to account for the rigidity of elastic cable net in terms of its geometric properties, material characteristics, component numbers, and cable orientations. Our results would provide new insight in revealing the connections between flexible structures and tensegrity structures, and could motivate innovative designs in both mechanical and civil engineered equipment.