Hyonyoung Han

Dexterous robotic grasping gains great benefits from tactile sensation, but delicate object exploration by tactile information is challenged by difficulty in rich and efficient data production. In this letter, we propose a tactile-based in-hand object perception approach, empowered by a sim-to-real approach toward a data-efficient learning process. A high-fidelity tactile input, measured by a pair of vision-based tactile sensors, was represented as a point cloud facilitating dual functionality of object classification and the associated pose estimation. For the classification, we constructed PoinTacNet , a variation of PointNet to fit into tactile data processing. A reliable simulation methodology on tactile input was employed for the pretraining of the model, transferred to the fine-tuning process facing real tactile data. Taking inspiration from human behaviors, a re-grasping strategy was imparted by means of conditional accumulation of class likelihood distribution. The result of the framework facilitates a high object classification accuracy of 83.89 $\%$ on the ten objects from McMaster-Carr's CAD models, which is significantly improved by the re-grasping. In addition, a set of benchmarks displays the computational efficiency in the sim-to-real transfer. In line with the successful classification, the posture of in-hand objects is estimated using point cloud registration algorithms, achieving an average angular and translational RMSE of 5.03 $^\circ$ and 2.41 mm, respectively. The proposed approach has the potential to enable robots to attain human-like haptic exploration skills for perceiving unstructured environments.

We propose a simple and robust HSV color-space-based algorithm that can automatically extract object position information without human intervention or prior knowledge. In manufacturing sites with high variability, it is difficult to recognize products through robot machine vision, especially in terms of extracting object information accurately, owing to various environmental factors such as the noise around objects, shadows, light reflections, and illumination interferences. The proposed algorithm, which does not require users to reset the HSV color threshold value whenever a product is changed, uses ROI referencing method to solve this problem. The algorithm automatically identifies the object’s location by using the HSV color-space-based ROI random sampling, ROI similarity comparison, and ROI merging. The proposed system utilizes an IoT device with several modules for the detection, analysis, control, and management of object data. The experimental results show that the proposed algorithm is very useful for industrial automation applications under complex and highly variable manufacturing environments.

This study proposes a system architecture based on integrated Computer-Aided Design (CAD) and Bill of Material (BOM) databases for a robot assembly planning system. The architecture consists of CAD, Product Data Management (PDM) and robot assembly planning applications based on CAD, BOM and robot databases. It manages shape and product structure data from 3D CAD files as data objects in CAD and BOM databases, and stores general robot assembly planning data from robot teaching results as robot path objects in the robot database. Robot engineers create robot assembly plans using a process BOM that incorporates data from the CAD, BOM and robot databases. Throughout the generation of robot assembly plans, the architecture uses the shape data of participating parts in CAD database to automatically calculate the positions of robot end-effectors. This paper describes the implementation of applications and databases in the architecture using FreeCAD, MariaDB and Apache Tomcat Web application server. It also outlines cases of experimental robot assembly planning using the implemented planning systems and robot arms at laboratory level.