Lukas Rustler

Learning sensorimotor contingencies—that is, the link between one’s actions and their sensory effects—is fundamental to developing body knowledge, understanding causality, and developing a sense of agency. In developmental psychology, this process is classically studied using the mobile paradigm, where infants learn that movement of a limb causes motion of a connected mobile. To expand our understanding of how infants learn this, we tested an embodied computational model that learns through two biologically inspired mechanisms: prediction and curiosity. Implemented on the child-sized iCub humanoid robot interacting with a mobile, the model detected sensorimotor contingencies across several experimental conditions using a variety of movement strategies. Our findings suggest that contingency learning cannot be captured by a single behavioral metric, such as the amount of movement, but instead emerges through a spectrum of exploratory behaviors. Analysis of the robot’s internal activity reveals that these behaviors emerge from the dynamic trade-off between prediction and curiosity—between exploitation and exploration. Our work provides a biologically motivated, physically embodied model of sensorimotor interaction that connects theories of infant learning with robotic implementations. The results allow us to generate testable hypotheses for developmental research and to inform the design of autonomous learning systems.

What is considered safe for a robot operator during physical human–robot collaboration (HRC) is specified in corresponding HRC standards (e.g., ISO 10218‐2:2025). The regime that allows collisions between the moving robot and the operator, called Power and Force Limiting (PFL), restricts the permissible contact forces. Using the same fixed contact thresholds on the entire robot surface results unnecessary productivity losses, as the robot needs to stop even when impact forces are within limits. Here we present a framework that decides whether the robot should interrupt motion based on estimated collision force computed individually for different parts of the robot body and dynamically on the fly, based on the Effective Mass (EM) of each robot link. We performed experiments on a simulated and real collaborative robot arm (UR10e) with sensitive skin (AIRSKIN) for collision detection and isolation. To demonstrate the generality of our method, we added experiments on simulated KUKA LBR iiwa robot, where collision sensing draws on joint torque sensing. On a mock pick‐and‐place scenario with both transient and quasi‐static collisions, we show an increase in productivity over 45% from using the standard approach. The method is applicable to any robot for which the EF can be calculated.

Depth sensing is an essential technology in robotics and many other fields. Many depth sensing (or RGB-D) cameras are available on the market and selecting the best one for your application can be challenging. In this work, we tested four stereoscopic RGB-D cameras that sense the distance by using two images from slightly different views. We empirically compared four cameras (Intel RealSense D435, Intel RealSense D455, StereoLabs ZED 2, and Luxonis OAK-D Pro) in three scenarios: (i) planar surface perception, (ii) plastic doll perception, (iii) household object perception (YCB dataset). We recorded and evaluated more than 3,000 RGB-D frames for each camera. For table-top robotics scenarios with distance to objects up to one meter, the best performance is provided by the D435 camera that is able to perceive with an error under 1 cm in all of the tested scenarios. For longer distances, the other three models perform better, making them more suitable for some mobile robotics applications. OAK-D Pro additionally offers integrated AI modules (e.g., object and human keypoint detection). ZED 2 is overall the best camera which is able to keep the error under 3 cm even at 4 meters. However, it is not a standalone device and requires a computer with a GPU for depth data acquisition. All data (more than 12,000 RGB-D frames) are publicly available at https://osf.io/f2seb.

Recent advancements in object shape completion have enabled impressive object reconstructions using only visual input. However, due to self-occlusion, the reconstructions have high uncertainty in the occluded object parts, which negatively impacts the performance of downstream robotic tasks such as grasping. In this letter, we propose an active visuo-haptic shape completion method called Act-VH that actively computes where to touch the objects based on the reconstruction uncertainty. Act-VH reconstructs objects from point clouds and calculates the reconstruction uncertainty using IGR, a recent state-of-the-art implicit surface deep neural network. We experimentally evaluate the reconstruction accuracy of Act-VH against five baselines in simulation and in the real world. We also propose a new simulation environment for this purpose. The results show that Act-VH outperforms all baselines and that an uncertainty-driven haptic exploration policy leads to higher reconstruction accuracy than a random policy and a policy driven by Gaussian Process Implicit Surfaces. As a final experiment, we evaluate Act-VH and the best reconstruction baseline on grasping 10 novel objects. The results show that Act-VH reaches a significantly higher grasp success rate than the baseline on all objects. Together, this letter opens up the door for using active visuo-haptic shape completion in more complex cluttered scenes.