Simon Bøgh

Recent progress in prosthetic manipulation highlights the need for perception-driven control strategies that can adapt to diverse objects and user intent. This work presents a modular vision-guided grasping pipeline that integrates VLM-based object identification, orientation-aligned geometric modeling, and per-finger grasp planning for dexterous prosthetic hands. A Vision–Language Model (VLM) identifies the target object and activates the grasping pipeline only when recognition is confident, supporting intent-aware operation. From the segmented point cloud, an object-aligned bounding box (OBB) is constructed to provide a compact, orientation-aware representation of the object’s global extents, enabling more accurate distance and collision queries than axis-aligned boxes. Using this representation, the system evaluates candidate fingertip trajectories and selects contact poses for each finger independently, followed by Damped Least Squares inverse kinematics for joint-level execution. Preliminary experiments on a limited set of representative objects using the Linker Hand O7 demonstrate that the proposed pipeline achieves consistent grasp execution and exhibits promising real-time performance within controlled scenarios. In simulation, the proposed pipeline achieved a maximum segmentation accuracy of 93.4%, while hardware experiments on the Linker Hand O7 achieved 93.2% segmentation accuracy, confirming stable grasp execution across representative objects. While the evaluation is not yet comprehensive, the results indicate that combining semantic object identification with lightweight geometric reasoning can support efficient and adaptable grasp generation suitable for future prosthetic applications.

Recent advancements in prosthetic technology have increasingly focused on enhancing dexterity and autonomy through intelligent control systems. Vision-based approaches offer promising results for enabling prosthetic hands to interact more naturally with diverse objects in dynamic environments. Building on this foundation, the paper presents a vision-guided grasping algorithm for a prosthetic hand, integrating perception, planning, and control for dexterous manipulation. A camera mounted on the set up captures the scene, and a Bounding Volume Hierarchy (BVH)-based vision algorithm is employed to segment an object for grasping and define its bounding box. Grasp contact points are then computed by generating candidate trajectories using Rapidly-exploring Random Tree Star (RRT*) algorithm, and selecting fingertip end poses based on the minimum Euclidean distance between these trajectories and the object’s point cloud. Each finger’s grasp pose is determined independently, enabling adaptive, object-specific configurations. Damped Least Square (DLS) based Inverse kinematics solver is used to compute the corresponding joint angles, which are subsequently transmitted to the finger actuators for execution. Our intention in this work was to present a proof-of-concept pipeline demonstrating that fingertip poses derived from a simple, computationally lightweight geometric representation, specifically an AABB-based segmentation can be successfully propagated through per-finger planning and executed in real time on the Linker Hand O7 platform. The proposed method is validated in simulation, and experimental integration on a Linker Hand O7 platform.

Autonomous experimental platforms increasingly rely on robust, vision-guided robotic manipulation to support reliable and repeatable laboratory operations. This work presents a modular motion-execution subsystem designed for integration into self-driving laboratory (SDL) workflows, focusing on the coupling of real-time visual perception with smooth and stable manipulator control. The framework enables autonomous detection, tracking, and interaction with textured objects through a hybrid scheme that couples advanced motion planning algorithms with real-time visual feedback. Kinematic analysis of the manipulator is performed using the screw theory formulations, which provide a rigorous foundation for deriving forward kinematics and the space Jacobian. These formulations are further employed to compute inverse kinematic solutions via the Damped Least Squares (DLS) method, ensuring stable and continuous joint trajectories even in the presence of redundancy and singularities. Motion trajectories toward target objects are generated using the RRT* algorithm, offering optimal path planning under dynamic constraints. Object pose estimation is achieved through a a vision workflow integrating feature-driven detection and homography-guided depth analysis, enabling adaptive tracking and dynamic grasping of textured objects. The manipulator’s performance is quantitatively evaluated using smoothness metrics, RMSE pose errors, and joint motion profiles, including velocity continuity, acceleration, jerk, and snap. Simulation results demonstrate that the proposed subsystem delivers stable, smooth, and reproducible motion execution, establishing a validated baseline for the manipulation layer of next-generation SDL architectures.

Self-driving laboratories are redefining autonomous experimentation by integrating robotic manipulation, computer vision, and intelligent planning to accelerate scientific discovery. This work presents a vision-guided motion planning framework for robotic manipulators operating in dynamic laboratory environments, with a focus on evaluating motion smoothness and control stability. The framework enables autonomous detection, tracking, and interaction with textured objects through a hybrid scheme that couples advanced motion planning algorithms with real-time visual feedback. Kinematic modeling of the manipulator is carried out using the screw theory formulations, which provides a rigorous foundation for deriving forward kinematics and the space Jacobian. These formulations are further employed to compute inverse kinematic solutions via the Damped Least Squares (DLS) method, ensuring stable and continuous joint trajectories even in the presence of redundancy and singularities. Motion trajectories toward target objects are generated using the RRT* algorithm, offering optimal path planning under dynamic constraints. Object pose estimation is achieved through a vision pipeline that integrates feature-based detection with homography-driven depth analysis, enabling adaptive tracking and dynamic grasping of textured objects. The manipulator’s performance is quantitatively evaluated using smoothness metrics, RMSE pose errors, and joint motion profiles including velocity continuity, acceleration, jerk, and snap. Simulation studies demonstrate the robustness and adaptability of the proposed framework in autonomous experimentation workflows, highlighting its potential to enhance precision, scalability, and efficiency in next-generation self-driving laboratories.

Motion planning schemes are used for planning motions of a manipulator from an initial pose to a final pose during a task execution. A motion planning scheme generally comprises of a trajectory planning method and an inverse kinematic solver to determine trajectories and joints solutions respectively. In this paper, 3 motion planning schemes developed based on Jacobian methods are implemented to traverse a redundant manipulator with a coupled finger gripper through given trajectories. RRT* algorithm is used for planning trajectories and screw theory based forward kinematic equations are solved for determining joint solutions of the manipulator and gripper. Inverse solutions are computed separately using 3 Jacobian based methods such as Jacobian Transpose (JT), Pseudo Inverse (PI), and Damped Least Square (DLS) methods. Space Jacobian and manipulability measurements of the manipulator and gripper are obtained using screw theory formulations. Smoothness and RMSE error of generated trajectories and velocity continuity, acceleration profile, jerk, and snap values of joint motions are analysed for determining an efficient motion planning method for a given task. Advantages and disadvantages of the proposed motion planning schemes mentioned above are analysed using simulation studies to determine a suitable inverse solution technique for the tasks.

Recent advances in robotics and autonomous systems have broadened the use of robots in laboratory settings, including automated synthesis, scalable reaction workflows, and collaborative tasks in self-driving laboratories (SDLs). This paper presents a comprehensive development of a mobile manipulator designed to assist human operators in such autonomous lab environments. Kinematic modeling of the manipulator is carried out based on the Denavit Hartenberg (DH) convention and inverse kinematics solution is determined to enable precise and adaptive manipulation capabilities. A key focus of this research is enhancing the manipulator ability to reliably grasp textured objects as a critical component of autonomous handling tasks. Advanced vision-based algorithms are implemented to perform real-time object detection and pose estimation, guiding the manipulator in dynamic grasping and following tasks. In this work, we integrate a vision method that combines feature-based detection with homography-driven pose estimation, leveraging depth information to represent an object pose as a $2$D planar projection within $3$D space. This adaptive capability enables the system to accommodate variations in object orientation and supports robust autonomous manipulation across diverse environments. By enabling autonomous experimentation and human-robot collaboration, this work contributes to the scalability and reproducibility of next-generation chemical laboratories