Angel Pascual del Pobil

This paper introduces UJI-Butler, an innovative multi-robot framework that blends symbolic and non-symbolic artificial intelligence methods. Unlike previous systems, UJI-Butler integrates large language models (LLMs) with a knowledge base akin to RAG-based systems, while imposing logical reasoning on LLM-generated results. It facilitates multi-modal interaction with human users through speech, sign language, and physical interaction, fostering a human-in-the-loop learning paradigm. By acquiring new knowledge through verbal communication and mastering manipulation skills via human-lead-through programming, UJI-Butler enhances transparency and trust by incorporating human feedback during operations. Experimental results demonstrate that UJI-Butler’s combination of symbolic and non-symbolic AI offers intuitive interaction and accelerates the learning process with experience. It adeptly stores and utilizes knowledge gained from verbal communication, recognizing hand gestures for requests. Additionally, UJI-Butler successfully performs user-taught physical skills and generalizes them to varying object sizes and locations. The explicit nature of acquired knowledge enables seamless transferability to other platforms and modification by human users. The code of the whole project is available on Github , in addition, video demonstrations of the UJI-Butler system are available online in a Youtube Playlist .

Authentication systems are crucial in the digital era, providing reliable protection of personal information. Most authentication systems rely on a single modality, such as the face, fingerprints, or password sensors. In the case of an authentication system based on a single modality, there is a problem in that the performance of the authentication is degraded when the information of the corresponding modality is covered. Especially, face identification does not work well due to the mask in a COVID-19 situation. In this paper, we focus on the multi-modality approach to improve the performance of occluded face identification. Multi-modal authentication systems are crucial in building a robust authentication system because they can compensate for the lack of modality in the uni-modal authentication system. In this light, we propose DemoID, a multi-modal authentication system based on face and voice for human identification in a challenging environment. Moreover, we build a demographic module to efficiently handle the demographic information of individual faces. The experimental results showed an accuracy of 99% when using all modalities and an overall improvement of 5.41%–10.77% relative to uni-modal face models. Furthermore, our model demonstrated the highest performance compared to existing multi-modal models and also showed promising results on the real-world dataset constructed for this study.

Science, Technology, Engineering, and Mathematics are key disciplines for the education of new generations in the modern digital world, and for the economic growth of developing countries. Access to computing resources in such countries can be difficult due to the cost of equipment. Nevertheless, mobile devices are becoming increasing available, as well as mobile networks for connecting to the internet. This work presents the STEM-bots, a tool for connecting people to computers through a mobile device. Particularly, we propose to use the Instant Messaging networks for sending commands to remote computers and getting the results right at hand. We argue that students, teachers, scientists, researchers, and interested people can benefit from such a tool for learning and working in scientific and engineering disciplines.

The application of connectionist learning techniques, namely unsupervised neural networks, to contact classification is investigated. This approach demonstrates the feasibility and appropriateness of using force sensing to solve this problem. Empirical results are provided for the chamferless two-dimensional peg-in-hole insertion model with friction. The advantages of learning approaches over geometric model-based techniques are discussed. Our neural network approach is simple but robust against unpredictable changes of task parameters, and it exhibits a gracefully degrading behavior and on-line adaptation to new task conditions.

In this paper we present a new, affordable, omnidirectional robot platform which is suitable for research and education in cooperative robotics. We design and implement the platform for the purpose of multi-agent object manipulation and transportation. The design consists of three omnidirectional wheels with two additional traction wheels, making multirobot object manipulation possible. It is validated by performing simple experiments using a setup with one robot and one target object. The execution flow of a simple task (Approach–Press–Lift–Hold–Set) is studied. In addition, we experiment to find the limits of the applied pressure and object orientation under certain conditions. The experiments demonstrate the significance of our inexpensive platform for research and education by proving its feasibility of use in topics such as collaborative robotics, physical interaction, and mobile manipulation.

Defining the collision-free C-space is crucial in robotics to find whether a robot can successfully perform a motion. However, the complexity of defining this space increases according to the robot’s degree of freedom and the number of obstacles. Heuristics techniques, such as Monte Carlo’s simulation, help developers address this problem and speed up the whole process. Many well-known motion planning algorithms, such as RRT, base their popularity on their ability to find sufficiently good representations of the collision-free C-space very quickly by exploiting heuristics methods, but this mathematical relationship is not highlighted in most textbooks and publications. Each book focuses the attention of the reader on C-space at the beginning, but this concept is left behind page after page. Moreover, even though heuristics methods are widely used to boost algorithms, they are never formalized as part of the Optimization techniques subject. The major goal of this chapter is to highlight the mathematical and intuitive relationship between C-space, heuristic methods, and sampling based planner.

Commercially viable automated picking in unstructured environments by a robot arm remains a difficult challenge. The problem of robot grasp planning has long been around but the existing solutions tend to be limited when it comes to deploy them in open-ended realistic scenarios. Practical picking systems are called for that can handle the different properties of the objects to be manipulated, as well as the problems arising from occlusions and constrained accessibility. This paper presents a practical solution to the problem of robot picking in an online shopping warehouse by means of a novel approach that integrates a carefully selected method with a new strategy, the centroid normal approach (CNA), on a cost-effective dual-arm robotic system with two grippers specifically designed for this purpose: a two-finger gripper and a vacuum gripper. Objects identified in the scene point cloud are matched to the grasping techniques and grippers to maximize success. Extensive experimentation provides clues as to what are the reasons for success and failure. We chose as benchmark the scenario proposed by the 2017 Amazon Robotics Challenge, since it represents a realistic description of a retail shopping warehouse case; it includes many challenging constraints, such as a wide variety of different product items with a diversity of properties, which are also presented with restricted visibility and accessibility.

Deep learning is the mainstream paradigm in computer vision and machine learning, but performance is usually not as good as expected when used for applications in robot vision. The problem is that robot sensing is inherently active, and often, relevant data is scarce for many application domains. This calls for novel deep learning approaches that can offer a good performance at a lower data consumption cost. We address here monocular depth estimation in warehouse automation with new methods and three different deep architectures. Our results suggest that the incorporation of sensor models and prior knowledge relative to robotic active vision, can consistently improve the results and learning performance from fewer than usual training samples, as compared to standard data-driven deep learning.