Song Xu

The 6D pose estimation of objects is of great significance for the intelligent assembly and sorting of industrial parts. In the industrial robot production scenarios, the 6D pose estimation of industrial parts mainly faces two challenges: one is the loss of information and interference caused by occlusion and stacking in the sorting scenario, the other is the difficulty of feature extraction due to the weak texture of industrial parts. To address the above problems, this paper proposes an attention-based pixel-level voting network for 6D pose estimation of weakly textured industrial parts, namely CB-PVNet. On the one hand, the voting scheme can predict the keypoints of affected pixels, which improves the accuracy of keypoint localization even in scenarios such as weak texture and partial occlusion. On the other hand, the attention mechanism can extract interesting features of the object while suppressing useless features of surroundings. Extensive comparative experiments were conducted on both public datasets (including LINEMOD, Occlusion LINEMOD and T-LESS datasets) and self-made datasets. The experimental results indicate that the proposed network CB-PVNet can achieve accuracy of ADD(-s) comparable to state-of-the-art using only RGB images while ensuring real-time performance. Additionally, we also conducted robot grasping experiments in the real world. The balance between accuracy and computational efficiency makes the method well-suited for applications in industrial automation.

Drive joints are key components of humanoid robots, and their technological development directly affects the development of humanoid robots. This paper reviews the origin and history of humanoid robots, analyzes the drive joints of humanoid robots, discusses the key technologies of the drive joints such as drive scheme and reducer, and researches the drive, transmission, force sensing, servo control, and thermal control technologies of the rotary drive integrated joints. It is pointed out that the development of rotary drive joints with integrated designs of high response, lightweight, variable stiffness, and high precision will be the future development trend of humanoid robot drive joints.

The Automated Guided Vehicles (AGVs) on the production lines mainly utilize positioning and navigation technology based on Light Detection and Ranging (LiDAR). There are problems with inaccurate positioning data due to data drift, environmental interference, and error accumulation. Based on Ultra-Wideband (UWB) sensors and data processing technology, the paper proposes a UWB sensor-assisted indoor localization method for AGVs in intelligent manufacturing factories. Three-dimensional spatial localization of the indoor production line is achieved by LiDAR and UWB sensors, which are used to acquire AGV coordinate information. During the operation of the AGV, the AGV communicates with the service end of the production line using the TCP communication protocol under the LAN to collect and store the coordinate data of the AGV in real time. Furthermore, this paper is based on statistical measurement principles and the “3σ principle”, preprocessing the data collected by the UWB sensor. The coordinate data collected by the two positioning methods are fitted and optimized using Kalman filtering. The experimental data are analyzed by trajectory analysis and calculating the average absolute error. The results show that the method in this paper controls the error range of AGV coordinates within ±0.2 meters, which meets the standard of indoor high-precision positioning. This paper combines LiDAR and UWB sensors with data processing techniques to provide a solution to the challenges associated with the high-precision indoor positioning of AGVs.

With the continuous development of robotics and computer vision technology, mobile robots have been widely applied in various fields. In this process, social navigation for robots have attracted considerable attention due to its human-robot interactivity. Social navigation allows robots to follow human social interaction norms and ensure the movement safety and social acceptability of navigation behaviors. This paper focuses on a review of social navigation methods for mobile robots. First, the development process of mobile robot social navigation is outlined. Then, social navigation methods are classified according to the type of social behavior perception. Also, this paper summarizes model-based social navigation methods and learning-based social navigation methods in detail. Finally, the challenges and future development trends of mobile robot social navigation are discussed.

With the continuous development of robotics and computer vision technology, mobile robots have been widely applied in various fields. In this process, semantic maps for robots have attracted considerable attention because they provide a comprehensive and anthropomorphic representation of the environment. On the one hand, semantic maps are a tool for robots to depict the environment, which can enhance the robot’s cognitive expression of space and build the communication bond between robots and humans. On the other hand, semantic maps contain spatial location and semantic properties of entities, which helps robots realize intelligent decision-making in human-centered indoor environments. In this paper, we review the primary approaches of semantic mapping proposed over the last few decades, and group them according to the type of information used to extract semantics. First, we give a formal definition of semantic map and describe the techniques of semantic extraction. Then, the characteristics of different solutions are comprehensively analyzed from different perspectives. Finally, the open issues and future trends regarding semantic maps are discussed in detail. We wish this review provides a comprehensive reference for researchers to drive future research in related field.

In the future, the goal of service robots is to operate in human-centric indoor environments, requiring close cooperation with humans. In order to enable the robot to perform various interactive tasks, it is necessary for robots to perceive and understand environments from a human perspective. Semantic map is an augmented representation of the environment, containing both geometric information and high-level qualitative features. It can help the robot to comprehensively understand the environment and bridge the gap in human-robot interaction. In this paper, we propose a unified semantic mapping system for indoor mobile robots. This system utilizes the techniques of scene classification and object detection to construct semantic representations of indoor environments by fusing the data of a camera and a laser. In order to improve the accuracy of semantic mapping, the temporal-spatial correlation of semantics is leveraged to realize data association of semantic maps. Also, the proposed semantic mapping system is scalable and portable, which can be applied to different indoor scenarios. The proposed system was evaluated with collected datasets captured in indoor environments. Extensive experimental results indicate that the proposed semantic mapping system exhibits great performance in the robustness and accuracy of semantic mapping.

Existing pre-trained language models (PLMs) have demonstrated the effectiveness of self-supervised learning for a broad range of natural language processing (NLP) tasks. However, most of them are not explicitly aware of domain-specific knowledge, which is essential for downstream tasks in many domains, such as tasks in e-commerce scenarios. In this paper, we propose K-PLUG, a knowledge-injected pre-trained language model based on the encoder-decoder transformer that can be transferred to both natural language understanding and generation tasks. We verify our method in a diverse range of e-commerce scenarios that require domain-specific knowledge. Specifically, we propose five knowledge-aware self-supervised pre-training objectives to formulate the learning of domain-specific knowledge, including e-commerce domain-specific knowledge-bases, aspects of product entities, categories of product entities, and unique selling propositions of product entities. K-PLUG achieves new state-of-the-art results on a suite of domain-specific NLP tasks, including product knowledge base completion, abstractive product summarization, and multi-turn dialogue, significantly outperforms baselines across the board, which demonstrates that the proposed method effectively learns a diverse set of domain-specific knowledge for both language understanding and generation tasks.

Mobile manipulation, which has more flexibility than fixed-base manipulation, has always been an important topic in the field of robotics. However, for sophisticated operation in complex environments, efficient localization and dynamic tracking grasp still face enormous challenges. To address these challenges, this paper proposes a mobile manipulation method integrating laser-reflector-enhanced adaptive Monte Carlo localization (AMCL) algorithm and a dynamic tracking and grasping algorithm. First, by fusing the information of laser-reflector landmarks to adjust the weight of particles in AMCL, the localization accuracy of mobile platforms can be improved. Second, deep-learning-based multiple-object detection and visual servo are exploited to efficiently track and grasp dynamic objects. Then, a mobile manipulation system integrating the above two algorithms into a robotic with a 6-degrees-of-freedom (DOF) operation arm is implemented in an indoor environment. Technical components, including localization, multiple-object detection, dynamic tracking grasp, and the integrated system, are all verified in real-world scenarios. Experimental results demonstrate the efficacy and superiority of our method.

Conventional approaches to global localization and navigation mainly rely on metric maps to provide precise geometric coordinates, which may cause the problem of large-scale structural ambiguity and lack semantic information of the environment. This paper presents a scalable vision-based topological mapping and navigation method for a mobile robot to work robustly and flexibly in large-scale environment. In the vision-based topological navigation, an image-based Monte Carlo localization method is presented to realize global topological localization based on image retrieval, in which fine-tuned local region features from an object detection convolutional neural network (CNN) are adopted to perform image matching. The combination of image retrieval and Monte Carlo provide the robot with the ability to effectively avoid perceptual aliasing. Additionally, we propose an effective visual localization method, simultaneously employing the global and local CNN features of images to construct discriminative representation for environment, which makes the navigation system more robust to the interference of occlusion, translation, and illumination. Extensive experimental results demonstrate that ERF-IMCS exhibits great performance in the robustness and efficiency of navigation.

Autonomous navigation is a basic prerequisite for mobile robot to realize environmental exploration. Current navigation methods are mainly based on metric maps, which require precise geometric coordinates and lack the capability to efficiently store semantic information of the environment. In this paper, we present a visual topological mapping and navigation method for mobile robot in large-scale environment, which is similar to the human navigation system. Topological map represents the environment as a topology diagram with nodes and edges in which the topological nodes record local semantic information of the environment, such as visual features, robot pose and scene properties. In the topological navigation stage, an image-based Monte Carlo localization is proposed to estimate the semantic pose of robot which can help robot judge whether it has reached the target location more flexibility. Experiments are conducted in real world environments and results indicate that the proposed system exhibits great performance in robustness of navigation.