Guodong Yang

Image fusion aims to integrate multi-modal data, enhancing information completeness while reducing redundancy. However, image fusion faces the challenge of information imbalance. This imbalance is reflected in both intra-image variations, where different regions exhibit significant differences in information density and importance, and cross-modal inconsistencies, where corresponding regions in different modalities contribute unequally to the fused result. Most existing image fusion methods, such as models based on CNN and window attention mechanism, adopt a fixed approach that collects the same amount of neighboring information for each pixel. However, this fixed approach lacks adaptability to local variations in information density, ultimately limiting fusion performance. To this end, we propose IA²GNN, an imbalance-aware adaptive graph neural network designed for image fusion. To tackle intra- and cross-modal information imbalance, we leverage the flexibility of graph structures to dynamically adjust node connections during feature extraction and fusion. Specifically, we adjust the connectivity of nodes in the graph to allow regions with rich details to establish more connections, thereby enhancing the extraction and preservation of key information in these regions. For less informative regions, we reduce connections to prevent over-modeling. In this way, we optimize information distribution, achieving a balance between detail retention and information integration in the fusion results. Experimental results demonstrate that the proposed model outperforms baseline methods across multiple image fusion tasks, achieving up to +0.058 improvement in SSIM and +0.351 gain in MI.

LiDAR-based loop closure detection is a crucial part of realizing robust SLAM algorithms for intelligent vehicles with LiDAR sensors. Existing methods often reduce the keypoint dimension to encode the global descriptor, which sacrifices the freedom of loop detection and correction. Based on the 6-DOF rigid transformation property of spatial triangles, we propose an algorithm for extracting and describing 3D keypoints from high-resolution spinning LiDAR intensity images to encode triangle descriptors, termed intensity triangle descriptor (ITD) . In comparison to the direct extraction of keypoints from the point cloud, the use of image-derived feature points provides additional photometric texture information and better handles uneven spatial density of the point cloud, which is advantageous in unstructured and geometrically degraded scenes. To enhance the stability of keypoints, the spatial positions of multi-frame image feature points are registered to a keyframe by an odometer for voxel downsampling and non-maximum suppression, with the objective of reducing unstable feature points. For high discrimination, the neighbor image patches of each vertex (keypoint) are aggregated to estimate a Gaussian mixture model (GMM) as the keypoint signature. An efficient two-stage loop closure detection method is then proposed for ITD, consisting of candidate retrieval based on triangle side lengths and vertex GMMs, followed by geometric verification of matched descriptor pairs. The effectiveness of the proposed method is evaluated on the STheReO, FusionPortable, and our self-collected datasets.

This letter presents the first trajectory planning method for hybrid robot to perform powerline inspection involving obstacle navigation and landing. We develop a geometric model that incorporates constraints for landing the hybrid robot on a powerline, obstacle avoidance, and objectives that maximize the visibility of the powerline during flight. The trajectory generation is achieved via solving a multiple shooting nonlinear programming problem with respect to system dynamics and geometric constraints. The formulation of the problem accommodates both powerline-to-powerline and air-to-powerline trajectory planning scenarios. It runs onboard and is capable of generating trajectories within 50 ms, regardless of whether the hybrid robot's initial state is positioned on the powerline or hovering above it. Through simulation experiments, we illustrate the impact of our proposed geometric model on trajectory planning. Furthermore, real-world experimental results validate the efficacy of the proposed planning method. Compared with the existing feedback-control-based work, the landing and obstacle navigation time are significantly reduced.

In this letter, we propose RI-LIO, a new reflectivity image assisted tightly-coupled LiDAR-inertial odometry (LIO) framework that introduces additional reflectivity texture information to efficiently reduce the drift of geometric-only methods. To achieve this, we construct an iterated extended Kalman filter framework by blending the point-to-plane geometric measurement and the reflectivity image measurement. Specifically, the geometric measurement is defined as the distance from the raw point of a new scan to its nearest neighbor plane in the global incremental kd-tree map. The searched nearest neighbor point is used to render a sparse reflectivity image after LiDAR motion distortion information is given by its corresponding raw point. Then, the reflectivity measurement is built to align the sparse reflectivity image with the dense reflectivity image of the current scan by minimizing the photometric errors directly. In addition, based on the mechanism of high-resolution LiDAR, a corrected spherical projection model is proposed to project spatial points into the image frame. Finally, extensive experiments are conducted in structured, unstructured and challenging open field scenarios. The results demonstrate that the proposed method outperforms existing geometric-only methods in terms of robustness and accuracy, especially in the rotation direction.

In recent years, various types of inspection robots have been developed to automate powerline inspection. The hybrid robot combines the advantages of climbing and flying robots and has a promising prospect in powerline inspection. But landing a hybrid robot on the target powerline among multiple ones is challenging. Flights require robust detection of powerlines and stable tracking of the target powerline. We propose a complete solution for autonomous landing of a hybrid robot on a powerline. First, a special feature extraction operator and the corresponding density-based feature recognition algorithm are designed to detect multi-scale powerlines. Second, a binocular vision-based depth estimation method for the landing point in the powerline is described. Third, two spatio-temporal dictionaries are established to track the target one in multiple powerlines. Meanwhile, landing strategies and control methods are presented to achieve a stable landing task. Finally, a hybrid robot is designed to validate the proposed method. The experiment results demonstrate the accuracy of the powerline detection and depth estimation algorithm, as well as the effectiveness of the robot in tracking and landing tasks.

Scoliosis is a common disease of the spine and requires regular monitoring due to its progressive properties. A preferred indicator to assess scoliosis is by the Cobb angle, which is currently measured either manually by the relevant medical staff or semi-automatically, aided by a computer. These methods are not only labor-intensive but also vary in precision by the inter-observer and intra-observer. Therefore, a reliable and convenient method is urgently needed. With the development of computer vision and deep learning, it is possible to automatically calculate the Cobb angles by processing X-ray or CT/MR/US images. In this paper, the research progress of Cobb angle measurement in recent years is reviewed from the perspectives of computer vision and deep learning. By comparing the measurement effects of typical methods, their advantages and disadvantages are analyzed. Finally, the key issues and their development trends are also discussed.

The failure of an insulator may compromise the safety of the entire power transmission system. Therefore, insulator defect detection is vital for the safe operation of power systems. However, insulator defects in an insulator image may have varying sizes, and several currently available methods do not have satisfactory detection accuracy for small defects. To address this issue, we propose an improved detection network for small insulator defects with a batch normalization convolutional block attention module (BN-CBAM) and a feature fusion module. The BN-CBAM is designed to better exploit channel information and enhance the effect of different channels on the feature map. In addition, we propose a feature fusion module that fuses multi-scale features from different layers to improve small object detection performance. Moreover, to address the scarcity of aerial images, a data augmentation method based on the fusion of the target segment and background is introduced. Experiments demonstrate that the proposed method achieves better small insulator defect detection performance than other state-of-the-art approaches. In addition, data augmentation methods enrich sample diversity and enhance the generalizability of the network.

Image-based segmentation of overhead power lines is critical for power line inspection. Real-time segmentation helps the inspection robot avoid obstacles or land on the wire during the inspection task. It is challenging for several studies to achieve real-time overhead power line segmentation with high accuracy. In addition, cluttered background brings great difficulties to overhead power lines segmentation. To address these issues, an efficient parallel branch network for real-time overhead power line segmentation is proposed. Our framework combines a context branch that generates useful global information with a spatial branch that preserves high-resolution segmentation details. The asymmetric factorized depth-wise bottleneck (AFDB) module is designed in the context branch to achieve more efficient short-range feature extraction and provide a large receptive field. Furthermore, the subnetwork-level skip connections in the classifier are proposed to fuse long-range features and lead to high accuracy. Experiments demonstrate that our framework achieves more than 90% segmentation accuracy.

There are a large number of insulators on the transmission line, and insulator damage will have a major impact on power supply security. Image-based segmentation of the insulators in the power transmission lines is a premise and also a critical task for power line inspection. In this paper, a modified conditional generative adversarial network for insulator pixel-level segmentation is proposed. The generator is reconstructed by encoder-decoder layers with asymmetric convolution kernel which can simplify the network complexity and extract more kinds of feature information. The discriminator is composed of a fully convolutional network based on patchGAN and learns the loss to train the generator. It is verified in experiments that the proposed method has better performances on mIoU and computational efficiency than Pix2pix, SegNet, and other state-of-the-art networks.

This paper has designed a special robot working in a narrow space. Planning an appropriate path is one of the significant problems. In this paper, a mathematical expression of the G value considering obstacles in the environment based on A* algorithm in low-dimensionality is proposed. The algorithm is a approaching path planning algorithm in static, known environments. We plan the path of the end-effector after it moves near the obstacles. As only the end-effector changes drastically, it is safe for other joints working in the collision environment if the other joints are obtained using inverse kinematics. The simulation result validates the effectiveness of our algorithm. We then describe the cost function in three dimensions briefly.