Jun Gong

Trajectory prediction is a critical technique for autonomous driving, robot navigation and intelligent surveillance. The recurrent neural networks (RNNs) models propagate temporal information through hidden states and have been widely applied in adapting to varying input lengths and generating multi-modal trajectories. However, the high cost of computation and accumulated errors limit the development of RNN based models in the field of trajectory prediction. We suggest that RNN based models can encode or predict multiple time steps simultaneously in one iteration. In this paper, we propose a RNN based Fragment Trajectory Prediction Network (FTPN), which combines multiple time steps as a whole fragment for encoding and relationship modeling. Specifically, a proposed fragment encoder embeds the fragments equally split from a trajectory instead of time steps, and then uses a relationship estimator to get the neighbors' influence on moving targets based on prior knowledge. After fusing target and neighbors features, the model iteratively predicts future embeddings over fragment dimension, and generates multimodal trajectories through a multi style decoder. The results show that our model achieves state-of-the-art performance on ETH-UCY, SDD and NBA datasets. Additional ablation experiments demonstrate that our method can greatly reduce computation costs while improving model performance.

Algorithm of outlier removal for learning-based 3D point cloud registration is usually regarded as a classification problem. The core for this to be successful is to learn the discriminative inlier/outlier feature representations. This letter proposes a two-stage efficient network (MuSCLe-Reg) with multi-scale local feature fusion embedding. Specifically, we have designed a two-stage registration architecture. Firstly, we construct a graph topology feature consisting of correspondences and their feature neighborhoods. Then, the feature representation of correspondences was enhanced through multi-scale feature mapping and fusion (MSF). In addition, we propose a local correspondence rectification strategy (LCR) based on feature neighbors to evaluate initial candidates and generate higher-quality correspondences. The experimental results on various datasets show that compared with existing learning-based algorithms, this network has better accuracy and stronger generalization ability. Especially, in robustness testing across varying numbers of correspondences in the 3DLoMatch dataset, the algorithm demonstrated superior estimation performance compared to current state-of-the-art registration techniques.

Trajectory prediction is an important task in autonomous driving and monitoring systems. Most of the existing methods pay little attention to the rapidly changing trajectory information, but how to effectively solve this problem is crucial to ensure pedestrian safety. The Gabor transform has inherent advantages for capturing instantaneously changing information. Therefore, for the first time, we introduce the Gabor transformation idea into pedestrian trajectory prediction and propose the Multi-scale Learnable Gabor Transform Network (MlgtNet), which establishes global and local contextual relationships from multi-dimensional and multi-scale perspectives. The network first uses the Multi-scale Feature Dimension Enhancement Module (MFDEM) ascending dimension trajectory sequence, and uses the Multi-scale Gabor Convolution Module (MGCM) to guide the model to establish the dependence of different distances from different dimensions to model the interrelationship between global/local features at different scales and different step sizes. Finally, the Feature Fusion Module (FFM) processes the multimodal information and fuses it with the multi-scale trajectory features to obtain the trajectory prediction representation in different visual fields. The representation results are then used for secondary fusion to obtain the global prediction results. Experimental results show that MlgtNet achieves state-of-the-art performance with its lightweight model size on the vast majority of widely used trajectory prediction datasets from different perspectives.

Pedestrian's future trajectory prediction is a key challenge in ego-centric view of autonomous driving system. Most of the current methods are flawed in capturing subtle change features in a lightweight model size. To solve this problem, we propose a multimodal forward generation transformer network based on encoder-decoder structure. Different from the traditional transformer, we improve layer normalization and propose frame normalization, which can more successfully capture minute time-variant properties. In addition, we believe that considering short-term pedestrian's future goals can help the ego-vehicle to predict more accurate and reasonable long-term pedestrians’ trajectory. Therefore, based on the idea of forward generation, the decoder considers the future short-term targets and uses trajectory-time correlation module to capture the relationship between estimated short-term future goals and global spatial-temporal context cues of the historical trajectory. Our model is evaluated on JAAD and PIE datasets and achieves state-of-the-art performance while maintaining a lightweight model size.

Anomaly detection has a wide range of applications and is especially important in industrial quality inspection. Currently, many top-performing anomaly detection models rely on feature embedding-based methods. However, these methods do not perform well on datasets with large variations in object locations. Reconstruction-based methods use reconstruction errors to detect anomalies without considering positional differences between samples. In this study, a reconstruction-based method using the noise-to-norm paradigm is proposed, which avoids the invariant reconstruction of anomalous regions. Our reconstruction network is based on M-net and incorporates multiscale fusion and residual attention modules to enable end-to-end anomaly detection and localization. Experiments demonstrate that the method is effective in reconstructing anomalous regions into normal patterns and achieving accurate anomaly detection and localization. On the MPDD and VisA datasets, our proposed method achieved more competitive results than the latest methods, and it set a new state-of-the-art standard on the MPDD dataset.

For unsupervised anomaly detection and localization, a common approach is learning the distribution of normal samples and then use it as a criterion to identify abnormalities. This article proposes a multilevel features restoration method based on damaged images (MLDFR) for anomaly detection and localization. MLDFR seeks to restore the “normal feature” of the test sample. Specifically, we damage the training samples to generate the corresponding samples, and then design a concurrent feature extractor utilizing convolutional neural network and transformer pretrained on ImageNet to completely represent the multilevel features of samples. Additionally, we fully consider the dependencies among local features over long distances and design a feature restoration module. On the challenging, widely used anomaly detection datase (MVTec-AD), metal parts defect detection dataset (MPDD), and beantech anomaly detection dataset (BTAD) of real-world datasets, MLDFR achieves state-of-the-art anomaly localization performance, as well as image-level detection with virtually flawless score. We further report ablation studies, demonstrating MLDFR's effectiveness and generalizability.

Anomaly detection has a wide range of applications and is especially important in industrial quality inspection. Currently, many top-performing anomaly-detection models rely on feature-embedding methods. However, these methods do not perform well on datasets with large variations in object locations. Reconstruction-based methods use reconstruction errors to detect anomalies without considering positional differences between samples. In this study, a reconstruction-based method using the noise-to-norm paradigm is proposed, which avoids the invariant reconstruction of anomalous regions. Our reconstruction network is based on M-net and incorporates multiscale fusion and residual attention modules to enable end-to-end anomaly detection and localization. Experiments demonstrate that the method is effective in reconstructing anomalous regions into normal patterns and achieving accurate anomaly detection and localization. On the MPDD and VisA datasets, our proposed method achieved more competitive results than the latest methods, and it set a new state-of-the-art standard on the MPDD dataset.

Accurate segmentation of retinal blood vessels is an important prerequisite for diagnosing many fundus diseases. Many methods based on artificially designed convolutional neural networks (CNNs) have been successively proposed and achieved good results. However, the image quality collected by fundus camera is poor, and the vessels topology in the image is complex, the pure convolution feature extraction in these CNN methods is limited. A novel multilevel remote relational modeling network (MRRM-Net) is proposed, which can balance global/local long-range contextual relations through building multidimensional and multilevel attention. Based on the symmetric structure, we first design the three-dimensional guided attention module (TGAM) in the encoder to establish long-range dependency descriptions with semantically rich spatial structure through similarity and then design a cross-level sensitive attention module (CSAM) at the skip connection position to adaptively and dynamically correct low-level error information while strengthening the long-range spatial structure features established by the attention guidance module. These two modules can gradually establish and fuse the long-range semantic context. Furthermore, we propose a novel feature fusion method (FFM) to pass high-level semantics into the decoder. The experimental results show that our proposed method has better superiority, especially for capillaries with complex background and variable structure, and has more robust data adaptability than other state-of-the-art network structures in a small capacity. On multiple public datasets, it shows better stability. MRRM-Net can provide a theoretical basis for real-time detection technology of retinopathy.