He Liu
He Liu has a total of 7 co-authors. Here are the top co-authors by number of shared publications:
- Jian Tao
Department of Big Data Research and Application Technology, China Aero-Polytechnology Establishment, Beijing 100028, China
1 shared publication - Mangui Liang
Beijing Jiaotong University
1 shared publication - Ran An
Department of Big Data Research and Application Technology, China Aero-Polytechnology Establishment, Beijing 100028, China
1 shared publication Shiyi Cui
School of Control and Computer Engineering, North China Electric Power University, Beijing, China
1 shared publication- Shuyan Liu
Department of Big Data Research and Application Technology, China Aero-Polytechnology Establishment, Beijing 100028, China
1 shared publication Wenjuan An
Institute of Information Science, Beijing Jiaotong University, Beijing, People’s Republic of China
1 shared publication
Latest Publications
- Open Access
- 13 May 2023
- Published by MDPI AG in Mathematics
- Vol. 11 (10) , 2279
An End-to-End Framework Based on Vision-Language Fusion for Remote Sensing Cross-Modal Text-Image Retrieval
Remote sensing cross-modal text-image retrieval (RSCTIR) has recently attracted extensive attention due to its advantages of fast extraction of remote sensing image information and flexible human–computer interaction. Traditional RSCTIR methods mainly focus on improving the performance of uni-modal feature extraction separately, and most rely on pre-trained object detectors to obtain better local feature representation, which not only lack multi-modal interaction information, but also cause the training gap between the pre-trained object detector and the retrieval task. In this paper, we propose an end-to-end RSCTIR framework based on vision-language fusion (EnVLF) consisting of two uni-modal (vision and language) encoders and a muti-modal encoder which can be optimized by multitask training. Specifically, to achieve an end-to-end training process, we introduce a vision transformer module for image local features instead of a pre-trained object detector. By semantic alignment of visual and text features, the vision transformer module achieves the same performance as pre-trained object detectors for image local features. In addition, the trained multi-modal encoder can improve the top-one and top-five ranking performances after retrieval processing. Experiments on common RSICD and RSITMD datasets demonstrate that our EnVLF can obtain state-of-the-art retrieval performance.
- 13 May 2023
- Published by MDPI AG in Mathematics
- Vol. 11 (10) , 2279
Open AccessAn End-to-End Framework Based on Vision-Language Fusion for Remote Sensing Cross-Modal Text-Image Retrieval
Remote sensing cross-modal text-image retrieval (RSCTIR) has recently attracted extensive attention due to its advantages of fast extraction of remote sensing image information and flexible human–computer interaction. Traditional RSCTIR methods mainly focus on improving the performance of uni-modal feature extraction separately, and most rely on pre-trained object detectors to obtain better local feature representation, which not only lack multi-modal interaction information, but also cause the training gap between the pre-trained object detector and the retrieval task. In this paper, we propose an end-to-end RSCTIR framework based on vision-language fusion (EnVLF) consisting of two uni-modal (vision and language) encoders and a muti-modal encoder which can be optimized by multitask training. Specifically, to achieve an end-to-end training process, we introduce a vision transformer module for image local features instead of a pre-trained object detector. By semantic alignment of visual and text features, the vision transformer module achieves the same performance as pre-trained object detectors for image local features. In addition, the trained multi-modal encoder can improve the top-one and top-five ranking performances after retrieval processing. Experiments on common RSICD and RSITMD datasets demonstrate that our EnVLF can obtain state-of-the-art retrieval performance.
- Open Access
- 1 January 2021
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 1748 (2) , 022014
Video stabilization algorithm for tunnel robots based on improved Kalman filter
The application of tunnel robots in power tunnels is becoming more and more extensive. Due to various reasons, video jitter will inevitably occur during the inspection process, which will affect the real-time processing of subsequent images. In order to reduce the impact of video jitter and meet the requirements of real-time image processing, it is necessary to study fast video image stabilization methods. The traditional Kalman filtering algorithm has less computational cost, but it will cause a large estimation error when the system motion state changes. In this regard, the paper proposes an improved Kalman filtering algorithm, which changes the corresponding filter parameters by real-time estimation of a system in motion to reduce the estimation error. And this algorithm performs Harris corner extraction in the set feature dense area, and combines the PyrLK optical flow algorithm with the homography matrix to calculate the motion trajectory, which improves the calculation speed and accuracy of the algorithm. Experimental results show that this algorithm has better video stabilization effect and faster calculation speed compared with the traditional Kalman filtering algorithm, and can better meet the real-time image processing requirements in the inspection process.
- 1 January 2021
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 1748 (2) , 022014
Open AccessVideo stabilization algorithm for tunnel robots based on improved Kalman filter
The application of tunnel robots in power tunnels is becoming more and more extensive. Due to various reasons, video jitter will inevitably occur during the inspection process, which will affect the real-time processing of subsequent images. In order to reduce the impact of video jitter and meet the requirements of real-time image processing, it is necessary to study fast video image stabilization methods. The traditional Kalman filtering algorithm has less computational cost, but it will cause a large estimation error when the system motion state changes. In this regard, the paper proposes an improved Kalman filtering algorithm, which changes the corresponding filter parameters by real-time estimation of a system in motion to reduce the estimation error. And this algorithm performs Harris corner extraction in the set feature dense area, and combines the PyrLK optical flow algorithm with the homography matrix to calculate the motion trajectory, which improves the calculation speed and accuracy of the algorithm. Experimental results show that this algorithm has better video stabilization effect and faster calculation speed compared with the traditional Kalman filtering algorithm, and can better meet the real-time image processing requirements in the inspection process.
- 9 June 2014
- Published by SAGE Publications in Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science
- Vol. 229 (3) , 580-588
An improved one-class support vector machine classifier for outlier detection
Outlier detection, as a type of one-class classification problem, is one of important research topics in data mining and machine learning. Its task is to identify sample points markedly deviating from the normal data. A reliable outlier detector needs to build a model which encloses the normal data tightly. In this paper, an improved one-class SVM (OC-SVM) classifier is proposed for outlier detection problems. We name this method OC-SVM with minimum within-class scatter (OC-WCSSVM), which exploits the inner-class structure of the training set via minimizing the within-class scatter of the training data. This can construct a more accurate hyperplane for outlier detection, such that the margin between the training data and the origin in a higher dimensional space is as large as possible, while at the same time the decision boundary around the normal data is as tight as possible. Experimental results on a synthetic dataset and 10 real-world datasets demonstrate that our proposed OC-WCSSVM algorithm is effective and superior to the compared algorithms.
- 9 June 2014
- Published by SAGE Publications in Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science
- Vol. 229 (3) , 580-588
An improved one-class support vector machine classifier for outlier detection
Outlier detection, as a type of one-class classification problem, is one of important research topics in data mining and machine learning. Its task is to identify sample points markedly deviating from the normal data. A reliable outlier detector needs to build a model which encloses the normal data tightly. In this paper, an improved one-class SVM (OC-SVM) classifier is proposed for outlier detection problems. We name this method OC-SVM with minimum within-class scatter (OC-WCSSVM), which exploits the inner-class structure of the training set via minimizing the within-class scatter of the training data. This can construct a more accurate hyperplane for outlier detection, such that the margin between the training data and the origin in a higher dimensional space is as large as possible, while at the same time the decision boundary around the normal data is as tight as possible. Experimental results on a synthetic dataset and 10 real-world datasets demonstrate that our proposed OC-WCSSVM algorithm is effective and superior to the compared algorithms.