Xin Zhou
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Xin Zhou has a total of 20 co-authors. Here are the top co-authors by number of shared publications:
- Lei Wang
Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, China and Key Laboratory of Target Cognition and Application Technology(TCAT), Beijing, China
2 shared publications - Xiyu Qi
Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China
2 shared publications - Yang Wang
Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China
2 shared publications - Yidan Zhang
University of the Chinese Academy of Sciences, Beijing, China
2 shared publications - Yongqiang Mao
Department of Electronic Engineering, Tsinghua University, Beijing, China
2 shared publications - Yunping Ge
Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China
2 shared publications
Latest Publications
- Open Access
- 30 June 2024
- Published by Combinatorial Press in Journal of Combinatorial Mathematics and Combinatorial Computing
- Vol. 120 (1) , 107-124
RICE: A Dataset and Baseline for Cloud Removal in Remote Sensing Images
Removing clouds is an essential preprocessing step in analyzing remote sensing images, as cloud-based overlays commonly occur in optical remote sensing images and can significantly limit the usability of the acquired data. Deep learning has exhibited remarkable progress in remote sensing, encompassing scene classification and change detection tasks. Nevertheless, the appli-cation of deep learning techniques to cloud removal in remote sensing images is currently con-strained by the limited availability of training datasets explicitly tailored for neural networks. This paper presents the Remote sensing Image Cloud rEmoving dataset (RICE) to address this challenge and proposes baseline models incorporating a convolutional attention mechanism, which has demonstrated superior performance in identifying and restoring cloud-affected regions, with quantitative results indicating a 3.08% improvement in accuracy over traditional methods. This mechanism empowers the network to comprehend better the spatial structure, local details, and inter-channel correlations within remote sensing images, thus effectively addressing the diverse distributions of clouds. Moreover, by integrating this attention mechanism, our models achieve a crucial comparison advantage, outperforming existing state-of-the-art techniques in terms of both visual quality and quantitative metrics. We propose adopting the Learned Per-ceptual Image Patch Similarity metric, which emphasizes perceptual similarity, to evaluate the quality of cloud-free images generated by the models. Our work not only contributes to advancing cloud removal techniques in remote sensing but also provides a comprehensive evaluation framework for assessing the fidelity of the generated images.
- 30 June 2024
- Published by Combinatorial Press in Journal of Combinatorial Mathematics and Combinatorial Computing
- Vol. 120 (1) , 107-124
Open AccessRICE: A Dataset and Baseline for Cloud Removal in Remote Sensing Images
Removing clouds is an essential preprocessing step in analyzing remote sensing images, as cloud-based overlays commonly occur in optical remote sensing images and can significantly limit the usability of the acquired data. Deep learning has exhibited remarkable progress in remote sensing, encompassing scene classification and change detection tasks. Nevertheless, the appli-cation of deep learning techniques to cloud removal in remote sensing images is currently con-strained by the limited availability of training datasets explicitly tailored for neural networks. This paper presents the Remote sensing Image Cloud rEmoving dataset (RICE) to address this challenge and proposes baseline models incorporating a convolutional attention mechanism, which has demonstrated superior performance in identifying and restoring cloud-affected regions, with quantitative results indicating a 3.08% improvement in accuracy over traditional methods. This mechanism empowers the network to comprehend better the spatial structure, local details, and inter-channel correlations within remote sensing images, thus effectively addressing the diverse distributions of clouds. Moreover, by integrating this attention mechanism, our models achieve a crucial comparison advantage, outperforming existing state-of-the-art techniques in terms of both visual quality and quantitative metrics. We propose adopting the Learned Per-ceptual Image Patch Similarity metric, which emphasizes perceptual similarity, to evaluate the quality of cloud-free images generated by the models. Our work not only contributes to advancing cloud removal techniques in remote sensing but also provides a comprehensive evaluation framework for assessing the fidelity of the generated images.
- Open Access
- 25 March 2024
- Published by MDPI AG in Applied Sciences
- Vol. 14 (7) , 2729
SatelliteRF: Accelerating 3D Reconstruction in Multi-View Satellite Images with Efficient Neural Radiance Fields
In the field of multi-view satellite photogrammetry, the neural radiance field (NeRF) method has received widespread attention due to its ability to provide continuous scene representation and realistic rendering effects. However, the satellite radiance field methods based on the NeRF are limited by the slow training speed of the original NeRF, and the scene reconstruction efficiency is low. Training for a single scene usually takes 8–10 h or even longer, which severely constrains the utilization and exploration of the NeRF approach within the domain of satellite photogrammetry. In response to the above problems, we propose an efficient neural radiance field method called SatelliteRF, which aims to quickly and efficiently reconstruct the earth’s surface through multi-view satellite images. By introducing innovative multi-resolution hash coding, SatelliteRF enables the model to greatly increase the training speed while maintaining high reconstruction quality. This approach allows for smaller multi-layer perceptron (MLP) networks, reduces the computational cost of neural rendering, and accelerates the training process. Furthermore, to overcome the challenges of illumination changes and transient objects encountered when processing multi-date satellite images, we adopt an improved irradiance model and learn transient embeddings for each image. This not only increases the adaptability of the model to illumination variations but also improves its ability to handle changing objects. We also introduce a loss function based on stochastic structural similarity (SSIM) to provide structural information of the scene for model training, which further improves the quality and detailed performance of the reconstructed scene. Through extensive experiments on the DFC 2019 dataset, we demonstrate that SatelliteRF is not only able to significantly reduce the training time for the same region from the original 8–10 h to only 5–10 min but also achieves better performance in terms of rendering and the reconstruction quality.
- 25 March 2024
- Published by MDPI AG in Applied Sciences
- Vol. 14 (7) , 2729
Open AccessSatelliteRF: Accelerating 3D Reconstruction in Multi-View Satellite Images with Efficient Neural Radiance Fields
In the field of multi-view satellite photogrammetry, the neural radiance field (NeRF) method has received widespread attention due to its ability to provide continuous scene representation and realistic rendering effects. However, the satellite radiance field methods based on the NeRF are limited by the slow training speed of the original NeRF, and the scene reconstruction efficiency is low. Training for a single scene usually takes 8–10 h or even longer, which severely constrains the utilization and exploration of the NeRF approach within the domain of satellite photogrammetry. In response to the above problems, we propose an efficient neural radiance field method called SatelliteRF, which aims to quickly and efficiently reconstruct the earth’s surface through multi-view satellite images. By introducing innovative multi-resolution hash coding, SatelliteRF enables the model to greatly increase the training speed while maintaining high reconstruction quality. This approach allows for smaller multi-layer perceptron (MLP) networks, reduces the computational cost of neural rendering, and accelerates the training process. Furthermore, to overcome the challenges of illumination changes and transient objects encountered when processing multi-date satellite images, we adopt an improved irradiance model and learn transient embeddings for each image. This not only increases the adaptability of the model to illumination variations but also improves its ability to handle changing objects. We also introduce a loss function based on stochastic structural similarity (SSIM) to provide structural information of the scene for model training, which further improves the quality and detailed performance of the reconstructed scene. Through extensive experiments on the DFC 2019 dataset, we demonstrate that SatelliteRF is not only able to significantly reduce the training time for the same region from the original 8–10 h to only 5–10 min but also achieves better performance in terms of rendering and the reconstruction quality.
- Open Access
- 11 January 2024
- Published by MDPI AG in Remote Sensing
- Vol. 16 (2) , 295
HeightFormer: A Multilevel Interaction and Image-Adaptive Classification–Regression Network for Monocular Height Estimation with Aerial Images
Height estimation has long been a pivotal topic within measurement and remote sensing disciplines, with monocular height estimation offering wide-ranging data sources and convenient deployment. This paper addresses the existing challenges in monocular height estimation methods, namely the difficulty in simultaneously achieving high-quality instance-level height and edge reconstruction, along with high computational complexity. This paper presents a comprehensive solution for monocular height estimation in remote sensing, termed HeightFormer, combining multilevel interactions and image-adaptive classification–regression. It features the Multilevel Interaction Backbone (MIB) and Image-adaptive Classification–regression Height Generator (ICG). MIB supplements the fixed sample grid in the CNN of the conventional backbone network with tokens of different interaction ranges. It is complemented by a pixel-, patch-, and feature map-level hierarchical interaction mechanism, designed to relay spatial geometry information across different scales and introducing a global receptive field to enhance the quality of instance-level height estimation. The ICG dynamically generates height partition for each image and reframes the traditional regression task, using a refinement from coarse to fine classification–regression that significantly mitigates the innate ill-posedness issue and drastically improves edge sharpness. Finally, the study conducts experimental validations on the Vaihingen and Potsdam datasets, with results demonstrating that our proposed method surpasses existing techniques.
- 11 January 2024
- Published by MDPI AG in Remote Sensing
- Vol. 16 (2) , 295
Open AccessHeightFormer: A Multilevel Interaction and Image-Adaptive Classification–Regression Network for Monocular Height Estimation with Aerial Images
Height estimation has long been a pivotal topic within measurement and remote sensing disciplines, with monocular height estimation offering wide-ranging data sources and convenient deployment. This paper addresses the existing challenges in monocular height estimation methods, namely the difficulty in simultaneously achieving high-quality instance-level height and edge reconstruction, along with high computational complexity. This paper presents a comprehensive solution for monocular height estimation in remote sensing, termed HeightFormer, combining multilevel interactions and image-adaptive classification–regression. It features the Multilevel Interaction Backbone (MIB) and Image-adaptive Classification–regression Height Generator (ICG). MIB supplements the fixed sample grid in the CNN of the conventional backbone network with tokens of different interaction ranges. It is complemented by a pixel-, patch-, and feature map-level hierarchical interaction mechanism, designed to relay spatial geometry information across different scales and introducing a global receptive field to enhance the quality of instance-level height estimation. The ICG dynamically generates height partition for each image and reframes the traditional regression task, using a refinement from coarse to fine classification–regression that significantly mitigates the innate ill-posedness issue and drastically improves edge sharpness. Finally, the study conducts experimental validations on the Vaihingen and Potsdam datasets, with results demonstrating that our proposed method surpasses existing techniques.
- Open Access
HeightFormer: A Multilevel Interaction and Image-adaptive Classification-regression Network for Monocular Height Estimation with Aerial Images
Height estimation has long been a pivotal topic within measurement and remote sensing disciplines, proving critical for endeavours such as 3D urban modelling, MR and autonomous driving. Traditional methods utilise stereo matching or multisensor fusion, both well-established techniques that typically necessitate multiple images from varying perspectives and adjunct sensors like SAR, leading to substantial deployment costs. Single image height estimation has emerged as an attractive alternative, boasting a larger data source variety and simpler deployment. However, current methods suffer from limitations such as fixed receptive fields, a lack of global information interaction, leading to noticeable instance-level height deviations. The inherent complexity of height prediction can result in a blurry estimation of object edge depth when using mainstream regression methods based on fixed height division. This paper presents a comprehensive solution for monocular height estimation in remote sensing, termed HeightFormer, combining multilevel interactions and image-adaptive classification-regression. It features the Multilevel Interaction Backbone (MIB) and Image-adaptive Classification-regression Height Generator (ICG). MIB supplements the fixed sample grid in CNN of the conventional backbone network with tokens of different interaction ranges. It is complemented by a pixel-, patch-, and feature map-level hierarchical interaction mechanism, designed to relay spatial geometry information across different scales and introducing a global receptive field to enhance the quality of instance-level height estimation. The ICG dynamically generates height partition for each image and reframes the traditional regression task, using a refinement from coarse to fine classification-regression that significantly mitigates the innate ill-posedness issue and drastically improves edge sharpness.
Open AccessHeightFormer: A Multilevel Interaction and Image-adaptive Classification-regression Network for Monocular Height Estimation with Aerial Images
Height estimation has long been a pivotal topic within measurement and remote sensing disciplines, proving critical for endeavours such as 3D urban modelling, MR and autonomous driving. Traditional methods utilise stereo matching or multisensor fusion, both well-established techniques that typically necessitate multiple images from varying perspectives and adjunct sensors like SAR, leading to substantial deployment costs. Single image height estimation has emerged as an attractive alternative, boasting a larger data source variety and simpler deployment. However, current methods suffer from limitations such as fixed receptive fields, a lack of global information interaction, leading to noticeable instance-level height deviations. The inherent complexity of height prediction can result in a blurry estimation of object edge depth when using mainstream regression methods based on fixed height division. This paper presents a comprehensive solution for monocular height estimation in remote sensing, termed HeightFormer, combining multilevel interactions and image-adaptive classification-regression. It features the Multilevel Interaction Backbone (MIB) and Image-adaptive Classification-regression Height Generator (ICG). MIB supplements the fixed sample grid in CNN of the conventional backbone network with tokens of different interaction ranges. It is complemented by a pixel-, patch-, and feature map-level hierarchical interaction mechanism, designed to relay spatial geometry information across different scales and introducing a global receptive field to enhance the quality of instance-level height estimation. The ICG dynamically generates height partition for each image and reframes the traditional regression task, using a refinement from coarse to fine classification-regression that significantly mitigates the innate ill-posedness issue and drastically improves edge sharpness.
- Open Access
- 7 June 2023
- Published by Springer Nature in Complex & Intelligent Systems
- Vol. 9 (6) , 6953-6969
DSC-MVSNet: attention aware cost volume regularization based on depthwise separable convolution for multi-view stereo
Deep learning has recently been proven to deliver excellent performance in multi-view stereo (MVS). However, it is difficult for deep learning-based MVS approaches to balance their efficiency and effectiveness. Towards this end, we propose the DSC-MVSNet, a novel coarse-to-fine and end-to-end framework for more efficient and more accurate depth estimation in MVS. In particular, we propose an attention aware 3D UNet-shape network, which first uses the depthwise separable convolutions for cost volume regularization. This mechanism enables effective aggregation of information and significantly reduces the model parameters and computation by transforming the ordinary convolution on cost volume as depthwise convolution and pointwise convolution. Besides, a 3D-Attention module is proposed to alleviate the feature mismatching problem in cost volume regularization and aggregate the important information of cost volume in three dimensions (i.e. channel, space, and depth). Moreover, we propose an efficient Feature Transfer Module to upsample the low-resolution (LR) depth map to a high-resolution (HR) depth map to achieve higher accuracy. With extensive experiments on two benchmark datasets, i.e. DTU and Tanks & Temples, we demonstrate that the parameters of our model are significantly reduced to $$25\%$$ of the state-of-the-art model MVSNet. Besides, our method outperforms or maintains on par accuracy with the state-of-the-art models. Our source code is available at https://github.com/zs670980918/DSC-MVSNet.- 7 June 2023
- Published by Springer Nature in Complex & Intelligent Systems
- Vol. 9 (6) , 6953-6969
Open AccessDSC-MVSNet: attention aware cost volume regularization based on depthwise separable convolution for multi-view stereo
Deep learning has recently been proven to deliver excellent performance in multi-view stereo (MVS). However, it is difficult for deep learning-based MVS approaches to balance their efficiency and effectiveness. Towards this end, we propose the DSC-MVSNet, a novel coarse-to-fine and end-to-end framework for more efficient and more accurate depth estimation in MVS. In particular, we propose an attention aware 3D UNet-shape network, which first uses the depthwise separable convolutions for cost volume regularization. This mechanism enables effective aggregation of information and significantly reduces the model parameters and computation by transforming the ordinary convolution on cost volume as depthwise convolution and pointwise convolution. Besides, a 3D-Attention module is proposed to alleviate the feature mismatching problem in cost volume regularization and aggregate the important information of cost volume in three dimensions (i.e. channel, space, and depth). Moreover, we propose an efficient Feature Transfer Module to upsample the low-resolution (LR) depth map to a high-resolution (HR) depth map to achieve higher accuracy. With extensive experiments on two benchmark datasets, i.e. DTU and Tanks & Temples, we demonstrate that the parameters of our model are significantly reduced to $$25\%$$ of the state-of-the-art model MVSNet. Besides, our method outperforms or maintains on par accuracy with the state-of-the-art models. Our source code is available at https://github.com/zs670980918/DSC-MVSNet.