Baihua Xiao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Baihua Xiao has a total of 69 co-authors. Here are the top co-authors by number of shared publications:
- Zhong Zhang
Tianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal University, Tianjin, China
16 shared publications - Shuang Liu
Tianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal University, Tianjin, China
15 shared publications - Chunheng Wang
Institute of Automation Chinese Academy of Sciences(CASIA)
11 shared publications - Cunzhao Shi
Institute of Automation Chinese Academy of Sciences(CASIA)
6 shared publications - Tariq S. Durrani
Department of Electronic and Electrical Engineering, University of Strathclyde, Glasgow, U.K.
6 shared publications - Chenbin Liang
Hangzhou Institute of Technology, Xidian University, Hangzhou, China
4 shared publications
Latest Publications
- Open Access
- 17 October 2026
- Published by Springer Nature in Scientific Data
TyphoonSat: A high-resolution satellite imagery dataset for tropical cyclone detection in the Northwest Pacific (2015–2024)
Tropical cyclones (TCs) in the Northwest Pacific pose significant threats to coastal regions, yet the lack of a standardized object-detection dataset hinders fair algorithm evaluation. We present TyphoonSat, a high-resolution satellite imagery dataset for TC detection spanning 2015–2024. It comprises 5390 Himawari-8/9 infrared cloud images (1601 × 1201 pixels, 5 km resolution) covering 100°E–180°E and 0°–60°N, with 7895 bounding-box annotations in YOLO format, derived from best-track records from the China Meteorological Administration (CMABST). The dataset supports coarse (all intensities regarded as one category) and fine-grained (six intensity levels) detection tasks. Benchmarks using YOLOv8 variants demonstrate strong performance for coarse detection (mAP@50 up to 0.923, CSI up to 0.791) and highlight challenges in distinguishing weak-to-moderate intensities, while SuperTY detection excels (CSI up to 0.609). This dataset provides a unified benchmark for TC detection and fosters advances in deep-learning-based meteorological remote sensing.
- 17 October 2026
- Published by Springer Nature in Scientific Data
Open AccessTyphoonSat: A high-resolution satellite imagery dataset for tropical cyclone detection in the Northwest Pacific (2015–2024)
Tropical cyclones (TCs) in the Northwest Pacific pose significant threats to coastal regions, yet the lack of a standardized object-detection dataset hinders fair algorithm evaluation. We present TyphoonSat, a high-resolution satellite imagery dataset for TC detection spanning 2015–2024. It comprises 5390 Himawari-8/9 infrared cloud images (1601 × 1201 pixels, 5 km resolution) covering 100°E–180°E and 0°–60°N, with 7895 bounding-box annotations in YOLO format, derived from best-track records from the China Meteorological Administration (CMABST). The dataset supports coarse (all intensities regarded as one category) and fine-grained (six intensity levels) detection tasks. Benchmarks using YOLOv8 variants demonstrate strong performance for coarse detection (mAP@50 up to 0.923, CSI up to 0.791) and highlight challenges in distinguishing weak-to-moderate intensities, while SuperTY detection excels (CSI up to 0.609). This dataset provides a unified benchmark for TC detection and fosters advances in deep-learning-based meteorological remote sensing.
- 8 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 64 (01962892) , 5621413
Cloud-Type Classification Using Vision-Language Model With Cloud Multimodal Adapter
The existing methods for cloud-type classification primarily rely on cloud observation data containing low-level semantic information, e.g., cloud images and millimeter-wave cloud radar (MMCR) observations, which limits the ability to bridge the gap between original cloud data and human cognitive understanding. In this paper, we apply Large Language Model (LLM) to generate cloud description texts which include high-level semantic information, and we integrate cloud images, MMCR observations and cloud description texts for cloud-type classification. To effectively fuse the information across modalities, we propose a novel adapter named Cloud Multi-Modal Adapter (CMMA) for Vision Language Model (VLM) fine-tuning. The proposed CMMA could adaptively fuse the image features with both radar and text features extracted from corresponding adapters, which enhances the fusion across different modalities and improves the model’s ability to learn from multi-modal data. Meanwhile, we propose the Tri-Modalities Contrastive Loss (TCL) to enhance cross-modal alignment while preserving intra-modal consistency. Extensive experiments demonstrate the effectiveness of our approach.
- 8 May 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 64 (01962892) , 5621413
Cloud-Type Classification Using Vision-Language Model With Cloud Multimodal Adapter
The existing methods for cloud-type classification primarily rely on cloud observation data containing low-level semantic information, e.g., cloud images and millimeter-wave cloud radar (MMCR) observations, which limits the ability to bridge the gap between original cloud data and human cognitive understanding. In this paper, we apply Large Language Model (LLM) to generate cloud description texts which include high-level semantic information, and we integrate cloud images, MMCR observations and cloud description texts for cloud-type classification. To effectively fuse the information across modalities, we propose a novel adapter named Cloud Multi-Modal Adapter (CMMA) for Vision Language Model (VLM) fine-tuning. The proposed CMMA could adaptively fuse the image features with both radar and text features extracted from corresponding adapters, which enhances the fusion across different modalities and improves the model’s ability to learn from multi-modal data. Meanwhile, we propose the Tri-Modalities Contrastive Loss (TCL) to enhance cross-modal alignment while preserving intra-modal consistency. Extensive experiments demonstrate the effectiveness of our approach.
- Open Access
- 6 January 2026
- Published by Elsevier in International Journal of Applied Earth Observation and Geoinformation
- Vol. 146
CAFTrans: Content-Aware Fusion Transformer for ground-based remote sensing cloud detection
- 6 January 2026
- Published by Elsevier in International Journal of Applied Earth Observation and Geoinformation
- Vol. 146
Open AccessCAFTrans: Content-Aware Fusion Transformer for ground-based remote sensing cloud detection
- Open Access
- 17 November 2025
- Published by Springer Nature in Research Square
Cloud Imagery Generation by Physics-Constrained Diffusion Networks
Satellite cloud imagery prediction holds significant application value in meteorology, aviation safety, and renewable energy management. However, due to the complex and nonlinear dynamics of cloud systems, existing deep learning methods often produce blurry, distorted, or flickering results with temporal inconsistencies when generating long-term, high-fidelity predictions. To address these challenges, this paper proposes a novel multi-stage training conditional diffusion prediction model. Our framework leverages the powerful generative modeling capabilities of diffusion models to produce cloud image frames with rich texture details, while incorporating a pre-trained RAFT optical flow estimator to introduce motion consistency loss that explicitly constrains the dynamic evolution between consecutive frames, thereby effectively enhancing temporal coherence in sequences. Furthermore, to overcome training instability in complex systems, we design an innovative three-stage progressive training strategy that gradually guides the model through fundamental content generation, dynamic consistency modeling, and realism optimization via adversarial training. Extensive experiments on real satellite cloud imagery datasets demonstrate that our method significantly outperforms existing models across multiple quantitative metrics and qualitative visual assessments. The generated prediction sequences not only exhibit superior visual clarity and realism but also show remarkable temporal consistency, validating the effectiveness and superiority of our framework for complex spatiotemporal prediction tasks. This work provides a robust and efficient novel approach for integrating generative models with physical constraints to achieve high-quality spatiotemporal forecasting.
- 17 November 2025
- Published by Springer Nature in Research Square
Open AccessCloud Imagery Generation by Physics-Constrained Diffusion Networks
Satellite cloud imagery prediction holds significant application value in meteorology, aviation safety, and renewable energy management. However, due to the complex and nonlinear dynamics of cloud systems, existing deep learning methods often produce blurry, distorted, or flickering results with temporal inconsistencies when generating long-term, high-fidelity predictions. To address these challenges, this paper proposes a novel multi-stage training conditional diffusion prediction model. Our framework leverages the powerful generative modeling capabilities of diffusion models to produce cloud image frames with rich texture details, while incorporating a pre-trained RAFT optical flow estimator to introduce motion consistency loss that explicitly constrains the dynamic evolution between consecutive frames, thereby effectively enhancing temporal coherence in sequences. Furthermore, to overcome training instability in complex systems, we design an innovative three-stage progressive training strategy that gradually guides the model through fundamental content generation, dynamic consistency modeling, and realism optimization via adversarial training. Extensive experiments on real satellite cloud imagery datasets demonstrate that our method significantly outperforms existing models across multiple quantitative metrics and qualitative visual assessments. The generated prediction sequences not only exhibit superior visual clarity and realism but also show remarkable temporal consistency, validating the effectiveness and superiority of our framework for complex spatiotemporal prediction tasks. This work provides a robust and efficient novel approach for integrating generative models with physical constraints to achieve high-quality spatiotemporal forecasting.
- 6 November 2025
- Published by Elsevier in Knowledge-Based Systems
- Vol. 331
SDLFusion: A salient-aware differentiated learning network for infrared and visible image fusion
- 6 November 2025
- Published by Elsevier in Knowledge-Based Systems
- Vol. 331
SDLFusion: A salient-aware differentiated learning network for infrared and visible image fusion
- Open Access
- 26 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 18 (19391404) , 25192-25203
Hierarchical Fusion Transformer for Multimodal Ground-Based Cloud Type Classification
Existing methods for multimodal ground-based cloud type classification are dominated by Convolutional Neural Networks (CNNs), and it fails to capture long-range dependencies. In this paper, we propose a novel Transformer-based architecture named Hierarchical Fusion Transformer (HFT) for multimodal ground-based cloud type classification, which leverages the advantages of self-attention and cross-attention to learn long-range dependencies and effectively fuse cloud images and meteorological element information. Specifically, we propose Visual and Meteorological Joint-Transformer (VM Joint-Trans) to capture global context across modalities, and present Visual and Meteorological Cross-Transformer (VM Cross-Trans) to align different modalities and reduce their inconsistencies. We design a hierarchical architecture to perform comprehensive fusion using VM Joint-Trans and VM Cross-Trans. Meanwhile, we propose the novel Multimodal Contrastive Learning (MCL) which not only constrains the tokens of cloud images and meteorological element information in the same layer, but also the tokens from the same modality in different layers, thereby improving the discriminative ability of model and reducing the modality gap. Furthermore, we release the Large-scale Multimodal Ground-based Cloud Database (LMGCD), containing 10,000 multimodal samples with seven categories. To the best of our knowledge, it is the largest database for multimodal ground-based cloud type classification. Experimental results validate the effectiveness of the proposed HFT for multimodal ground-based cloud type classification.
- 26 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 18 (19391404) , 25192-25203
Open AccessHierarchical Fusion Transformer for Multimodal Ground-Based Cloud Type Classification
Existing methods for multimodal ground-based cloud type classification are dominated by Convolutional Neural Networks (CNNs), and it fails to capture long-range dependencies. In this paper, we propose a novel Transformer-based architecture named Hierarchical Fusion Transformer (HFT) for multimodal ground-based cloud type classification, which leverages the advantages of self-attention and cross-attention to learn long-range dependencies and effectively fuse cloud images and meteorological element information. Specifically, we propose Visual and Meteorological Joint-Transformer (VM Joint-Trans) to capture global context across modalities, and present Visual and Meteorological Cross-Transformer (VM Cross-Trans) to align different modalities and reduce their inconsistencies. We design a hierarchical architecture to perform comprehensive fusion using VM Joint-Trans and VM Cross-Trans. Meanwhile, we propose the novel Multimodal Contrastive Learning (MCL) which not only constrains the tokens of cloud images and meteorological element information in the same layer, but also the tokens from the same modality in different layers, thereby improving the discriminative ability of model and reducing the modality gap. Furthermore, we release the Large-scale Multimodal Ground-based Cloud Database (LMGCD), containing 10,000 multimodal samples with seven categories. To the best of our knowledge, it is the largest database for multimodal ground-based cloud type classification. Experimental results validate the effectiveness of the proposed HFT for multimodal ground-based cloud type classification.
- 16 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 63 (01962892) , 1-13
Cloud-Type Classification Using Multimodal Integration Transformer Based on Cloud Images and Millimeter-Wave Cloud Radar Observations
The existing methods fail to simultaneously utilize the appearance information and the internal structure of clouds for cloud type classification, resulting in incomplete cloud representation. In this paper, we exploit cloud images and Millimeter-wave Cloud Radar (MMCR) observations for cloud type classification, and propose a novel Transformer network named Multi-modal Integration Transformer (MMITrans) to describe completed information of clouds. To this end, we design MMITrans as three subnetworks, i.e., vision subnetwork, MMCR subnetwork and multi-modal fusion network. Specifically, we extract the visual features from the cloud images through the vision subnetwork. Meanwhile, we first convert MMCR observations into several cloud-related indicators, and propose the Indicator-Tokenization to effectively tokenize them and obtain the indicator features using the MMCR subnetwork. Furthermore, we propose the Multi-modal Cross Attention in the multi-modal fusion network to sufficiently fuse the visual features and the indicator features in a multiple-input way. We perform a series of experiments on Cloud images and Millimeter-wave cloud radar observations Dataset, i.e., CMD-Beijing and CMD-Gansu, and the experimental results demonstrate the superiority of the proposed MMITrans.
- 16 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Geoscience and Remote Sensing
- Vol. 63 (01962892) , 1-13
Cloud-Type Classification Using Multimodal Integration Transformer Based on Cloud Images and Millimeter-Wave Cloud Radar Observations
The existing methods fail to simultaneously utilize the appearance information and the internal structure of clouds for cloud type classification, resulting in incomplete cloud representation. In this paper, we exploit cloud images and Millimeter-wave Cloud Radar (MMCR) observations for cloud type classification, and propose a novel Transformer network named Multi-modal Integration Transformer (MMITrans) to describe completed information of clouds. To this end, we design MMITrans as three subnetworks, i.e., vision subnetwork, MMCR subnetwork and multi-modal fusion network. Specifically, we extract the visual features from the cloud images through the vision subnetwork. Meanwhile, we first convert MMCR observations into several cloud-related indicators, and propose the Indicator-Tokenization to effectively tokenize them and obtain the indicator features using the MMCR subnetwork. Furthermore, we propose the Multi-modal Cross Attention in the multi-modal fusion network to sufficiently fuse the visual features and the indicator features in a multiple-input way. We perform a series of experiments on Cloud images and Millimeter-wave cloud radar observations Dataset, i.e., CMD-Beijing and CMD-Gansu, and the experimental results demonstrate the superiority of the proposed MMITrans.
- Open Access
- 29 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 18 (19391404) , 19563-19576
Ground-Based Remote Sensing Cloud Image Segmentation Using Convolution-MLP Network
Recently, MLPs-based methods in computer vision have attracted much attention due to the ability of learning long-range dependencies. However, MLPs-based methods usually treat all the tokens equally, which is difficult to segment challenging cloud regions. In this paper, we propose a novel network named Convolution-MLP Network (Con-MLPNet) for ground-based remote sensing cloud image segmentation, which could effectively learn long-range dependencies via the combination of MLPs and the attention mechanism. To this end, we propose the Attention Guided MLPs (AGMs) module to highlight salient features and suppress irrelevant features from the spatial and channel aspects. Meanwhile, different from existing MLPs methods where the long-range dependencies are learned from one single scale, we propose the Dilated MLPs (DMLPs) to learn long-range dependencies at different scales by sampling different channels of tokens. Furthermore, we design the Parallel Dilated MLPs (PDMs) module to integrate multiple DMLPs with different parameters in order to extract multi-scale information. We conduct a series of experiments on three public ground-based cloud image segmentation datasets, i.e., TLCDD, SWIMSEG and TCDD, and the results demonstrate that the proposed Con-MLPNet achieves state-of-the-art performance. Specifically, on the TLCDD dataset, our method surpasses the competing method across all five evaluation metrics, with the improvements of 3.3% in Precision, 2.48% in Recall, 3.74% in F-score, 1.76% in Accuracy, and 4.0% in IoU over the second-best results.
- 29 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 18 (19391404) , 19563-19576
Open AccessGround-Based Remote Sensing Cloud Image Segmentation Using Convolution-MLP Network
Recently, MLPs-based methods in computer vision have attracted much attention due to the ability of learning long-range dependencies. However, MLPs-based methods usually treat all the tokens equally, which is difficult to segment challenging cloud regions. In this paper, we propose a novel network named Convolution-MLP Network (Con-MLPNet) for ground-based remote sensing cloud image segmentation, which could effectively learn long-range dependencies via the combination of MLPs and the attention mechanism. To this end, we propose the Attention Guided MLPs (AGMs) module to highlight salient features and suppress irrelevant features from the spatial and channel aspects. Meanwhile, different from existing MLPs methods where the long-range dependencies are learned from one single scale, we propose the Dilated MLPs (DMLPs) to learn long-range dependencies at different scales by sampling different channels of tokens. Furthermore, we design the Parallel Dilated MLPs (PDMs) module to integrate multiple DMLPs with different parameters in order to extract multi-scale information. We conduct a series of experiments on three public ground-based cloud image segmentation datasets, i.e., TLCDD, SWIMSEG and TCDD, and the results demonstrate that the proposed Con-MLPNet achieves state-of-the-art performance. Specifically, on the TLCDD dataset, our method surpasses the competing method across all five evaluation metrics, with the improvements of 3.3% in Precision, 2.48% in Recall, 3.74% in F-score, 1.76% in Accuracy, and 4.0% in IoU over the second-best results.
- 21 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 5119-5129
Completed Interaction Networks for Pedestrian Trajectory Prediction
The social and environmental interactions, as well as the pedestrian goal are crucial for pedestrian trajectory prediction. This is because they could learn both complex interactions in the scenes and the intentions of the pedestrians. However, most existing methods either learn the one-moment social interactions, or supervise the pedestrian trajectories using long-term goal, resulting in suboptimal prediction performances. In this paper, we propose a novel network named Completed Interaction Network (CINet) to simultaneously consider the social interactions in all moments, the environmental interactions and the short-term goal of pedestrians in a unified framework for pedestrian trajectory prediction. Specifically, we propose the Spatio-Temporal Transformer Layer (STTL) to fully mine the spatio-temporal information among historical trajectories of all pedestrians in order to obtain the social interactions in all moments. Additionally, we present the Gradual Goal Module (GGM) to capture the environmental interactions under the supervision of the short-term goal, which is beneficial to understanding the intentions of the pedestrian. Afterwards, we employ the cross-attention to effectively integrate the all-moment social and environmental interactions. The experimental results on three standard pedestrian datasets, i.e., ETH/UCY, SDD and inD demonstrate that our method achieves a new state-of-the-art performance. Furthermore, the visualization results indicate that our method could predict trajectories more reasonably in complex scenarios such as sharp turns, infeasible areas and so on.
- 21 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 5119-5129
Completed Interaction Networks for Pedestrian Trajectory Prediction
The social and environmental interactions, as well as the pedestrian goal are crucial for pedestrian trajectory prediction. This is because they could learn both complex interactions in the scenes and the intentions of the pedestrians. However, most existing methods either learn the one-moment social interactions, or supervise the pedestrian trajectories using long-term goal, resulting in suboptimal prediction performances. In this paper, we propose a novel network named Completed Interaction Network (CINet) to simultaneously consider the social interactions in all moments, the environmental interactions and the short-term goal of pedestrians in a unified framework for pedestrian trajectory prediction. Specifically, we propose the Spatio-Temporal Transformer Layer (STTL) to fully mine the spatio-temporal information among historical trajectories of all pedestrians in order to obtain the social interactions in all moments. Additionally, we present the Gradual Goal Module (GGM) to capture the environmental interactions under the supervision of the short-term goal, which is beneficial to understanding the intentions of the pedestrian. Afterwards, we employ the cross-attention to effectively integrate the all-moment social and environmental interactions. The experimental results on three standard pedestrian datasets, i.e., ETH/UCY, SDD and inD demonstrate that our method achieves a new state-of-the-art performance. Furthermore, the visualization results indicate that our method could predict trajectories more reasonably in complex scenarios such as sharp turns, infeasible areas and so on.
- 22 February 2024
- Published by Elsevier in Neurocomputing
- Vol. 579
A comprehensive review of image retargeting
- 22 February 2024
- Published by Elsevier in Neurocomputing
- Vol. 579
A comprehensive review of image retargeting