Window-Based Vision Transformer Network Implementation For Multi-Label Image Classification In Remote Sensing

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Swift process in technology and widespread availability of low-cost internet have led to a substantial rise in data volume in remote sensing, especially for high-resolution and very-high resolution images. Still, these images contain more complex information, and it is not appropriate to analyze the images using a solitary scene-level label while ignoring the distinct features provided by other labels in the images. In multi-label image classification applications, multiple labels are assigned to an image, reflecting various objects or features present in the scene. The classification of these images is critically important for monitoring environmental changes over large geographical areas, disaster management, urban planning, agriculture and forestry management, natural resource conservation, and military intelligence. Nowadays, many methods are used in such image classification problems, primarily deep learning algorithms. However, current deep learning approaches for multi-label remote sensing images often struggle to capture both local fine-grained details and global contextual relationships simultaneously, leaving a gap for models that can efficiently integrate these complementary representations. In this study, advanced neural networks are explored and evaluated for Multi-label AID dataset which contains 3000 images and 17 different labels; AlexNet, VGG16, DenseNet-201, Inception-v3 and ConvNeXt as the CNN models, ViT, SwinT as transformer models and MaxViT as the hybrid model that initially contains both CNN and transformer network. OneCycleLR as scheduler and AsymmetricLoss (ASL) as loss function are employed for each model to systematically evaluate their impact on model performance. MaxViT was chosen because its multi-scale window-based attention can jointly model local and global dependencies, making it particularly suitable for the complex spatial patterns in remote-sensing imagery compared with other hybrid architectures. The window-based MaxViT algorithm, which has not been previously applied to the Multi-label AID dataset in the current literature, has been evaluated. This constitutes the first application of MaxViT to this dataset and provides a novel benchmark for multi-label remote-sensing classification. This algorithm has demonstrated superior performance on this dataset, significantly outperforming existing models and setting a new benchmark with an mAP of 84.98%.

This publication has 26 references indexed in Scilit: