Anesmar Olino de Albuquerque

Training semantic segmentation models is especially costly in remote sensing, since most problems require a new dataset as targets vary with resolution, sensors, and region. Approaches to reduce this cost are increasingly common, but most add machinery that expands few labeled pixels into a denser training signal using the model’s own predictions, where a confident prediction looks the same whether it is correct or incorrect. The hypothesis here is that these confident errors are the most valuable pixels to label, and that a human examining the image can identify them directly. We propose iSAGE (Iterative Sparse Annotation Guided by Expert), an open-source framework in which the annotator clicks confident errors in the prediction overlay, the clicks train the model under an error-weighted loss, and the updated model surfaces the next errors, with no automatic label expansion at any step. With at most one labeled pixel per class per frame per iteration, iSAGE recovers 96.9% of dense performance on BsB Aerial (74.79% mIoU from 0.040% of the pixels) and matches the dense baseline on ISPRS Vaihingen (76.65% vs. 76.93% from 0.011% of the pixels), surpassing published weakly supervised methods by nearly 4 mIoU points. Four automatic selection strategies run through the same loop plateau 7.3 to 14.4 points below iSAGE, and raising their budgets far higher does not close the gap; in a 35-method comparison, iSAGE is the only iterative human-in-the-loop framework without automatic label generation.

The rapid expansion of offshore wind energy requires effective monitoring to balance renewable energy development with environmental and marine spatial planning. This study proposes a novel offshore wind farm detection methodology integrating Sentinel-1 SAR time series, a time-shift augmentation strategy, and semantic-to-instance segmentation transformation. The methodology consists of (1) constructing a dataset with offshore wind farms labeled from Sentinel-1 SAR time series, (2) applying a time-shift augmentation strategy by randomizing image sequences during training (avoiding overfitting due to chronological ordering), (3) evaluating six deep learning architectures (U-Net, U-Net++, LinkNet, DeepLabv3+, FPN, and SegFormer) across time-series lengths of 1, 5, 10, and 15 images, and (4) converting the semantic segmentation results into instance-level detections using Geographic Information System tools. The results show that increasing the time-series length from 1 to 15 images significantly improves performance, with the Intersection over Union increasing from 63.29% to 81.65% and the F-score from 77.52% to 89.90%, using the best model (LinkNet). Also, models trained with time-shift augmentation achieved a 25% higher IoU and an 18% higher F-score than those trained without it. The semantic-to-instance transformation achieved 99.7% overall quality in per-object evaluation, highlighting the effectiveness of our approach.

Karst depressions represent significant geomorphological features crucial in environmental monitoring and conservation. This research aims to detect and classify karst depressions according to their evolutionary stages in the Bambuí Group, located between the Cerrado and Caatinga biomes in Central Brazil, using Planet time series provided by Norway's International Climate and Forest Initiative (NICFI) Satellite Data Program and deep learning‐based semantic segmentation models. A new deep learning training dataset was developed containing manually labelled reference data and a time series of monthly images from Planet NICFI data over a year. The research classified three evolutionary stages of karst depressions: (1) temporary lakes, (2) depressions with concentric halos and subsidence and (3) vegetated depressions. These stages represent distinct geomorphological processes, from initial water accumulation to more advanced stages involving subsidence and vegetation development in the depression areas. The study compared six state‐of‐the‐art semantic segmentation architectures (U‐Net, U‐Net++, DeepLabV3+, LinkNet, FPN and PSPNet), each combined with three backbones (EfficientNet‐B7, ResNet‐101 and ResNeXt‐101), resulting in 18 model configurations. The best performing model (U‐Net with EfficientNet‐B7) achieved a mean Intersection over Union (mIoU) of 80 and IoU scores of 97.9 for the background, 80.93 for first stage, 79.89 for second stage and 63.35 for third stage, highlighting the challenges of detecting more advanced stages due to increasing vegetation cover and geomorphological complexity. The sliding window approach was employed to classify the entire image mosaic, testing various stride values (8, 16, 32, 64, 128 and 256), with smaller strides improving segmentation accuracy at the cost of higher computational demands. The results demonstrate the importance of integrating spectro‐spatio‐temporal data to detect multiple evolutionary stages and improve the robustness of semantic segmentation. This research provides a comprehensive dataset and benchmark for future studies on karst depressions, contributing to understanding geomorphological evolution and conservation planning in Central Brazil.

Offshore wind farms represent a renewable energy source with a significant global growth trend, and their monitoring is strategic for territorial and environmental planning. This study's primary objective is to detect offshore wind plants at an instance level using semantic segmentation models and Sentinel-1 time series. The secondary objectives are: (a) to develop a database consisting of labeled data and S-1 time series; (b) to compare the performance of five deep semantic segmentation architectures (U-Net, U-Net++, Feature Pyramid Network - FPN, DeepLabv3+, and LinkNet); (c) develop a novel augmentation strategy that shuffles the positions of the images within the time series; (d) investigate different dimensions of time series intervals (1, 5, 10, and 15 images); and (e) evaluate the semantic-to-instance conversion procedure. LinkNet was the top-performing model, followed by U-Net++ and U-Net, while FPN and DeepLabv3+ presented the worst results. The evaluation of semantic segmentation models reveals enhanced Intersection over Union (IoU) (25%) and F-score metrics (18%) with the augmentation of time series images. The study showcases the augmentation strategy's capability to mitigate biases and precisely detect invariant targets. Furthermore, the conversion from semantic to instance segmentation demonstrates its efficacy in accurately isolating individual instances within classified regions - simplifying training data and reducing annotation effort and complexity.

Wind energy is one of Brazil’s most promising energy sources, and the rapid growth of wind plants has increased the need for accurate and efficient inspection methods. The current onsite visits, which are laborious and costly, have become unsustainable due to the sheer scale of wind plants across the country. This study proposes a novel data-centric approach integrating semantic segmentation and GIS to obtain instance-level predictions of wind plants by using free orbital satellite images. Additionally, we introduce a new annotation pattern, which includes wind turbines and their shadows, leading to a larger object size. The elaboration of data collection used the panchromatic band of the China–Brazil Earth Resources Satellite (CBERS) 4A, with a 2-m spatial resolution, comprising 21 CBERS 4A scenes and more than 5000 wind plants annotated manually. This database has 5021 patches, each with 128 × 128 spatial dimensions. The deep learning model comparison involved evaluating six architectures and three backbones, totaling 15 models. The sliding windows approach allowed us to classify large areas, considering different pass values to obtain a balance between performance and computational time. The main results from this study include: (1) the LinkNet architecture with the Efficient-Net-B7 backbone was the best model, achieving an intersection over union score of 71%; (2) the use of smaller stride values improves the recognition process of large areas but increases computational power, and (3) the conversion of raster to polygon in GIS platforms leads to highly accurate instance-level predictions. This entire pipeline can be easily applied for mapping wind plants in Brazil and be expanded to other regions worldwide. With this approach, we aim to provide a cost-effective and efficient solution for inspecting and monitoring wind plants, contributing to the sustainability of the wind energy sector in Brazil and beyond.

The state of Amapá within the Amazon biome has a high complexity of ecosystems formed by forests, savannas, seasonally flooded vegetation, mangroves, and different land uses. The present research aimed to map the vegetation from the phenological behavior of the Sentinel-1 time series, which has the advantage of not having atmospheric interference and cloud cover. Furthermore, the study compared three different sets of images (vertical–vertical co-polarization (VV) only, vertical–horizontal cross-polarization (VH) only, and both VV and VH) and different classifiers based on deep learning (long short-term memory (LSTM), Bidirectional LSTM (Bi-LSTM), Gated Recurrent Units (GRU), Bidirectional GRU (Bi-GRU)) and machine learning (Random Forest, Extreme Gradient Boosting (XGBoost), k-Nearest Neighbors, Support Vector Machines (SVMs), and Multilayer Perceptron). The time series englobed four years (2017–2020) with a 12-day revisit, totaling 122 images for each VV and VH polarization. The methodology presented the following steps: image pre-processing, temporal filtering using the Savitsky–Golay smoothing method, collection of samples considering 17 classes, classification using different methods and polarization datasets, and accuracy analysis. The combinations of the VV and VH pooled dataset with the Bidirectional Recurrent Neuron Networks methods led to the greatest F1 scores, Bi-GRU (93.53) and Bi-LSTM (93.29), followed by the other deep learning methods, GRU (93.30) and LSTM (93.15). Among machine learning, the two methods with the highest F1-score values were SVM (92.18) and XGBoost (91.98). Therefore, phenological variations based on long Synthetic Aperture Radar (SAR) time series allow the detailed representation of land cover/land use and water dynamics.

This letter proposes a novel method to obtain panoptic predictions by extending the semantic segmentation task with a few non-learning image processing steps, presenting the following benefits: 1) annotations do not require a specific format [e.g., common objects in context (COCO)]; 2) fewer parameters (e.g., single loss function and no need for object detection parameters); and 3) a more straightforward sliding windows implementation for large image classification (still unexplored for panoptic segmentation). Semantic segmentation models do not individualize touching objects, as their predictions can merge; i.e., a single polygon represents many targets. Our method overcomes this problem by isolating the objects using borders on the polygons that may merge. The data preparation requires generating a one-pixel border, and for unique object identification, we create a list with the isolated polygons, attribute a different value to each one, and use the expanding border (EB) algorithm for those with borders. Although any semantic segmentation model applies, we used the U-Net with three backbones (EfficientNet-B5, EfficientNet-B3, and EfficientNet-B0). The results show that the following hold: 1) the EfficientNet-B5 had the best results with 70% mean intersection over union (mIoU); 2) the EB algorithm presented better results for better models; 3) the panoptic metrics show a high capability of identifying things and stuff with 65 panoptic quality (PQ); and 4) the sliding windows on a $2560\times 2560$ -pixel area has shown promising results, in which the ratio of merged objects by correct predictions was lower than 1% for all classes.

Vehicle classification is a hot computer vision topic, with studies ranging from ground-view to top-view imagery. Top-view images allow understanding city patterns, traffic management, among others. However, there are some difficulties for pixel-wise classification: most vehicle classification studies use object detection methods, and most publicly available datasets are designed for this task, creating instance segmentation datasets is laborious, and traditional instance segmentation methods underperform on this task since the objects are small. Thus, the present research objectives are as follows: first, propose a novel semisupervised iterative learning approach using the geographic information system software, second, propose a box-free instance segmentation approach, and third, provide a city-scale vehicle dataset. The iterative learning procedure considered the following: first, labeling a few vehicles from the entire scene, second, choosing training samples near those areas, third, training the deep learning model (U-net with efficient-net-B7 backbone), fourth, classifying the whole scene, fifth, converting the predictions into shapefile, sixth, correcting areas with wrong predictions, seventh, including them in the training data, eighth repeating until results are satisfactory. We considered vehicle interior and borders to separate instances using a semantic segmentation model. When removing the borders, the vehicle interior becomes isolated, allowing for unique object identification. Our procedure is very efficient and accurate for generating data iteratively, which resulted in 122 567 mapped vehicles. Metrics-wise, our method presented higher intersection over union when compared to box-based methods (82% against 72%), and per-object metrics surpassed 90% for precision and recall.

Panoptic segmentation combines instance and semantic predictions, allowing the detection of countable objects and different backgrounds simultaneously. Effectively approaching panoptic segmentation in remotely sensed data is very promising since it provides a complete classification, especially in areas with many elements as the urban setting. However, some difficulties have prevented the growth of this task: (a) it is very laborious to label large images with many classes, (b) there is no software for generating DL samples in the panoptic segmentation format, (c) remote sensing images are often very large requiring methods for selecting and generating samples, and (d) most available software is not friendly to remote sensing data formats (e.g., TIFF). Thus, this study aims to increase the operability of panoptic segmentation in remote sensing by providing: (1) a pipeline for generating panoptic segmentation datasets, (2) software to create deep learning samples in the Common Objects in Context (COCO) annotation format automatically, (3) a novel dataset, (4) leverage the Detectron2 software for compatibility with remote sensing data, and (5) evaluate this task on the urban setting. The proposed pipeline considers three inputs (original image, semantic image, and panoptic image), and our software uses these inputs alongside point shapefiles to automatically generate samples in the COCO annotation format. We generated 3400 samples with 512 × 512 pixel dimensions and evaluated the dataset using Panoptic-FPN. Besides, the metric analysis considered semantic, instance, and panoptic metrics, obtaining 93.865 mean intersection over union (mIoU), 47.691 Average (AP) Precision, and 64.979 Panoptic Quality (PQ). Our study presents the first effective pipeline for generating panoptic segmentation data for remote sensing targets.