A Convolutional Vision Transformer for Semantic Segmentation of Side-Scan Sonar Data
Preprint
Abstract
Distinguishing among different marine benthic habitat characteristics is of key importance in a wide set of seabed operations ranging from installations of oil rigs to laying networks of cables and monitoring the impact of humans on marine ecosystems. The Side-Scan Sonar (SSS) is a widely used imaging sensor in this regard. It produces high-resolution seafloor maps by logging the intensities of sound waves reflected back from the seafloor. In this work, we leverage these acoustic intensity maps to produce pixel-wise categorization of different seafloor types. We propose a novel architecture adapted from the Vision Transformer (ViT) in an encoder-decoder framework. Further, in doing so, the applicability of ViTs is evaluated on smaller datasets. To overcome the lack of CNN-like inductive biases, thereby making ViTs more conducive to applications in low data regimes, we propose a novel feature extraction module to replace the Multi-layer Perceptron (MLP) block within transformer layers and a novel module to extract multiscale patch embeddings. A lightweight decoder is also proposed to complement this design in order to further boost multiscale feature extraction. With the modified architecture, we achieve state-of-the-art results and also meet real-time computational requirements. We make our code available at ~\url{https://github.com/hayatrajani/s3seg-vitKeywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Swin-Unet: Unet-Like Pure Transformer for Medical Image Segmentation2023
- ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and BeyondInternational Journal of Computer Vision, 2023
- Dual-branch framework: AUV-based target recognition method for marine surveyEngineering Applications of Artificial Intelligence, 2022
- DcNet: Dilated Convolutional Neural Networks for Side-Scan Sonar Image Semantic SegmentationJournal of Ocean University of China, 2021
- Side-scan sonar images segmentation for AUV with recurrent residual convolutional neural network module and self-guidance moduleApplied Ocean Research, 2021
- Searching for MobileNetV32019
- Sidescan Sonar Segmentation Using Texture Descriptors and Active ContoursIEEE Journal of Oceanic Engineering, 2007
- Unsupervised segmentation using a self-organizing map and a noise model estimation in sonar imageryPattern Recognition, 2000
- Sonar image segmentation using an unsupervised hierarchical MRF modelIEEE Transactions on Image Processing, 2000
- Three-Class Markovian Segmentation of High-Resolution Sonar ImagesComputer Vision and Image Understanding, 1999