HideAndSeg: an AI-based tool with automated prompting for octopus segmentation in natural habitats
Preprint
Abstract
Analyzing octopuses in their natural habitats is challenging due to their camouflage capability, rapid changes in skin texture and color, non-rigid body deformations, and frequent occlusions, all of which are compounded by variable underwater lighting and turbidity. Addressing the lack of large-scale annotated datasets, this paper introduces HideAndSeg, a novel, minimally supervised AI-based tool for segmenting videos of octopuses. It establishes a quantitative baseline for this task. HideAndSeg integrates SAM2 with a custom-trained YOLOv11 object detector. First, the user provides point coordinates to generate the initial segmentation masks with SAM2. These masks serve as training data for the YOLO model. After that, our approach fully automates the pipeline by providing a bounding box prompt to SAM2, eliminating the need for further manual intervention. We introduce two unsupervised metrics - temporal consistency $DICE_t$ and new component count $NC_t$ - to quantitatively evaluate segmentation quality and guide mask refinement in the absence of ground-truth data, i.e., real-world information that serves to train, validate, and test AI models. Results show that HideAndSeg achieves satisfactory performance, reducing segmentation noise compared to the manually prompted approach. Our method can re-identify and segment the octopus even after periods of complete occlusion in natural environments, a scenario in which the manually prompted model fails. By reducing the need for manual analysis in real-world scenarios, this work provides a practical tool that paves the way for more efficient behavioral studies of wild cephalopods.
Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- TMAE-YOLO: precision detection of mud crabs in underwater environmentsAquaculture International, 2025
- YOLO-SAG: An improved wildlife object detection algorithm based on YOLOv8nEcological Informatics, 2024
- A video object segmentation-based fish individual recognition method for underwater complex environmentsEcological Informatics, 2024
- A Comprehensive Review on Deep Learning Architecture for Pre-processing of Underwater ImagesSN Computer Science, 2024
- Dynamic skin behaviors in cephalopodsCurrent Opinion in Neurobiology, 2024
- WilDect-YOLO: An efficient and robust computer vision-based accurate object localization model for automated endangered wildlife detectionEcological Informatics, 2022
- Color Change in Cephalopods2021
- Machine learning to detect marine animals in UAV imagery: effect of morphology, spacing, behaviour and habitatRemote Sensing in Ecology and Conservation, 2021
- End-to-End Object Detection with Transformers2020
- Cephalopod chromatophores: neurobiology and natural historyBiological Reviews, 2001