Chain of Visual Perception: Harnessing Multimodal Large Language Models for Zero-shot Camouflaged Object Detection
Open Access
- 28 October 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 8805-8814
Abstract
In this paper, we introduce a novel multimodal camo-perceptive framework (MMCPF) aimed at handling zero-shot Camouflaged Object Detection (COD) by leveraging the powerful capabilities of Multimodal Large Language Models (MLLMs). Recognizing the inherent limitations of current COD methodologies, which predominantly rely on supervised learning models demanding extensive and accurately annotated datasets, resulting in weak generalization, our research proposes a zero-shot MMCPF that circumvents these challenges. Although MLLMs hold significant potential for broad applications, their effectiveness in COD is hindered and they would make misinterpretations of camouflaged objects. To address this challenge, we further propose a strategic enhancement called the Chain of Visual Perception (CoVP), which significantly improves the perceptual capabilities of MLLMs in camouflaged scenes by leveraging both linguistic and visual cues more effectively. We validate the effectiveness of MMCPF on five widely used COD datasets, containing CAMO, COD10K, NC4K, MoCA-Mask and OVCamo. Experiments show that MMCPF can outperform all existing state-of-the-art zero-shot COD methods, and achieve competitive performance compared to weakly-supervised and fully-supervised methods, which demonstrates the potential of MMCPF. The Github link of this paper is https://github.com/luckybird1994/MMCPF.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- Segment Anything in Medical ImagesarXiv, 2024
- Segment Anything2023
- Concealed Object DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2021
- Camouflaged Object Detection and Tracking: A SurveyInternational Journal of Image and Graphics, 2020
- Co-saliency Detection Based on Hierarchical Consistency2019
- Anabranch network for camouflaged object segmentationComputer Vision and Image Understanding, 2019
- Structure-Measure: A New Way to Evaluate Foreground Maps2017