Paying attention to other animal detections improves camera trap classification models
Open Access
- 4 March 2026
- journal article
- Published by Wiley in Methods in Ecology and Evolution
- Vol. 17 (4) , 1248-1258
Abstract
In ecological studies, automated species classification models are increasingly used to process large volumes of camera trap images. Most current classification models rely on a two‐step pipeline: a detector first locates and crops animals, followed by a classifier that predicts species independently for each crop. While effective, these models still struggle under challenging conditions and ignore temporal context or information from nearby animals available in sequences of camera trap images, whereas human annotators often use it in difficult cases. We propose to leverage self‐attention, a core mechanism of Large Language Models and Vision Transformers, to enable the model to learn relationships between crops occurring in similar contexts. Our self‐attention module operates directly on the set of crop embeddings, producing new representations enriched with information from other crops, improving species classification. The module fits into the two‐step pipeline without requiring structural change and adds only minimal computational overhead. To address the lack of annotated multi‐species sequences, we develop a training strategy that synthetically generates multi‐species sequences from mono‐specific ones. Compared with an independent classifier baseline, our multi‐crop model achieves higher accuracy on mono‐specific sequences, both real and synthetic. In multi‐species settings, evaluated using synthetic test sets, we also observe a substantial improvement in accuracy. Using real but weakly annotated multi‐species sequences, we reformulate the task as multi‐label set classification and conduct a visual analysis to highlight the benefits of our approach. By leveraging the information brought by all detections of animals in the image and others of the same sequence, our approach reduces species misclassifications and enables more accurate estimates for downstream ecological analyses focusing on, for instance, species richness, occupancy and species interaction.Keywords
All Related Versions
Funding Information
- Biodiversa+
This publication has 17 references indexed in Scilit:
- To crop or not to crop: Comparing whole‐image and cropped classification on a large dataset of camera trap imagesIET Computer Vision, 2024
- CoTracker: It Is Better to Track Together2024
- Using the multivariate Hawkes process to study interactions between multiple species from camera trap dataEcology, 2024
- Camtrap DP: an open standard for the FAIR exchange and archiving of camera trap dataRemote Sensing in Ecology and Conservation, 2023
- Being confident in confidence scores: calibration in deep learning models for camera trap image sequencesbioRxiv, 2023
- Wildlife Insights: A Platform to Maximize the Potential of Camera Trap and Other Passive Sensor Wildlife Data for the PlanetEnvironmental Conservation, 2019
- Automatically identifying, counting, and describing wild animals in camera-trap images with deep learningProceedings of the National Academy of Sciences, 2018
- Deep Residual Learning for Image Recognition2016
- Rethinking the Inception Architecture for Computer Vision2016
- Snapshot Serengeti, high-frequency annotated camera trap images of 40 mammalian species in an African savannaScientific Data, 2015