A Scalable Open-Source Framework for Machine Learning-Based Image Collection, Annotation and Classification: A Case Study for Automatic Fish Species Identification
Open Access
- 2 November 2022
- journal article
- research article
- Published by MDPI AG in Sustainability
- Vol. 14 (21) , 14324
Abstract
Citizen science platforms, social media and smart phone applications enable the collection of large amounts of georeferenced images. This provides a huge opportunity in biodiversity and ecological research, but also creates challenges for efficient data handling and processing. Recreational and small-scale fisheries is one of the fields that could be revolutionised by efficient, widely accessible and machine learning-based processing of georeferenced images. Most non-commercial inland and coastal fisheries are considered data poor and are rarely assessed, yet they provide multiple societal benefits and can have substantial ecological impacts. Given that large quantities of georeferenced fish images are being collected by fishers every day, artificial intelligence (AI) and computer vision applications offer a great opportunity to automate their analyses by providing species identification, and potentially also fish size estimation. This would deliver data needed for fisheries management and fisher engagement. To date, however, many AI image analysis applications in fisheries are focused on the commercial sector, limited to specific species or settings, and are not publicly available. In addition, using AI and computer vision tools often requires a strong background in programming. In this study, we aim to facilitate broader use of computer vision tools in fisheries and ecological research by compiling an open-source user friendly and modular framework for large-scale image storage, handling, annotation and automatic classification, using cost- and labour-efficient methodologies. The tool is based on TensorFlow Lite Model Maker library, and includes data augmentation and transfer learning techniques applied to different convolutional neural network models. We demonstrate the potential application of this framework using a small example dataset of fish images taken through a recreational fishing smartphone application. The framework presented here can be used to develop region-specific species identification models, which could potentially be combined into a larger hierarchical model.Keywords
All Related Versions
Funding Information
- European Regional Development Fund (01.2.2-LMT-K-718-02-0006)
This publication has 35 references indexed in Scilit:
- Artificial intelligence in healthcareNature Biomedical Engineering, 2018
- Data augmentation for improving deep learning in image classification problem2018
- Remote electronic monitoring as a potential alternative to on-board observers in small-scale fisheriesBiological Conservation, 2018
- Angler apps as a source of recreational fisheries data: opportunities, challenges and proposed standardsFish and Fisheries, 2016
- Deep Residual Learning for Image Recognition2016
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Status and Solutions for the World’s Unassessed FisheriesScience, 2012
- Computer Vision2012
- Citizen Science as an Ecological Research Tool: Challenges and BenefitsAnnual Review of Ecology, Evolution, and Systematics, 2010
- ImageNet: A large-scale hierarchical image database2009