Data Fusion for Integrative Species Identification Using Deep Learning
Preprint
Abstract
A bstract: DNA analyses have revolutionized species identification and taxonomic work. Yet, persistent challenges arise from little differentiation among species and considerable variation within species, particularly among closely related groups. While images are commonly used as an alternative modality for automated identification tasks, their usability is limited by the same concerns. An integrative strategy, fusing molecular and image data through machine learning, holds significant promise for fine-grained species identification. However, a systematic overview and rigorous statistical testing concerning molecular and image preprocessing and fusion techniques, including practical advice for biologists, are missing so far. We introduce a machine learning scheme that integrates both molecular and image data for species identification. Initially, we systematically assess and compare three different DNA arrangements (aligned, unaligned, SNP-reduced) and two encoding methods (fractional, ordinal). Later, artificial neural networks are used to extract visual and molecular features, and we propose strategies for fusing this information. Specifically, we investigate three strategies: I) fusing directly after feature extraction, II) fusing features that passed through a fully connected layer after feature extraction, and III) fusing the output scores of both unimodal models. We systematically and statistically evaluate these strategies for four eukaryotic datasets, including two plant (Asteraceae, Poaceae) and two animal families (Lycaenidae, Coccinellidae) using Leave-One-Out Cross-Validation (LOOCV). In addition, we developed an approach to understand molecular-and image-specific identification failure. Aligned sequences with nucleotides encoded as decimal number vectors achieved the highest identification accuracy among DNA data preprocessing techniques in all four datasets. Fusing molecular and visual features directly after feature extraction yielded the best results for three out of four datasets (52-99%). Overall, combining DNA with image data significantly increased accuracy in three out of four datasets, with plant datasets showing the most substantial improvement (Asteraceae: +19%, Poaceae: +13.6%). Even for Lycaenidae with high identification accuracy based on molecular data (>96%), a statistically significant improvement (+2.1%) was observed. Detailed analysis of confusion rates between and within genera shows that DNA alone tends to identify the genus correctly, but often fails to recognize the species. The failure to resolve species is alleviated by including image data in the training. This increase in resolution hints at. a hierarchical role of modalities in which molecular data coarsely groups the specimens to then be guided towards a more fine-grained identification by the connected image. We systematically showed and explained, for the first time, that optimizing the preprocessing and integration of molecular and image data otters significant benefits, particularly for genetically similar and morphologically indistinguishable species, enhancing species identification by reducing modality-specific failure rates and information gaps. Our results can inform integration efforts for various organism groups, improving automated identification across a wide range of eukaryotic species.Keywords
All Related Versions
- Published version: Systematic Biology.
This publication has 85 references indexed in Scilit:
- From leaves to labels: Building modular machine learning networks for rapid herbarium specimen analysis with LeafMachine2Applications in Plant Sciences, 2023
- Gray whale detection in satellite imagery using deep learningRemote Sensing in Ecology and Conservation, 2023
- Prokaryotic and eukaryotic promoters identification based on residual network transfer learningBioprocess and Biosystems Engineering, 2022
- Image recognition using convolutional neural networks for classification of honey bee subspeciesApidologie, 2022
- FishResNet: Automatic Fish Classification Approach in Underwater ScenarioSN Computer Science, 2021
- Phylogenomics supported by geometric morphometrics reveals delimitation of sexual species within the polyploid apomictic Ranunculus auricomus complex (Ranunculaceae)Taxon, 2020
- Maximizing fish detection with eDNA metabarcodingEnvironmental DNA, 2020
- Influence of image quality on the identification of psyllids using convolutional neural networksBiosystems Engineering, 2019
- Random Forests2008
- Basic Local Alignment Search ToolJournal of Molecular Biology, 1990