Benchmarking Automated and Semi-Automated Vocal Clustering Methods
Open Access
- 13 January 2026
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 16 (2) , 810
Abstract
Analyzing large datasets of animal vocalizations requires efficient bioacoustic methods for categorizing the vocalization types. This study evaluates the effectiveness of different vocalization clustering methods, comparing fully automated and semi-automated methods against the gold standard of manual expert annotations. Effective methods achieve good clustering performance whilst minimizing human effort. We release a new dataset of 1454 zebra finch vocalizations manually clustered by experts, on which we evaluate (i) fully automated clustering using off-the-shelf methods based on sound embeddings and (ii) a semi-automated workflow relying on refining the embedding-derived clusters. Clustering performance is assessed using the Adjusted Rand Index (ARI) and Normalized Mutual Information (NMI). Results indicate that while fully automated methods provide a useful baseline, they generally fall short of human-level consistency. In contrast, the semi-automated workflow achieved agreement scores comparable to inter-expert reliability, approaching the levels of expert manual clustering. This demonstrates that refining embedding-derived clusters reduces annotation time while maintaining gold standard accuracy. We conclude that semi-automated workflows offer an optimal strategy for bioacoustics, enabling the scalable analysis of large datasets without compromising the precision required for robust behavioral insights.Keywords
Funding Information
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (31003A_182638)
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (205320_215494/1)
This publication has 26 references indexed in Scilit:
- Categorical and semantic perception of the meaning of call types in zebra finchesScience, 2025
- Positive Transfer of the Whisper Speech Transformer to Human and Animal Voice Activity Detection2024
- Sparse ensemble neural code for a complete vocal repertoireCell Reports, 2023
- Unsupervised classification to improve the quality of a bird song recording datasetEcological Informatics, 2022
- Parameterizing animal sounds and motion with animal-attached tags to study acoustic communicationBehavioral Ecology and Sociobiology, 2022
- Canary Song Decoder: Transduction and Implicit Segmentation with ESNs and LTSMs2021
- Koe: Web‐based software to classify acoustic units and analyse sequence structure in animal vocalizationsMethods in Ecology and Evolution, 2020
- The vocal repertoire of the domesticated zebra finch: a data-driven approach to decipher the information-bearing acoustic features of communication signalsAnimal Cognition, 2015
- 7. Quadratic Assignment Problems: Formulations and Bounds2012
- The Hungarian method for the assignment problemNaval Research Logistics Quarterly, 1955