Correlation of Fréchet Audio Distance With Human Perception of Environmental Audio Is Embedding Dependant
Preprint
Abstract
This paper explores whether considering alternative domain-specific embeddings to calculate the Fréchet Audio Distance (FAD) metric can help the FAD to correlate better with perceptual ratings of environmental sounds. We used embeddings from VGGish, PANNs, MS-CLAP, L-CLAP, and MERT, which are tailored for either music or environmental sound evaluation. The FAD scores were calculated for sounds from the DCASE 2023 Task 7 dataset. Using perceptual data from the same task, we find that PANNs-WGM-LogMel produces the best correlation between FAD scores and perceptual ratings of both audio quality and perceived fit with a Spearman correlation higher than 0.5. We also find that music-specific embeddings resulted in significantly lower results. Interestingly, VGGish, the embedding used for the original Fréchet calculation, yielded a correlation below 0.1. These results underscore the critical importance of the choice of embedding for the FAD metric design.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Adapting Frechet Audio Distance for Generative Music Evaluation2024
- Natural Language Supervision For General-Purpose Audio Representations2024
- CLAP Learning Audio Concepts from Natural Language Supervision2023
- Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation2023
- HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection2022
- FSD50K: An Open Dataset of Human-Labeled Sound EventsIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021
- PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern RecognitionIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020
- Audio Set: An ontology and human-labeled dataset for audio events2017
- CNN architectures for large-scale audio classification2017