Probing Acoustic Representations for Phonetic Properties
Authors:
Preprint
Abstract
Pre-trained acoustic representations such as wav2vec and DeCoAR have attained impressive word error rates (WER) for speech recognition benchmarks, particularly when labeled data is limited. But little is known about what phonetic properties these various representations acquire, and how well they encode transferable features of speech. We compare features from two conventional and four pre-trained systems in some simple frame-level phonetic classification tasks, with classifiers trained on features from one version of the TIMIT dataset and tested on features from another. All contextualized representations offered some level of transferability across domains, and models pre-trained on more audio data give better results; but overall, DeCoAR, the system with the simplest architecture, performs best. This type of benchmarking analysis can thus uncover relative strengths of various proposed acoustic representations.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Audio Albert: A Lite Bert for Self-Supervised Learning of Audio Representation2021
- Speech-XLNet: Unsupervised Acoustic Model Pretraining for Self-Attention Networks2020
- What Does a Network Layer Hear? Analyzing Hidden Representations of End-to-End ASR Through Speech Synthesis2020
- Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders2020
- What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties2018
- Audio Word2Vec: Unsupervised Learning of Audio Segment Representations Using Sequence-to-Sequence Autoencoder2016
- Deep convolutional acoustic word embeddings using word-pair side information2016
- Speaker-independent phone recognition using hidden Markov modelsIEEE Transactions on Acoustics, Speech, and Signal Processing, 1989