Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speech
Authors:
Preprint
Abstract
In this paper, we propose a novel deep neural network architecture, Speech2Vec, for learning fixed-length vector representations of audio segments excised from a speech corpus, where the vectors contain semantic information pertaining to the underlying spoken words, and are close to other vectors in the embedding space if their corresponding underlying spoken words are semantically similar. The proposed model can be viewed as a speech version of Word2Vec. Its design is based on a RNN Encoder-Decoder framework, and borrows the methodology of skipgrams or continuous bag-of-words for training. Learning word embeddings directly from speech enables Speech2Vec to make use of the semantic information carried by speech that does not exist in plain text. The learned word embeddings are evaluated and analyzed on 13 widely used word similarity benchmarks, and outperform word embeddings learned by Word2Vec from the transcriptions.Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- Enriching Word Vectors with Subword InformationTransactions of the Association for Computational Linguistics, 2017
- An embedded segmental K-means model for unsupervised segmentation and clustering of speech2017
- A segmental framework for fully-unsupervised large-vocabulary speech recognitionComputer Speech & Language, 2017
- Deep convolutional acoustic word embeddings using word-pair side information2016
- Deep multimodal semantic embeddings for speech and images2015
- SimLex-999: Evaluating Semantic Models With (Genuine) Similarity EstimationComputational Linguistics, 2015
- Librispeech: An ASR corpus based on public domain audio books2015
- Fixed-dimensional acoustic embeddings of variable-length segments in low-resource settings2013
- Large-scale learning of word relatedness with constraints2012
- Contextual correlates of synonymyCommunications of the ACM, 1965