Unsupervised Speech Segmentation: A General Approach Using Speech Language Models
Authors:
Preprint
Abstract
In this paper, we introduce an unsupervised approach for Speech Segmentation, which builds on previously researched approaches, e.g., Speaker Diarization, while being applicable to an inclusive set of acoustic-semantic distinctions, paving a path towards a general Unsupervised Speech Segmentation approach. Unlike traditional speech and audio segmentation, which mainly focuses on spectral changes in the input signal, e.g., phone segmentation, our approach tries to segment the spoken utterance into chunks with differing acoustic-semantic styles, focusing on acoustic-semantic information that does not translate well into text, e.g., emotion or speaker. While most Speech Segmentation tasks only handle one style change, e.g., emotion diarization, our approach tries to handle multiple acoustic-semantic style changes. Leveraging recent advances in Speech Language Models (SLMs), we propose a simple unsupervised method to segment a given speech utterance. We empirically demonstrate the effectiveness of the proposed approach by considering several setups. Results suggest that the proposed method is superior to the evaluated baselines on boundary detection, segment purity, and over-segmentation. Code is available at https://github.com/avishaiElmakies/unsupervised_speech_segmentation_using_slm.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- An Overview of Language Models: Recent Developments and OutlookAPSIPA Transactions on Signal and Information Processing, 2024
- Speech Emotion Diarization: Which Emotion Appears When?2023
- AudioLM: A Language Modeling Approach to Audio GenerationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023
- Word Segmentation on Discovered Phone Units With Dynamic Programming and Self-Supervised ScoringIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2022
- Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentation2020
- Automatic measurement of vowel duration via structured predictionThe Journal of the Acoustical Society of America, 2016
- Automatic Measurement of Voice Onset Time and Prevoicing Using Recurrent Neural Networks2016