The effect of visual speech information on linguistic release from masking
Open Access
- 1 January 2023
- journal article
- research article
- Published by Acoustical Society of America (ASA) in The Journal of the Acoustical Society of America
- Vol. 153 (1) , 602-612
Abstract
Listeners often experience challenges understanding a person (target) in the presence of competing talkers (maskers). This difficulty reduces with the availability of visual speech information (VSI; lip movements, degree of mouth opening) and during linguistic release from masking (LRM; masking decreases with dissimilar language maskers). We investigate whether and how LRM occurs with VSI. We presented English targets with either Dutch or English maskers in audio-only and audiovisual conditions to 62 American English participants. The signal-to-noise ratio (SNR) was easy at 0 audio-only and −8 dB audiovisual in Experiment 1 and hard at −8 and −16 dB in Experiment 2 to assess the effects of modality on LRM across the same and different SNRs. We found LRM in the audiovisual condition for all SNRs and in audio-only for −8 dB, demonstrating reliable LRM for audiovisual conditions. Results also revealed that LRM is modulated by modality with larger LRM in audio-only indicating that introducing VSI weakens LRM. Furthermore, participants showed higher performance for Dutch maskers compared to English maskers with and without VSI. This establishes that listeners use both VSI and dissimilar language maskers to overcome masking. Our study shows that LRM persists in the audiovisual modality and its strength depends on the modality.Keywords
Funding Information
- National Science Foundation (2126888)
- National Science Foundation (1545900)
This publication has 34 references indexed in Scilit:
- Masking Release Due to Linguistic and Phonetic Dissimilarity Between the Target and Masker SpeechAmerican Journal of Audiology, 2013
- Linguistic contributions to speech-on-speech masking for native and non-native listeners: Language familiarity and semantic contentThe Journal of the Acoustical Society of America, 2012
- Speech-on-speech masking with variable access to the linguistic content of the masker speechThe Journal of the Acoustical Society of America, 2010
- Mixed-effects modeling with crossed random effects for subjects and itemsJournal of Memory and Language, 2008
- Object-based auditory and visual attentionTrends in Cognitive Sciences, 2008
- Aging and Speech-on-Speech MaskingEar & Hearing, 2008
- The foreign language cocktail party problem: Energetic and informational masking effects in non-native speech perceptionThe Journal of the Acoustical Society of America, 2008
- Note on informational masking (L)The Journal of the Acoustical Society of America, 2003
- Audiovisual integration in perception of real wordsPerception & Psychophysics, 1992
- The Bkb (Bamford-Kowal-Bench) Sentence Lists for Partially-Hearing ChildrenBritish Journal of Audiology, 1979