DiDOTS: Knowledge Distillation from Large-Language-Models for Dementia Obfuscation in Transcribed Speech
Open Access
- 1 January 2025
- journal article
- Published by Privacy Enhancing Technologies Symposium Advisory Board in Proceedings on Privacy Enhancing Technologies
- Vol. 2025 (1) , 198-215
Abstract
Dementia is a sensitive neurocognitive disorder affecting tens of millions of people worldwide and its cases are expected to triple by 2050. Alarmingly, recent advancements in dementia classification make it possible for adversaries to violate affected individuals’ privacy and infer their sensitive condition from speech transcriptions. Existing obfuscation methods in text have never been applied for dementia and depend on the availability of large labeled datasets which are challenging to collect for sensitive medical attributes. In this work, we bridge this research gap and tackle the above issues by leveraging Large-Language-Models (LLMs) with diverse prompt designs (zero-shot, few-shot, and knowledge-based) to obfuscate dementia in speech transcripts. Our evaluation shows that LLMs are more effective dementia obfuscators compared to competing methods. However, they have billions of parameters which renders them hard to train, store and share, and they are also fragile suffering from hallucination, refusal and contradiction effects among others. To further mitigate these, we propose a novel method, DiDOTS. DiDOTS distills knowledge from LLMs using a teacher–student paradigm and parameter-efficient fine-tuning. DiDOTS has one order of magnitude fewer parameters compared to its teacher LLM and can be fine-tuned using three orders of magnitude less parameters compared to full fine-tuning. Our evaluation shows that compared to prior work DiDOTS retains the performance of LLMs achieving 1.3x and 2.2x improvement in privacy performance on two datasets, while humans rate it as better in preserving utility even when compared to state-of-the-art paraphrasing models.Keywords
All Related Versions
This publication has 36 references indexed in Scilit:
- ADscreen: A speech processing-based screening system for automatic identification of patients with Alzheimer's disease and related dementiaArtificial Intelligence in Medicine, 2023
- Enriching Word Vectors with Subword InformationTransactions of the Association for Computational Linguistics, 2017
- A meta-analysis of correlations between depression and first person singular pronoun useJournal of Research in Personality, 2017
- An Empirical Analysis of Formality in Online CommunicationTransactions of the Association for Computational Linguistics, 2016
- Optimizing Statistical Machine Translation for Text SimplificationTransactions of the Association for Computational Linguistics, 2016
- Android permissions2012
- Longitudinal detection of dementia through lexical and syntactic changes in writing: a case study of three British novelistsLiterary and Linguistic Computing, 2011
- The Meteor metric for automatic evaluation of machine translationMachine Translation, 2009
- LANGUAGE AND DEMENTIA: NEUROPSYCHOLOGICAL ASPECTSAnnual Review of Applied Linguistics, 2008
- Automatically Categorizing Written Texts by Author GenderLiterary and Linguistic Computing, 2002