Benchmarking Local Large Language Models for Predicting Vestibular Dysfunction Based on Caloric Testing: A Comparison with Conventional Machine Learning on Stabilometry Data
Open Access
- 26 October 2026
- journal article
- research article
- Published by MDPI AG in Audiology Research
- Vol. 16 (6) , 162
Abstract
Background/Objectives: In-context learning enables large language models (LLMs) to perform clinical prediction without task-specific training; however, their performance on structured tabular data relative to conventional machine learning remains unclear. This exploratory study evaluated the performance, scalability, and efficiency of local LLMs for predicting caloric-test-defined vestibular dysfunction from stabilometry data. Methods: We analyzed stabilometric data from 238 participants (75 with vestibular dysfunction, 163 controls). Forty-three local LLMs spanning multiple families were evaluated using in-context learning on a local setup using CPU-dominant inference with partial GPU offloading, without external data transmission. Performance and inference time were measured via repeated hold-out validation and compared with logistic regression (LR) and tree-based ensemble methods. Results: The tree-based ensemble models showed the highest predictive performance. LLM performance was heterogeneous and varied by model family and size. Mid-sized models achieved accuracy comparable to LR, although their performance was not statistically distinguishable from either LR or the majority-class baseline, whereas smaller models frequently failed to produce usable binary outputs. Larger models did not consistently provide proportional gains relative to their computational cost. Conclusions: Local LLMs did not outperform conventional machine learning for maximizing predictive accuracy on this small tabular dataset. However, they may serve as a complementary, privacy-preserving approach that enables deployment without task-specific training. Future work should validate these findings with larger datasets, more robust validation, standardized outcome definitions, and constrained decoding.Keywords
This publication has 37 references indexed in Scilit:
- In-context learning enables multimodal large language models to classify cancer pathology imagesNature Communications, 2024
- Large language models encode clinical knowledgeNature, 2023
- Foundation models for generalist medical artificial intelligenceNature, 2023
- Vertigo and DizzinessOtolaryngologic Clinics of North America, 2018
- Japanese standard for clinical stabilometry assessment: Current status and future directionsAuris Nasus Larynx, 2018
- Clinical stabilometry standardizationGait & Posture, 2013
- Disorders of Balance and Vestibular Function in US AdultsArchives of internal medicine (1960), 2009
- Novel subtype of idiopathic bilateral vestibulopathy: bilateral absence of vestibular evoked myogenic potentials in the presence of normal caloric responsesJournal of Neurology, 2009
- Extent of Lesions in Idiopathic Sudden Hearing Loss With VertigoJAMA Otolaryngology–Head & Neck Surgery, 2005
- Random ForestsMachine Learning, 2001