Naoko Matsuo

Background: The Functional Independence Measure (FIM) is a standard functional assessment tool in rehabilitation medicine; however, formal assessments are often conducted only at limited intervals. Free-text clinical notes contain more frequent observations of patients’ functional status and may complement these assessments. Objective: This study aimed to estimate item-level FIM scores from Japanese free-text rehabilitation notes using a locally deployed large language model (LLM) and to evaluate estimation performance; item-level estimability; and differences across diseases, therapy types, and admission FIM severity groups. Methods: We retrospectively analyzed free-text notes from physical therapy (PT), occupational therapy (OT), and speech-language pathology (SLP) for patients hospitalized with cerebral infarction (CeI), hip fracture (HF), or vertebral fracture (VBF) at Saiseikai Moriyama Municipal Hospital between October 2019 and September 2024. Using zero-shot prompting, an LLM estimated all FIM items from each note. Estimated scores were matched to reference FIM scores recorded for the same patient on the same calendar date. Performance was evaluated using the mean absolute error (MAE), root mean squared error, and quadratic weighted kappa. Estimation rates and comparisons with modal-score and last-observation-carried-forward (LOCF) baselines were also examined. Results: Among 2,952,702 eligible note-item pairs from 164,039 retained rehabilitation-note-level outputs, the model produced 980,000 valid estimates, corresponding to an overall estimation rate of 33.2%. Among 170,744 note-item opportunities with an available same-day reference score, 57,246 (33.5%) were estimable, of which 77.7% (44,481/57,246) were within 1 point of the reference score. Estimates derived from OT notes had a lower MAE than those derived from PT notes in CeI and HF, whereas the difference was not significant in VBF. The LLM outperformed the modal-score baseline but had a higher MAE than the LOCF overall (0.825 vs 0.648). Conclusions: The locally deployed LLM shows the potential to derive item-level FIM estimates from Japanese rehabilitation notes. Among estimable note-item pairs with a same-day reference FIM score, 77.7% were within 1 point of the reference score. These findings support the potential use of LLM-based estimation to structure functional information contained in routine rehabilitation notes and complement periodic FIM assessments. Further integration of multidisciplinary clinical records and external validation are needed to improve estimability and evaluate its value for longitudinal functional monitoring.

BACKGROUND The Functional Independence Measure (FIM) is a standard functional assessment tool in rehabilitation medicine; however, formal assessments are often conducted only at limited intervals. Free-text clinical notes contain more frequent observations of patients’ functional status and may complement these assessments. OBJECTIVE This study aimed to estimate item-level FIM scores from Japanese free-text rehabilitation notes using a locally deployed large language model (LLM) and to evaluate estimation performance; item-level estimability; and differences across diseases, therapy types, and admission FIM severity groups. METHODS We retrospectively analyzed free-text notes from physical therapy (PT), occupational therapy (OT), and speech-language pathology (SLP) for patients hospitalized with cerebral infarction (CeI), hip fracture (HF), or vertebral fracture (VBF) at Saiseikai Moriyama Municipal Hospital between October 2019 and September 2024. Using zero-shot prompting, an LLM estimated all FIM items from each note. Estimated scores were matched to reference FIM scores recorded for the same patient on the same calendar date. Performance was evaluated using the mean absolute error (MAE), root mean squared error, and quadratic weighted kappa. Estimation rates and comparisons with modal-score and last-observation-carried-forward (LOCF) baselines were also examined. RESULTS Among 2,952,702 eligible note-item pairs from 164,039 retained rehabilitation-note-level outputs, the model produced 980,000 valid estimates, corresponding to an overall estimation rate of 33.2%. Among 170,744 note-item opportunities with an available same-day reference score, 57,246 (33.5%) were estimable, of which 77.7% (44,481/57,246) were within 1 point of the reference score. Estimates derived from OT notes had a lower MAE than those derived from PT notes in CeI and HF, whereas the difference was not significant in VBF. The LLM outperformed the modal-score baseline but had a higher MAE than the LOCF overall (0.825 vs 0.648). CONCLUSIONS The locally deployed LLM shows the potential to derive item-level FIM estimates from Japanese rehabilitation notes. Among estimable note-item pairs with a same-day reference FIM score, 77.7% were within 1 point of the reference score. These findings support the potential use of LLM-based estimation to structure functional information contained in routine rehabilitation notes and complement periodic FIM assessments. Further integration of multidisciplinary clinical records and external validation are needed to improve estimability and evaluate its value for longitudinal functional monitoring.