Eiji Aramaki

Background: The Functional Independence Measure (FIM) is a standard functional assessment tool in rehabilitation medicine; however, formal assessments are often conducted only at limited intervals. Free-text clinical notes contain more frequent observations of patients’ functional status and may complement these assessments. Objective: This study aimed to estimate item-level FIM scores from Japanese free-text rehabilitation notes using a locally deployed large language model (LLM) and to evaluate estimation performance; item-level estimability; and differences across diseases, therapy types, and admission FIM severity groups. Methods: We retrospectively analyzed free-text notes from physical therapy (PT), occupational therapy (OT), and speech-language pathology (SLP) for patients hospitalized with cerebral infarction (CeI), hip fracture (HF), or vertebral fracture (VBF) at Saiseikai Moriyama Municipal Hospital between October 2019 and September 2024. Using zero-shot prompting, an LLM estimated all FIM items from each note. Estimated scores were matched to reference FIM scores recorded for the same patient on the same calendar date. Performance was evaluated using the mean absolute error (MAE), root mean squared error, and quadratic weighted kappa. Estimation rates and comparisons with modal-score and last-observation-carried-forward (LOCF) baselines were also examined. Results: Among 2,952,702 eligible note-item pairs from 164,039 retained rehabilitation-note-level outputs, the model produced 980,000 valid estimates, corresponding to an overall estimation rate of 33.2%. Among 170,744 note-item opportunities with an available same-day reference score, 57,246 (33.5%) were estimable, of which 77.7% (44,481/57,246) were within 1 point of the reference score. Estimates derived from OT notes had a lower MAE than those derived from PT notes in CeI and HF, whereas the difference was not significant in VBF. The LLM outperformed the modal-score baseline but had a higher MAE than the LOCF overall (0.825 vs 0.648). Conclusions: The locally deployed LLM shows the potential to derive item-level FIM estimates from Japanese rehabilitation notes. Among estimable note-item pairs with a same-day reference FIM score, 77.7% were within 1 point of the reference score. These findings support the potential use of LLM-based estimation to structure functional information contained in routine rehabilitation notes and complement periodic FIM assessments. Further integration of multidisciplinary clinical records and external validation are needed to improve estimability and evaluate its value for longitudinal functional monitoring.

Background: Social media offers valuable information for public health surveillance, but detecting valid symptom expressions in noisy and informal text remains challenging. Fine-tuned language models can achieve strong test accuracy while relying on spurious patterns, limiting their reliability in real-world applications. Objective: This study aimed to develop a more robust symptom detection model by incorporating surveillance-relevant attributes into the training objective and introducing a counterfactual-based evaluation framework to assess attribute sensitivity. Methods: We proposed a concept-aware learning framework (CALF), which combines multiattribute counterfactual data augmentation with multitask learning. We compared standard fine-tuning (baseline) with data-level interventions, a model-level intervention, hybrid strategies, and prompt-based methods across multiple encoders and LLMs for data augmentation. Performance was evaluated on the following 3 datasets: the MedWeb test using macro F1-score (m-F1), the MedWeb checklist using flip accuracy (FA), and a supplementary test set using F1-score for flu-or-fever detection (f-F1). FA measures sensitivity to surveillance-relevant attributes, and the supplementary test score serves as an indicator for out-of-distribution performance. Results: The baseline achieved an m-F1-score of 0.871 on the test set but a low overall FA on the checklist (FA=0.342). It also has limited sensitivity to recovery status, timing of occurrence, and uncertainty of expression. Compared with the baseline, CALF has a comparable MedWeb test score (m-F1=0.880 vs 0.871; P=.18), but a substantially higher FA (FA=0.830 vs 0.342; P=.001) and supplementary test score (f-F1=0.856 vs 0.797; P=.007). CALF also improved sensitivity to recovery, timing, and uncertainty across pretrained encoders. The gains were generally larger when GPT-4.1 (OpenAI) was used for data augmentation. Under GPT-4.1, CALF often achieved the highest overall FA among encoder-based methods. When Qwen2.5-7B-Instruct (Alibaba Cloud) was used, an alternative hybrid strategy may achieve comparable or slightly better results. Checklist performance was strongly associated with supplementary test performance (ρ=0.811), whereas MedWeb test performance showed weak association with supplementary test performance (ρ=–0.042). Prompting GPT-4.1 achieved the highest FA overall (FA=0.920) and the highest supplementary test score (f-F1=0.910), but a lower test set score than the baseline (m-F1=0.776 vs 0.871). Conclusions: Standard held-out metrics alone do not adequately characterize robustness to dataset-specific or spuriously correlated features. Models may perform well on the test set while remaining insensitive to attributes that influence whether a symptom mention is relevant to surveillance. The MedWeb checklist complements conventional evaluation by measuring sensitivity to controlled attribute changes and may indicate out-of-distribution performance. CALF improved sensitivity to surveillance-relevant attributes while maintaining strong classification performance, supporting concept-aware learning as a practical strategy for developing more robust encoder-based models.

Background: Symptom detection is essential in global disease surveillance to detect potential outbreaks, as symptoms are the first observable signs of infection. To reflect real-time ground truth conditions during pandemics, social media has emerged as a valuable data source. Moreover, effective digital disease surveillance systems must operate across diverse linguistic settings, and large language models (LLMs) have been shown to perform inconsistently across languages, tending to have lower performance in low-resource languages. While multilingual approaches have been explored in various health-related natural language processing tasks, a critical gap remains in understanding whether LLM-based symptom detection can perform consistently across languages for global disease surveillance. Southeast Asia demonstrates this challenge, combining diverse languages and the potential for emerging infectious disease outbreaks, making it a case for evaluating how multilingual performance disparities manifest in symptom detection. Objective: This study aims to evaluate multilingual disparities in symptom detection using a LLM, as well as the associated error mechanisms across languages and symptom types, to better understand their implications for global disease surveillance. Methods: This study uses the MedWeb dataset, a multilingual pseudo–social media text dataset with multiple symptom labels. The data consist of 12 languages, covering diverse regions and language resource classifications. The symptoms included in this study are fever, headache, runny nose, cough, diarrhea, hay fever, influenza, and cold. We used GPT-5 as the symptom detection system, representing a strong model for health-related tasks. The results are evaluated using the macro precision, recall, and F ₁ -score. An error analysis was conducted to identify the error mechanisms underlying incorrect symptom predictions across languages and to calculate the impact of each error category. Results: Our findings show that performance varied across languages, language resource groups, and symptom types. Southeast Asian (SEA) languages generally achieved lower scores than non-SEA languages, with Japanese obtaining the highest F ₁ -score (0.812) and Lao the lowest (0.716). High-resource languages achieved the most consistent performance, while low-resource languages obtained the lowest overall scores. At the symptom level, diarrhea, headache, cough, and fever showed stable detection across languages, while hay fever, runny nose, influenza, and cold exhibited greater variability. Hay fever showed the widest variability, forming 2 distinct performance clusters aligned with language resource and region classifications. The error analysis revealed 4 misclassification patterns: explicitly mentioned symptoms, cross-lingual variation, symptom overgeneralization, and context misinterpretation. Cross-lingual variation was the most frequent error category, while errors related to explicitly mentioned symptoms showed the largest potential impact on model performance. Conclusions: Due to performance disparities, achieving more reliable and equitable LLM-based symptom detection from social media text for global disease surveillance would benefit from broader representation in training data for low-resource languages, improved cultural and linguistic sensitivity, and stronger contextual understanding of symptom-related expressions.

Culture significantly influences human psychology, making it essential for psychological research, particularly in social psychology, to acknowledge its importance. However, the majority of psychological research ignores or marginalises culture and context. This is problematic because, despite representing a small portion of humanity (specifically Western, Educated, Industrialized, Rich and Democratic (WEIRD) populations) mainstream psychology is published as if it represented the whole humanity. Among the subfields of psychology, social psychology is arguably the one in which this issue should be least prevalent. To investigate this, we conducted a study on titles and abstracts of articles from 69 social psychology journals published between 1990 and 2020, examining how cultural diversity and awareness of its importance changed over time. The results indicate that cultural awareness is more prominent in journals focused on culture, international or local issues; while remaining almost entirely absent in journals focused on theory, reviews and business. Furthermore, cultural awareness displays an imbalance regarding culture: while WEIRD mentions remain most common, mentions of other regions appear to be disproportionately represented or ‘inflated’ in specific contexts. The generalizability of psychological theories will remain impossible if culture is not integrated as a central factor in research.

Background: Workday happiness is associated with workplace performance and burnout, but frequent questionnaire-based assessment is burdensome in real-world workplace settings. Free-text daily reports may provide a lower-burden way to monitor day-to-day changes in workday happiness. Objective: This study aimed to examine whether daily diary text can be used to estimate longitudinal within-person changes in workday happiness and explore text-related factors associated with model performance, such as average sentence length and lexical diversity. Methods: We collected free-text daily reports and self-reported workday happiness scores from employees in 2 Japanese companies. Company A provided training data from 92 participants over 2 months (1725 reports), and company B provided test data from 11 participants over 6 months (652 reports). We used 2 text-based approaches: a bidirectional encoder representations from transformers (BERT)–based regression model trained on company A data and a locally deployed Japanese large language model (Llama) used in a zero-shot setting. Model performance was evaluated for each participant using the Pearson correlation coefficient between self-reported and estimated workday happiness scores, with r =0.40 used as a pragmatic feasibility benchmark. Mean absolute error (MAE) and root mean squared error (RMSE) were also calculated on the original workday happiness scale from 0 to 10. Results: A total of 81.8% (9/11) of the participants met or exceeded the feasibility benchmark of r =0.40. Positive correlations were observed for 81.8% (9/11) of the participants; for 9.1% (1/11) of the participants, the correlation coefficient could not be calculated because the self-reported score remained constant, and 9.1% (1/11) showed a small negative correlation. Participant-level correlations ranged from −0.06 to 0.65 for the BERT model and from −0.05 to 0.81 for the local large language model. Error-based metrics also varied across participants: BERT MAE ranged from 1.04 to 2.78, and RMSE ranged from 1.25 to 3.50, whereas Llama MAE ranged from 0.76 to 4.39, and RMSE ranged from 1.06 to 4.64. Conclusions: Our study suggests that daily report text supports low-burden, longitudinal estimation of individual-level workday happiness. However, performance varied across participants, and further work is needed to improve generalizability, reduce attrition, address possible measurement bias, and clarify appropriate workplace use.

While current evaluations of LLM cultural alignment predominantly rely on static benchmarks in Western contexts, their ability to navigate generative, high-context socio-pragmatic demands in non-Western environments remains critically underexplored. This study investigates how multilingual LLMs adapt to the Japanese workplace—a stringent stress-test environment characterized by strong high-context communication norms and rigid honorific conventions—using Hofstede’s six cultural dimensions as a heuristic framework. We evaluated five state-of-the-art LLMs (LLM-jp, Phi, Llama, Qwen, and GLM) through a large-scale crowdsourced human evaluation. Based on 1,718 valid evaluator sessions, native Japanese raters assessed model outputs to generate a holistic Japanese Workplace Cultural Alignment Score (JWCAS). To dissect the underlying communicative strategies, we paired this with a three-layer diagnostic sub-score analysis (Linguistic Form, Socio-Cultural Values, and Social Action). Our results reveal that leading multilingual models (Phi and GLM) achieved overall JWCAS scores comparable to, or significantly higher than, the native Japanese model (LLM-jp). Crucially, our sub-score analysis demonstrates that holistic evaluation metrics can obscure deep pragmatic deficits: while LLM-jp overfits to surface-level linguistic politeness (Layer 1), it shows critical weaknesses in socio-cultural values (Layer 2) and context-aware social strategies (Layer 3). In contrast, leading multilingual models demonstrate balanced competence across all layers. These findings suggest that true cultural competence requires moving beyond native linguistic mastery, highlighting the necessity of multi-dimensional diagnostic frameworks for cross-cultural AI alignment.

Background: Cancer survivors often experience complex and coexisting emotions throughout diagnosis, treatment, and posttreatment life. Emotion classification of patient narratives may help in understanding survivorship experiences; however, evidence remains limited for multidimensional classification using cancer survivor interview narratives. Objective: This study aimed to develop and evaluate natural language processing–based emotion classification models using Japanese cancer survivor interview narratives and to examine whether polarity and multidimensional emotion labels provide complementary perspectives. Methods: We analyzed verbatim transcripts from 15 cancer survivor interviews published by the Cancer Note, Nonprofit Organization. Survivor utterances were extracted, noninformative conversational elements were removed, texts were segmented at Japanese punctuation marks, and 5 consecutive sentences were grouped into 1 chunk. Two annotators labeled 1998 text chunks with 3-class sentiment polarity labels (positive, neutral, or negative) and multilabel Plutchik 8-emotion labels (joy, trust, fear, surprise, sadness, disgust, anger, and anticipation). Japanese BERT (Bidirectional Encoder Representations from Transformers) and LUKE (Language Understanding with Knowledge-based Embeddings) were fine-tuned to build a multiclass polarity classifier and a multilabel 8-emotion classifier. Performance was evaluated using precision, recall, F ₁ -score, macroaveraged metrics, Micro- F ₁ for polarity, and Hamming loss for multilabel classification. For comparison, the same architectures were fine-tuned on WRIME (writers’ and readers’ intensities of emotion for their estimation), a Japanese social media emotion dataset, and evaluated on Cancer Note texts as a domain-transfer analysis. The 95% CIs were estimated using bootstrap resampling with 1000 iterations. Results: Neutral was the most frequent polarity label, trust was the most frequent 8-emotion label, and anger was the least frequent emotion label. Label distributions were imbalanced, with most-to-least frequency ratios of 3.47 for polarity and 8.10 for 8-emotion labels. In the 3-class sentiment polarity task, interview-trained models outperformed WRIME-trained transfer models. Interview Text-BERT achieved the highest micro- F ₁ of 0.696 (95% CI 0.676‐0.716), whereas Interview Text-LUKE achieved the highest macro- F ₁ of 0.660 (95% CI 0.639‐0.682). In the 8-emotion multilabel task, Interview Text-LUKE achieved the highest macro- F ₁ of 0.427 (95% CI 0.398‐0.453) and the lowest Hamming loss of 0.078 (95% CI 0.073‐0.082). WRIME-trained transfer models showed lower performance, particularly in the 8-emotion task. Sadness and trust co-occurred most frequently, suggesting that positive and negative emotional elements may coexist in the same narratives. Conclusions: This exploratory study suggests the feasibility of domain-specific emotion classification for Japanese cancer survivor interview narratives. Models fine-tuned on target-domain narratives generally outperformed WRIME-trained transfer models, although the best architecture differed by task and metric. Polarity labels and Plutchik 8-emotion labels provided complementary perspectives on complex and coexisting emotions in survivorship narratives. However, performance for rare emotions remained limited, and the models should be regarded as preliminary research tools rather than clinically actionable systems. Larger, more diverse, prospectively or externally validated datasets, imbalance-aware methods, and user-centered evaluation are needed before clinical translation.