Gondy Leroy

ObjectiveTo determine whether descriptions of autism behaviors generated by a large language model (LLM) can substitute for caregiver and clinician descriptions, judged by clinicians at the level of individual DSM-5 criteria rather than by downstream classifier accuracy.MethodsWe assembled 342,187 DSM-5-labeled descriptions covering seven criteria (A1–A3, B1–B4): 16,885 caregiver survey responses, 9,514 sentences from clinical notes, and 315,788 GPT-4 generations. BERTopic reduced each source–criterion combination to two topics (42 in total), each shown as ranked keywords with three representative descriptions. Three clinicians, blinded to source, rated every topic 0–2 for usefulness, specificity, and DSM-5 relevance. A linear model with Type III ANOVA tested source, criterion, and metric effects. Cosine similarity between topic term vectors compared content across sources; the mean share of each document held by its dominant topic measured breadth.ResultsSynthetic descriptions scored highest overall (mean 1.59), followed by caregiver (1.45) and clinical (1.30) descriptions (F(2,24)=5.38, p=0.012). A source × criterion interaction (F(12,24)=5.81, p<0.001) changed the ranking by criterion: clinical notes scored highest on A1 (1.67) and B3 (1.61), caregiver descriptions on A3 (1.72) and B1 (1.94), and synthetic descriptions on A2 (1.89) and B2 (1.78); caregiver and clinical descriptions tied on B4 (1.50). Synthetic topics were closer to caregiver topics (cosine similarity 0.122) than to clinical topics (0.050). Synthetic output was narrower: one topic held 93.7% of synthetic documents versus 76.1% (caregiver) and 77.8% (clinical), and over 99.7% for A1–A3.ConclusionCriterion-prompted GPT-4 descriptions were rated as useful as or more useful than human descriptions for concrete behaviors, but caregiver narratives rated higher for relational social criteria, and synthetic output repeated a small set of behaviors. Synthetic data can supplement human descriptions; its volume should not be read as coverage.

Background: Generative AI (GenAI) is increasingly used by health information consumers to interpret medical content and support decision-making. Although these systems provide accessible and timely information, they may also produce inaccurate or misleading outputs. Effective use of GenAI, therefore, depends on users’ ability to calibrate trust based on information accuracy. However, little is known about how learned dependency on GenAI (the habitual reliance on AI systems for solving problems) influences trust calibration in health information contexts. Objective: This study examines how learned dependency on GenAI affects health information consumers’ calibration of trust in AI-generated information, and whether text-based visual attention cues (TVCs), such as highlighting critical information in text, mitigate overreliance on incorrect outputs. Methods: We conducted 2 randomized controlled experiments: the first involved 338 college students, and the second replicated the study with 563 Amazon Mechanical Turk participants. Both studies used a 2 × 2 between-participants design, manipulating (1) information accuracy (correct vs incorrect) and (2) TVCs (text highlight vs no text highlight). Participants evaluated AI-generated health information presented alongside source text. Trust was measured using a multi-item scale, and learned dependency on GenAI was assessed using a validated self-reported measure. Linear regression models were used to examine main and interaction effects. Results: Across both experiments, information accuracy had a significant positive effect on trust, with participants expressing greater trust in correct than in incorrect AI-generated information (experiment 1: B =2.107, 95% CI 1.337-2.878; P <.001 and experiment 2: B =0.203, 95% CI 0.115-0.290; P <.001). Learned dependency was positively associated with trust in both experiments (experiment 1: B =0.277, 95% CI 0.033-0.521; P =.03 and experiment 2: B =0.822, 95% CI 0.715-0.929; P <.001), such that users with greater dependency trusted AI outputs more overall. Critically, the interaction between information accuracy and learned dependency was negative and significant in both experiments (experiment 1: B =−0.399, 95% CI −0.695 to −0.104; P <.001 and experiment 2: B =−0.459, 95% CI −0.577 to −0.340; P <.001), indicating that higher dependency reduces users’ sensitivity to information inaccuracy. Text highlighting did not significantly affect trust in either experiment, nor did it moderate the relationship between learned dependency and trust. Conclusions: This study demonstrates that while users generally trust accurate AI-generated health information more than inaccurate information, higher self-reported learned dependency is associated with weaker trust calibration, predicting greater susceptibility to incorrect outputs. TVCs, such as text highlighting, are insufficient to mitigate this effect. These findings highlight the need for more effective design interventions to support critical evaluation and reduce overreliance on GenAI in health information environments.

Introduction: Early autism diagnosis remains challenging due to reliance on clinical observation and limited specialist availability. Addressing these barriers through automated diagnostic labeling and the integration of parental input may help mitigate the problem. Methods: We trained a BioBERT machine learning model to label individual autism behavioral descriptions using the seven DSM-5 diagnostic criteria (A1-A3, B1-B4). This approach offers transparent clinical decision-making by providing detailed diagnostic information for individual behaviors and avoiding final case-level black-box decisions. We evaluated the model's performance on labeling lay ( N = 35,971) and clinical ( N = 145,603) behavior descriptions, as well as its transferability between the two. In addition, we compared the data sources by evaluating the diagnostic utility of lay and clinical examples across four dimensions, and of AI-generated summaries across two dimensions. Results: We found that BioBERT can label both types of input, although it achieved higher precision (69%) on clinical descriptions and higher recall (83%) on lay descriptions. Sample size did not explain differences in performance. Transferring models from one data type to another results in a performance drop. Overall, training first on clinical data yielded the best-performing diagnostic models. When evaluating the examples from both data sources, the results show similar scores for the Utility, Specificity, Clinical Relevance, and Impact on Daily Life dimensions, and the cosine similarity analysis revealed substantial overlap (0.42) in vocabulary between the two. The utility of examples for A diagnostic behaviors was generally scored higher than that for B diagnostic behaviors. AI-generated summary scores showed a similar pattern between A and B examples but they were only moderately representative of these examples. Discussion: These results demonstrate that lay behavioral descriptions can provide diagnostically valuable information comparable to clinical observations, although they are not readily summarized by AI. The integration of lay information into the diagnostic workflows could accelerate autism diagnosis without compromising clinical utility.

Background : Autism Spectrum Disorders (ASD) is a neurodevelopmental condition where early diagnosis is extremely important for optimal treatment effects. Unfortunately, the current age of diagnosis is made late due to a variety of factors, including a lack of clinicians with ASD expertise. We developed Autism Diagnostic Identification Software (ADIS), an AI-based clinical decision support tool that identifies autism-relevant behaviors in narrative clinical text and labels them with DSM5 diagnostic criteria. Using current DSM5 rules, an autism diagnosis is then suggested. Our aim is to provide pre-decision support, which differs from explainable AI approaches, in which AI decisions are clarified post hoc. Methods . To evaluate the impact of AI suggestions, we conducted a vignette-based user study with 21 clinicians (48% with completed medical training), each of whom reviewed four real pediatric cases (two with ADIS support and two without), resulting in 84 diagnostic decisions. The cases were chosen so that ADIS also suggested correct and incorrect decisions to study participants. Results : Overall diagnostic accuracy was 57.14% when ADIS was active versus 66.67% without ADIS, a nonsignificant difference. Decision confidence was higher with ADIS (F(1,80) = 3.71, p = .058), and significantly higher among clinicians who had completed their medical training (F(1,80) = 18.45, p < .001). There was also significant interaction between ADIS correctness and training completion (F(1,38) = 5.05, p = .031): trainees showed 0% accuracy when ADIS was wrong versus 68.75% when it was correct, whereas trained clinicians achieved 66.67% and 56.25% accuracy in those conditions, respectively. Despite these differences in accuracy, confidence was high regardless of ADIS correctness. Most participants (90%) reported that ADIS use was learned quickly and rated its features as well integrated. Conclusions : These findings suggest that AI support can help inexperienced clinicians, but also highlight the importance of AI accuracy, given the observed overreliance on AI. ADIS is usable and perceived as helpful, but its deployment will be most effective when paired with AI literacy training that mitigates automation bias.

BACKGROUND Generative artificial intelligence (GenAI) is increasingly used by health information consumers to interpret medical content and support decision-making. While these systems provide accessible, timely information, they may also produce inaccurate or misleading outputs. Effective use of GenAI, therefore, depends on users’ ability to calibrate trust based on information accuracy. However, little is known about how learned dependency on GenAI influences trust calibration in health information contexts. OBJECTIVE This study examines how learned dependency on GenAI affects health information consumers’ trust calibration in AI-generated information and whether visual attention cues (e.g., highlighting critical information) mitigate overreliance on incorrect outputs. METHODS We conducted a randomized controlled experiment with 338 participants. The study employed a 2 × 2 design manipulating (1) information accuracy (correct vs incorrect) and (2) visual attention cues (highlight vs no highlight). Participants evaluated AI-generated health information presented alongside source text. Trust was measured using a multi-item scale, and learned dependency on GenAI was assessed using a validated self-reported measure. Linear regression models were used to examine main and interaction effects. RESULTS Information accuracy had a strong positive effect on trust (β = 2.107, 95% CI [1.337, 2.878], p < .001), indicating that participants generally trusted correct information more than incorrect information. Learned dependency on GenAI was also positively associated with trust (β = 0.277, 95% CI [0.033, 0.521], p = .026). Importantly, the interaction between information accuracy and learned dependency was negative and significant (β = −0.399, 95% CI [−0.695, −0.104], p = .008), suggesting that higher dependency reduces users’ ability to differentiate between accurate and inaccurate information. In contrast, visual attention cues did not significantly affect trust (β = 0.149, 95% CI [−0.622, 0.920], p = .704), nor did they moderate the effect of dependency (β = −0.009, 95% CI [−0.305, 0.287], p = .950). CONCLUSIONS This study demonstrates that while users generally trust accurate AI-generated health information more than inaccurate information, learned dependency weakens trust calibration, increasing susceptibility to incorrect outputs. Visual attention cues alone are insufficient to mitigate this effect. These findings highlight the need for more effective design interventions to support critical evaluation and reduce overreliance on GenAI in health information environments. Keywords: Learned Dependency; GenAI; Trust Calibration; Attention Mechanism; Automation Bias; Health Information; Human–GenAI Interaction.

Background: Health literacy, the ability to understand and act on health information, is critical for patient outcomes and health care system effectiveness. While plain language guidelines enhance text-based communication, audio-based health information remains underexplored, despite the growing use of digital assistants and smart devices in health care. Traditional readability formulas, such as Flesch-Kincaid, provide limited insights into the complexity of health-related texts and fail to address challenges specific to audio formats. Factors like syntax and semantic features significantly influence comprehension and retention across modalities. Objective: This study investigates features that affect comprehension of medical information delivered via text or audio formats. We also examine existing readability formulas and their correlation with perceived and actual difficulty of health information for both modalities. Method: We developed a parallel corpus of health-related information that differed in delivery format: text or audio. We used text from the British Medical Journal (BMJ) Lay Summary (n=193), WebMD (n=40), Patient Instruction (n=40), Simple Wikipedia (n=243), and BMJ journal (n=200). Participants (n=487) read or listened to a health text and then completed a questionnaire evaluating perceived difficulty of the text, measured using a 5-point Likert scale, and actual difficulty measured using multiple-choice and true-false questions (comprehension) as well as free recall of information (retention). Questions were generated by generative artificial intelligence (ChatGPT-4.0). Underlying syntactic, semantic, and domain-specific features, as well as common readability formulas, were evaluated for their relation to information difficulty. Results: Text versions were perceived as easier than audio, with BMJ Lay Summary scoring 1.76 versus 2.1 and BMJ journal 2.59 versus 2.83 (lower is easier). Comprehension accuracy was higher for text across all sources (eg, BMJ journal: 76% vs 58%; Patient Instructions: 86% vs 66%). Retention was better for text, with significant differences in exact word matching for Patient Instructions and BMJ journal. Longer texts increased perceived difficulty in text but reduced free recall in both modalities (−0.23,−0.25 in audio). Higher content word frequency improved retention (0.23, 0.21) and lowered perceived difficulty (−0.20 in audio). Verb-heavy content eased comprehension (−0.29 in audio), while nouns and adjectives increased difficulty (0.20, 0.18). Readability formulas’ outcomes were unrelated to comprehension or retention, but correlated with perceived difficulty in text (eg, Smog Index: 0.334 correlation). Conclusions: Text was more effective for conveying complex health information, but audio can be suitable for easier content. In addition, several textual features affect information comprehension and retention for both modalities. Finally, existing readability formulas did not explain actual difficulty. This study highlighted the importance of tailoring health information delivery to content complexity by using appropriate style and modality.