SHSyed-Amad Hussain

Syed-Amad Hussain

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Syed-Amad Hussain has a total of 30 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Toward trustworthy chatbots: a protocol for red teaming for health related conversations

    by Syed-Amad Hussain, Daniel I. Jackson, Ashley Lewis, Eric Fosler-Lussier, Emre Sezgin

    Health-related chatbots require safety assurance beyond factual correctness. We propose a red-teaming protocol for patient-facing AI structured around three pillars: error stratification, dual-pronged testing, and vulnerability-informed mitigation. We distinguish Document Adherence (DA) from Instruction Adherence (IA), deploying adversarial “attacks” across both single-turn and multi-turn exchanges to provoke system failures. We then applied layered mitigations informed by the vulnerabilities revealed by these attacks. We evaluate this framework on a retrieval-augmented generation (RAG) based chatbot designed to assist with health-related social needs (HRSN).The protocol identified behavioral noncompliance as the dominant risk. While robust in DA (0/60 errors), the system struggled with IA (15% error rate). Crucially, multi-turn stress tests revealed vulnerabilities hidden in single-turn checks: error rates spiked to 50% for advice queries and 40% for user distress. All high-severity failures occurred during these sustained interactions. Of our mitigations, prompt augmentation reduced total errors by 60%, while document augmentation mitigated single-turn distress errors. Combined, they eliminated high-severity errors entirely by forcing “safe failure” loops. We suggest this cycle of stratified analysis, depth-based testing, and targeted mitigation can be a guiding framework for securing clinical conversational agents.


  • Open Access

    Socially Grounded Exemplars Improve Synthetic Conversations for Health-Related Social Needs Navigation

    by Syed-Amad Hussain, Daniel I. Jackson, Samanvith Thotapalli, Marissa B. McClellan, Madeleine Stanco, Grace Varney, Sterling Gleeson, Florencia Nugroho, William Leever, Eric Fosler-Lussier, et al.

    Health-Related Social Needs (HRSNs) significantly impact health outcomes, yet traditional care often fails to address them effectively. While conversational agents offer scalable support, their deployment is hindered by privacy risks and a lack of specialized training data for clinical applications. Synthetic data generation offers a solution to address this gap; standard pipelines often prompt LLMs using structured user personas, comprising demographics, constraints, and goals, to emulate dialogues. However, current methods relying on coarse demographic attributes often yield generic or stereotyped personas that lack real-world nuance. To improve the realism of synthetic data, we introduce Socially Grounded Exemplars (SGEs), which translate abstract persona attributes into granular, conversational descriptors. We implemented a two-stage pipeline using GPT-4o to generate SGEs, which then grounded synthetic dialogue generation under various prompting strategies. We evaluated the approach using automatic diversity metrics (Vendi Score) and blinded pairwise preference ratings by community behavioral health specialists (CBHS). Validation confirmed the feasibility of input generation, with GPT-4o achieving an 85% term acceptability rate for SGEs. In conversation generation, dynamic SGEs significantly improved lexical diversity, achieving a Vendi Score of 289.41 compared to 252.36 for the control baseline. CBHS ranked the model combining dynamic SGEs with implicit name-based cueing highest (Bradley-Terry Score: 0.753), surpassing both the SGE-only model (0.663) and the explicit demographics model (0.348). Raters favored the name-augmented model for “Specificity & Natural Authenticity” (30.0%), while explicit demographic labeling reduced perceived authenticity. We show SGEs leverage LLM parametric knowledge to produce diverse synthetic data, surpassing the limitations of rigid demographic ontologies. Our findings indicate that implicit cueing through names yields more authentic representations than explicit labeling, reducing the risk of stereotyped outputs. This framework supports the creation of privacy-preserving, conversational datasets informing tasks (e.g. evaluation, agentic workflows, and model distillation) in sensitive healthcare contexts.

  • Open Access

    Toward Trustworthy Chatbots: A Protocol for Red Teaming for Health Related Conversations

    by Syed-Amad Hussain, Daniel I. Jackson, Ashley Lewis, Eric Fosler-Lussier, Emre Sezgin

    Introduction: Health-related chatbots are increasingly used to mediate conversations that carry clinical significance and emotional weight. Retrieval-augmented generation (RAG) can reduce factual errors (“hallucinations”), but the risks remain, with additional challenges coming from chatbots acting against behavioral safety and scope rules. Red teaming, an adversarial testing process that deliberately probes systems for failures before deployment, offers a way to surface potential risks. We describe a task-informed red-teaming protocol for health-related and patient-facing chatbots.. Methods: Our protocol is composed of an error stratification, single and multi-turn attack evaluation, and a framework for mitigation techniques. We define an error framework that distinguishes Knowledge Adherence (KA: staying faithful to retrieved documents) from Behavioral Adherence (BA: following safety, tone, and scope instructions). Our single-turn attacks consist of seven attack vectors reflecting real-world pressures, including advice-seeking, user distress, and prompt injection. A subset of these vectors are evaluated in multi-turn attacks. We evaluate two mitigation strategies: (1) prompt augmentation, which adds explicit guardrails to the chatbot prompt, and (2) document augmentation, which adds a localized FAQ document to the retrieval corpus. Finally, we apply this protocol to a social care chatbot (specifically supporting Health-Related Social Needs (HRSN)), developed as an agentic workflow that queries a vetted HRSN resource index. The evaluation corpus comprises 140 single-turn probes and 20 multi-turn stress tests. We assess correctness and risk severity via human annotation. Results: Our error framework identified that the primary safety risk was a failure to follow behavioral rules, rather than a lack of factual knowledge. Furthermore, multi-turn stress tests revealed critical vulnerabilities that single-turn testing missed, directly informing our choice of targeted mitigations. In single-turn tests, the chatbot was factually robust, yielding 0/60 KA errors; however, it struggled with behavioral instructions, producing a 15% (12/80) BA error rate, with 21% (4/19) of those being high-severity. Notable vulnerabilities included advice_query (BA 30%, 6/20) and prompt_injection (BA 20%, 4/20). User_distress triggered the hallucination of unverified contact details in 20% (4/20) of cases. In multi-turn stress tests, error rates rose sharply under conversational persistence: advice_query BA errors reached 50% (5/10) and user_distress reached 40% (4/10), accounting for all high-severity errors (4/4). Prompt augmentation reduced total errors across these vectors by 60% (15/60→6/60). Document augmentation eliminated all single-turn user_distress errors (to 0/20) and reduced advice_query errors (7/20→4/20). When combined in multi-turn tests, these mitigations eliminated high-severity errors entirely, reducing BA errors to 20% (advice_query) and 30% (user_distress) by forcing the chatbot into loops. Conclusion: We demonstrate that a protocol combining single-turn breadth, multi-turn depth, and layered mitigations materially improves chatbot safety and offers a practical template for patient-facing chatbots. Future work should expand on this protocol with chatbots in more diverse clinical domains, and with a larger panel of evaluators.

  • Open Access

    From Keywords to Context: Bridging Expert Insight and Language Models for Multidimensional Sleep Health Classification in Clinical Notes

    by Syed-Amad Hussain, Ariana Calloway, Joseph W Sirrianni, Eric Fosler-Lussier, Mattina Davenport

    Accurate detection of multidimensional sleep health (MSH) information from electronic health records (EHRs) is critical for improving clinical decision-making but remains challenging due to sparse documentation and class imbalance. This study investigates whether integrating expert-guided annotations and keyword-based heuristics with large language models (LLMs) enhances the extraction of nuanced MSH indicators from clinical narratives. Using a novel, expertly annotated dataset (NCH-Sleep), we trained and evaluated models to classify clinical notes across nine clinically relevant MSH categories. Our baseline model demonstrated substantial predictive capability using raw text alone. Incorporating manually annotated spans (oracle annotations) dramatically improved performance, highlighting the benefit of targeted expert guidance. Additionally, employing curated keyword annotations within varying context windows significantly enhanced model interpretability while retaining strong predictive accuracy. Through detailed bias analyses, we identified consistent performance across demographics and clinical settings, although specific disparities underscored the importance of balanced expert oversight. Our findings emphasize the value of expert-informed supervision and heuristic approaches in building scalable, interpretable clinical NLP systems for sleep health classification.

  • Open Access

    Empirical Review of LLM-driven Classification of Multidimensional Sleep Health Mentions from Free-Text Clinical Notes

    by Syed-Amad Hussain, Ariana Calloway, Joseph Sirrianni, Eric Fosler-Lussier, Mattina Davenport

    Accurate multidimensional sleep health (MSH) information is often fragmented and inconsistently represented within hospital infrastructures, leaving crucial details buried in unstructured clinical notes rather than discrete fields. This inconsistency complicates large-scale phenotyping, secondary analyses, and clinical decision support regarding sleep-related outcomes. In this work, we systematically explore contemporary natural language processing techniques, prompt-based large language models (LLMs) and fine-tuned discriminative classifiers, to bridge this critical gap. We evaluate performance on extracting nine key MSH dimensions (timing, duration, efficiency, sleep disorders, daytime sleepiness, interventions, medication, behavior, and satisfaction) from clinical narratives using public datasets (MIMIC-III derivatives) and an internally annotated pediatric sleep corpus. Initially, we assess generative LLM performance using dynamic few-shot prompting, analyzing impacts from varying prompt structures, example quantity, and domain-specificity without explicit task-specific fine-tuning. Subsequently, we fine-tune generative LLM architectures on both in-task and out-of-task data to quantify performance improvements and limitations. Lastly, we benchmark these generative approaches against encoder-based discriminative classifiers (ModernBERT), designed to directly estimate binary presence of each MSH class within full clinical notes. Our experiments demonstrate that fine-tuned discriminative models consistently provide higher classification accuracy, lower inference latency, and more robust span-level identification than either prompted or fine-tuned generative LLMs, given adequate training data. Nonetheless, generative LLMs retain moderate utility in low-data scenarios. Importantly, our results highlight persistent challenges, including difficulty extracting subtle sleep constructs such as sleep efficiency and daytime sleepiness, and biases associated with patient demographics and clinical departments. We conclude by suggesting future research directions: refining span extraction methods, mitigating biases in model performance, and exploring advanced chain-of-thought prompting techniques to achieve reliable, scalable MSH phenotyping within real-world clinical systems.


  • Open Access

    Extracting Medical Information From Free-Text and Unstructured Patient-Generated Health Data Using Natural Language Processing Methods: Feasibility Study With Real-world Data

    by Emre Sezgin, Syed-Amad Hussain, Steve Rust, Yungui Huang

    Background: Patient-generated health data (PGHD) captured via smart devices or digital health technologies can reflect an individual health journey. PGHD enables tracking and monitoring of personal health conditions, symptoms, and medications out of the clinic, which is crucial for self-care and shared clinical decisions. In addition to self-reported measures and structured PGHD (eg, self-screening, sensor-based biometric data), free-text and unstructured PGHD (eg, patient care note, medical diary) can provide a broader view of a patient’s journey and health condition. Natural language processing (NLP) is used to process and analyze unstructured data to create meaningful summaries and insights, showing promise to improve the utilization of PGHD. Objective: Our aim is to understand and demonstrate the feasibility of an NLP pipeline to extract medication and symptom information from real-world patient and caregiver data. Methods: We report a secondary data analysis, using a data set collected from 24 parents of children with special health care needs (CSHCN) who were recruited via a nonrandom sampling approach. Participants used a voice-interactive app for 2 weeks, generating free-text patient notes (audio transcription or text entry). We built an NLP pipeline using a zero-shot approach (adaptive to low-resource settings). We used named entity recognition (NER) and medical ontologies (RXNorm and SNOMED CT [Systematized Nomenclature of Medicine Clinical Terms]) to identify medication and symptoms. Sentence-level dependency parse trees and part-of-speech tags were used to extract additional entity information using the syntactic properties of a note. We assessed the data; evaluated the pipeline with the patient notes; and reported the precision, recall, and F1 scores. Results: In total, 87 patient notes are included (audio transcriptions n=78 and text entries n=9) from 24 parents who have at least one CSHCN. The participants were between the ages of 26 and 59 years. The majority were White (n=22, 92%), had more than one child (n=16, 67%), lived in Ohio (n=22, 92%), had mid- or upper-mid household income (n=15, 62.5%), and had higher level education (n=24, 58%). Out of 87 notes, 30 were drug and medication related, and 46 were symptom related. We captured medication instances (medication, unit, quantity, and date) and symptoms satisfactorily (precision >0.65, recall >0.77, F1>0.72). These results indicate the potential when using NER and dependency parsing through an NLP pipeline on information extraction from unstructured PGHD. Conclusions: The proposed NLP pipeline was found to be feasible for use with real-world unstructured PGHD to accomplish medication and symptom extraction. Unstructured PGHD can be leveraged to inform clinical decision-making, remote monitoring, and self-care including medical adherence and chronic disease management. With customizable information extraction methods using NER and medical ontologies, NLP models can feasibly extract a broad range of clinical information from unstructured PGHD in low-resource settings (eg, a limited number of patient notes or training data).


  • Open Access

    Extracting Medical Information From Free-Text and Unstructured Patient-Generated Health Data Using Natural Language Processing Methods: Feasibility Study With Real-world Data (Preprint)

    by Emre Sezgin, Syed-Amad Hussain, Steve Rust, Yungui Huang

    BACKGROUND Patient-generated health data (PGHD) captured via smart devices or digital health technologies can reflect an individual health journey. PGHD enables tracking and monitoring of personal health conditions, symptoms, and medications out of the clinic, which is crucial for self-care and shared clinical decisions. In addition to self-reported measures and structured PGHD (eg, self-screening, sensor-based biometric data), free-text and unstructured PGHD (eg, patient care note, medical diary) can provide a broader view of a patient’s journey and health condition. Natural language processing (NLP) is used to process and analyze unstructured data to create meaningful summaries and insights, showing promise to improve the utilization of PGHD. OBJECTIVE Our aim is to understand and demonstrate the feasibility of an NLP pipeline to extract medication and symptom information from real-world patient and caregiver data. METHODS We report a secondary data analysis, using a data set collected from 24 parents of children with special health care needs (CSHCN) who were recruited via a nonrandom sampling approach. Participants used a voice-interactive app for 2 weeks, generating free-text patient notes (audio transcription or text entry). We built an NLP pipeline using a zero-shot approach (adaptive to low-resource settings). We used named entity recognition (NER) and medical ontologies (RXNorm and SNOMED CT [Systematized Nomenclature of Medicine Clinical Terms]) to identify medication and symptoms. Sentence-level dependency parse trees and part-of-speech tags were used to extract additional entity information using the syntactic properties of a note. We assessed the data; evaluated the pipeline with the patient notes; and reported the precision, recall, and F1 scores. RESULTS In total, 87 patient notes are included (audio transcriptions n=78 and text entries n=9) from 24 parents who have at least one CSHCN. The participants were between the ages of 26 and 59 years. The majority were White (n=22, 92%), had more than one child (n=16, 67%), lived in Ohio (n=22, 92%), had mid- or upper-mid household income (n=15, 62.5%), and had higher level education (n=24, 58%). Out of 87 notes, 30 were drug and medication related, and 46 were symptom related. We captured medication instances (medication, unit, quantity, and date) and symptoms satisfactorily (precision >0.65, recall >0.77, F1>0.72). These results indicate the potential when using NER and dependency parsing through an NLP pipeline on information extraction from unstructured PGHD. CONCLUSIONS The proposed NLP pipeline was found to be feasible for use with real-world unstructured PGHD to accomplish medication and symptom extraction. Unstructured PGHD can be leveraged to inform clinical decision-making, remote monitoring, and self-care including medical adherence and chronic disease management. With customizable information extraction methods using NER and medical ontologies, NLP models can feasibly extract a broad range of clinical information from unstructured PGHD in low-resource settings (eg, a limited number of patient notes or training data).


  • Open Access

    Performance Evaluation of Extracting Medical Information from Patient Notes using Natural Language Processing Methods (Preprint)

    by Emre Sezgin, Syed-Amad Hussain, Steve Rust, Yungui Huang

    UNSTRUCTURED Patient-generated health data (PGHD) is becoming a necessary component of remote monitoring and clinical decision-making as well as self-care and managing chronic conditions. Medical diaries and patient notes have been the primary source of unstructured PGHD, enabling patients to collect and communicate their health information remotely and in real-time to their providers. Yet there is no established mechanism or pipeline for processing free text patient notes (“unstructured PGHD”) or integrating these notes into clinical decision making in a low-resource setting (i.e., not depending on a large dataset or processing power). In the literature, there are a number of studies reporting NLP applications on clinical notes to identify symptoms and conditions, but these studies have been limited to public data (e.g. social media posts by patients). In this paper, we evaluate the performance of an hybrid (deep learning + rule-based) NLP pipeline on a low-resource PHGD dataset through an empirical evaluation of automatic component extraction where we measure the model’s ability to conduct automatic entity extraction using ontologies (medication-dose: RxNORM, symptoms: SNOMED) from patient notes.

  • Open Access

    Locating Youth Exposed to Parental Justice Involvement in the Electronic Health Record: Development of a Natural Language Processing Model

    by Samantha Boch, Syed-Amad Hussain, Sven Bambach, Cameron DeShetler, Deena Chisolm, Simon Linwood

    Background: Parental justice involvement (eg, prison, jail, parole, or probation) is an unfortunately common and disruptive household adversity for many US youths, disproportionately affecting families of color and rural families. Data on this adversity has not been captured routinely in pediatric health care settings, and if it is, it is not discrete nor able to be readily analyzed for purposes of research. Objective: In this study, we outline our process training a state-of-the-art natural language processing model using unstructured clinician notes of one large pediatric health system to identify patients who have experienced a justice-involved parent. Methods: Using the electronic health record database of a large Midwestern pediatric hospital-based institution from 2011-2019, we located clinician notes (of any type and written by any type of provider) that were likely to contain such evidence of family justice involvement via a justice-keyword search (eg, prison and jail). To train and validate the model, we used a labeled data set of 7500 clinician notes identifying whether the patient was ever exposed to parental justice involvement. We calculated the precision and recall of the model and compared those rates to the keyword search. Results: The development of the machine learning model increased the precision (positive predictive value) of locating children affected by parental justice involvement in the electronic health record from 61% (a simple keyword search) to 92%. Conclusions: The use of machine learning may be a feasible approach to addressing the gaps in our understanding of the health and health services of underrepresented youth who encounter childhood adversities not routinely captured—particularly for children of justice-involved parents.