Zihao Wang

Background: Large language models (LLMs) are increasingly used to generate and evaluate patient-facing medical materials. Through persona prompting, LLMs can be instructed to simulate responses from individuals with different demographic and experiential backgrounds, offering a scalable approach for approximating patient feedback. However, it remains unclear how closely these simulated persona-based responses reflect real patient perspectives, and whether this alignment is consistent across demographic subgroups. Objective: This study assessed whether contemporary LLMs can simulate diverse patient perspectives on hospital discharge summaries when prompted with personas defined by education, gender, and health care use. We further examined whether persona-based simulation performance was equitable across subgroups by comparing LLM-generated responses with responses from a general-population human cohort under a standardized evaluation framework. Methods: We evaluated 4 contemporary LLMs, GPT-4.1, GPT-5.2, Claude Sonnet 4.5, and Opus 4.6, against responses from 99 participants recruited through Prolific (Prolific Academic Ltd). Participants answered 10 questions for discharge summaries that varied in length and medical complexity. Questions included information-based items assessing factual comprehension and perception-based items assessing subjective understanding. Model-human alignment was measured using modal mean absolute error and expected absolute ordinal distance. We also conducted a chain-of-thought sensitivity analysis for cases with the greatest divergence between LLM-generated persona and matched human responses. Results: Across LLMs, persona responses aligned more closely with human responses on information-based items than on perception-based items. Education was the most consistent stratification axis: every model aligned more closely with the high-education subgroup than with the low-education subgroup on information items, suggesting that LLMs capture broad factual-comprehension differences associated with educational background. Perception items, in which subjective understanding and uncertainty dominate human responses, were simulated less reliably. Alignment also depended on discharge summary characteristics, with short but clinically complex summaries producing the largest divergence. Gender-related differences were smaller and inconsistent in direction, reflecting item-level modal flips rather than systematic distributional bias. Chain-of-thought improved alignment on some factual items but did not recover the full range of less common human responses. Across models, LLMs underselected “I don’t know” and “Not provided” responses relative to humans, indicating a tendency to commit to determinate answers even when human participants expressed uncertainty. Conclusions: Contemporary LLMs can approximate common response patterns within demographic subgroups, particularly for factual comprehension and education-based personas. However, they are less reliable for capturing heterogeneous and perception-based patient responses. Equity performance varied substantially across models, making model selection an important design consideration. Persona-based LLM simulation should be used as an assistive screening tool for patient materials and should complement, not replace, human evaluation with subgroup validation.

The wide deployment of Large Language Models (LLMs) has given rise to strong demands for optimizing their inference performance. Today’s techniques serving this purpose primarily focus on reducing latency and improving throughput through algorithmic and hardware enhancements, while largely overlooking their privacy side effects, particularly in a multi-user environment. In our research, for the first time, we discovered a set of new timing side channels in LLM systems, arising from shared caches and GPU memory allocations, which can be exploited to infer both confidential system prompts and those issued by other users. These vulnerabilities echo security challenges observed in traditional computing systems, highlighting an urgent need to address potential information leakage in LLM serving infrastructures. In this paper, we report novel attack strategies designed to exploit such timing side channels inherent in LLM deployments, specifically targeting the Key-Value (KV) cache and semantic cache widely used to enhance LLM inference performance. Our approach leverages timing measurements and classification models to detect cache hits, allowing an adversary to infer private prompts with high accuracy. We also propose a token-by-token search algorithm to efficiently recover shared prompt prefixes in the caches, showing the feasibility of stealing system prompts and those produced by peer users. Our experimental studies on black-box testing of popular online LLM services demonstrate that such privacy risks are completely realistic, with significant consequences. Our findings underscore the need for robust mitigation to protect LLM systems against such emerging threats.

The rapid advancements of large language models (LLMs) have raised public concerns about the privacy leakage of personally identifiable information (PII) within their extensive training datasets. Recent studies have demonstrated that an adversary could extract highly sensitive privacy data from the training data of LLMs with carefully designed prompts. However, these attacks suffer from the model's tendency to hallucinate and catastrophic forgetting (CF) in the pre-training stage, rendering the veracity of divulged PIIs negligible. In our research, we propose a novel attack, Janus, which exploits the fine-tuning interface to recover forgotten PIIs from the pre-training data in LLMs. We formalize the privacy leakage problem in LLMs and explain why forgotten PIIs can be recovered through empirical analysis on open-source language models. Based upon these insights, we evaluate the performance of Janus on both open-source language models and two latest LLMs, i.e., GPT-3.5-Turbo and LLaMA-2-7b. Our experiment results show that Janus amplifies the privacy risks by over 10 times in comparison with the baseline and significantly outperforms the state-of-the-art privacy extraction attacks including prefix attacks and in-context learning (ICL). Furthermore, our analysis validates that existing fine-tuning APIs provided by OpenAI and Azure AI Studio are susceptible to our Janus attack, allowing an adversary to conduct such an attack at a low cost.

Voice personal assistant (VPA) platforms (e.g., Amazon Alexa) allow developers to deploy their voice apps on third-party servers. However, this strategy introduces unexpected privacy risks to VPA customers. Malicious developers can dynamically change their app's behaviors to circumvent the platform's vetting process. This paper aims to systematically analyze Alexa's voice app ecosystem (i.e., Alexa skills), focusing on behavior manipulation (also referred to as skill behavior change). We identify the root causes of malicious skills getting published and propose a defense solution to effectively protect users. First, we uncover Amazon's skill vetting strategy and the privacy issues relevant to their vetting. We reveal that, in addition to the skill certification process before a skill gets published, Amazon also deploys a skill monitoring scheme after the skill is published. We further discover limitations of this monitoring scheme that have not been explored in previous research. Lastly, to address these issues, we propose a run-time skill monitoring approach to check the consistency of the skill behaviors when users interact with skills. Our findings suggest a call for action to improve the vetting process for VPA skills without placing a burden on skill developers and help developers adhere to policies.

Research on side-channel leaks has long been focusing on the information exposure from a single channel (memory, network traffic, power, etc.). Less studied is the risk of learning from multiple side channels related to a target activity (e.g., website visits) even when individual channels are not informative enough for an effective attack. Although the prior research made the first step on this direction, inferring the operations of foreground apps on iOS from a set of global statistics, still less clear are how to determine the maximum information leaks from all target-related side channels on a system, what can be learnt about the target from such leaks and most importantly, how to control information leaks from the whole system, not just from an individual channel. To answer these fundamental questions, we performed the first systematic study on multi-channel inference, focusing on iOS as the first step. Our research is based upon a novel attack technique, called Mischief, which given a set of potential side channels related to a target activity (e.g., foreground apps), utilizes probabilistic search to approximate an optimal subset of the channels exposing most information, as measured by Merit Score, a metric for correlation-based feature selection. On such an optimal subset, an inference attack is modeled as a multivariate time series classification problem, so the state-of-the-art deep-learning based solution, InceptionTime in particular, can be applied to achieve the best possible outcome. Mischief is found to work effectively on today's iOS (16.2), identifying foreground apps, website visits, sensitive IoT operations (e.g., opening the door) with a high confidence, even in an open-world scenario, which demonstrates that the protection Apple puts in place against the known attack is inadequate. Also importantly, this new understanding enables us to develop more comprehensive protection, which could elevate today's side-channel research from suppressing leaks from individual channels to controlling information exposure across the whole system.