Avisha Das

The objective is to create an automated knowledge extraction tool for cancer research that builds high-quality academic corpora for LLM fine-tuning while investigating its effectiveness in interleukin-6 and bladder cancer domains. To address the current gap in knowledge retrieval techniques for cancer research data collection, we propose KnowledgePipeline, a novel automated tool that incorporates diverse aspects of academic papers and metadata. Our tool integrates content, co-citations, and co-authorship networks to construct domain-specific academic corpora suitable for fine-tuning LLMs. We leverage two LLMs (GPTJ-6.7B and Galactica30B) trained on domain-specific question-answer pairs from the refined data. The system’s evaluation focuses on both the quality of extracted knowledge and the performance of fine-tuned models in open-ended question-answering tasks. We see that KnowledgePipeline offers a scalable, automated framework for domain-specific knowledge retrieval and fine-tuned applications in cancer research, advancing literature discovery and addressing critical biomedical challenges. It achieved high relevance scores of 68% for IL-6 and 74.5% for bladder cancer, with a fine-tuned Galactica-30B model demonstrating promising capabilities.

Critical findings in radiology reports are life threatening conditions that need to be communicated promptly to physicians (“critical findings”) for timely man-agement of patients. Flagging radiology reports of such incidents could facilitate opportune communication of critical findings. With advancements in natural language processing (NLP), large language models (LLMs) can be trained with task-specific instructions and examples to mine information from narrative texts. We believe that similar methods can be applied to radiology reports to identify and extract critical findings from these reports. However, due to the rarity of such critical events, there is a dearth of manually labeled datasets of critical findings in radiology reports. To overcome this limitation, we train instruction-tuned MISTRAL-based language models in a two-phase weakly supervised fine-tuning setup on unlabeled radiology reports from Mayo Clinic (n=15000). The weakly fine-tuned model is then used to automatically extract critical terms from both internal and external test datasets - Mayo Clinic (n=80) and MIMIC-III (n=123) 1 respectively against the expert-annotation. We also evaluated performance of the models on a large-scale MIMIC-IV reports (n=5000) using automated LLM-aided evaluation metrics - G-eval and Prometheus. We observe that, for both manual and LLM-based evaluations, weakly supervised fine-tuning improves model performance, showing successful task-specific alignment. For community use, we released the trained model with open-source academic license∗

Large language models (LLMs) with billions of parameters and trained on massive amounts of crowdsourced public data have made a dramatic impact on natural language processing (NLP) tasks. Domain specific 'finetuning' of LLMs has further improved model behavior through task-specific alignment and refinement. However with the widespread development and deployment of LLMs, existing vulnerabilities in these models have made way for perpetrators to manipulate them for malicious intentions. The widespread integration of LLMs in clinical NLP underscores the looming threat of privacy leakage and targeted attacks like prompt injection or data poisoning. In this work, we designed a systematic framework to expose vulnerabilities of clinical generative language model with a specific emphasis on its application to clinical notes. We design three attack pipelines to highlight model's susceptibility to core types of targeted attacks-(i) instruction-based data poisoning, (ii) trigger-based model editing and (iii) membership inference on de-identified breast cancer clinical notes. Our proposed framework is the first work to investigate the extent of LLM-based attacks in the clinical domain. Our findings reveal successful manipulation of LLM behavior, prompting concerns about the stealthiness and effectiveness of such attacks. Through this work, we hope to emphasize on the urgency of comprehending these vulnerabilities in LLMs and encourage the mindful and responsible usage of LLMs in the clinical domain.

Training Large Language Models (LLMs) with billions of parameters on a dataset and publishing the model for public access is the standard practice currently. Despite their transformative impact on natural language processing, public LLMs present notable vulnerabilities given the source of training data is often web-based or crowdsourced, and hence can be manipulated by perpetrators. We delve into the vulnerabilities of clinical LLMs, particularly BioGPT which is trained on publicly available biomedical literature and clinical notes from MIMIC-III, in the realm of data poisoning attacks. Exploring susceptibility to data poisoning-based attacks on de-identified breast cancer clinical notes, our approach is the first one to assess the extent of such attacks and our findings reveal successful manipulation of LLM outputs. Through this work, we emphasize on the urgency of comprehending these vulnerabilities in LLMs, and encourage the mindful and responsible usage of LLMs in the clinical domain.

Massive textual content has enabled rapid advances in natural language modeling. The use of pre-trained deep neural language models has significantly improved natural language understanding tasks. However, the extent to which these systems can be applied to content generation is unclear. While a few informal studies have claimed that these models can generate ‘high quality’ readable content, there is no prior study on analyzing the generated content from these models based on sampling and fine-tuning hyperparameters. We conduct an in-depth comparison of several language models for open-ended story generation from given prompts. Using a diverse set of automated metrics, we compare the performance of transformer-based generative models – OpenAI’s GPT2 (pre-trained and fine-tuned) and Google’s pretrained Transformer-XL and XLNet to human-written textual references. Studying inter-metric correlation along with metric ranking reveals interesting insights – the high correlation between the readability scores and word usage in the text. A study of the statistical significance and empirical evaluations between the scores (human and machine-generated) at higher sampling hyperparameter combinations (t = {0.75, 1:0}, k = {100, 150, 250}) reveal that the top pre-trained and fine-tuned models generated samples condition well on the prompt with an increased occurrence of unique and difficult words. The GPT2-medium model fine-tuned on the 1024 Byte-pair Encoding (BPE) tokenized version of the dataset along with pre-trained Transformer-XL models generated samples close to human written content on three metrics: prompt-based overlap, coherence, and variation in sentence length. A study of overall model stability and performance shows that fine-tuned GPT2 language models have the least deviation in metric scores from human performance.

Phishing and spear phishing are typical examples of masquerade attacks since trust is built up through impersonation for the attack to succeed. Given the prevalence of these attacks, considerable research has been conducted on these problems along multiple dimensions. We reexamine the existing research on phishing and spear phishing from the perspective of the unique needs of the security domain, which we call security challenges: real-time detection, active attacker, dataset quality and base-rate fallacy. We explain these challenges and then survey the existing phishing/spear phishing solutions in their light. This viewpoint consolidates the literature and illuminates several opportunities for improving existing solutions. We organize the existing literature based on detection techniques for different attack vectors (e.g., URLs, websites, emails) along with studies on user awareness. For detection techniques we examine properties of the dataset, feature extraction, detection algorithms used, and performance evaluation metrics. This work can help guide the development of more effective defenses for phishing, spear phishing and email masquerade attacks of the future, as well as provide a framework for a thorough evaluation and comparison.