Muhammad Ibtsaam Qadir

Surgical video interpretation is a promising medical artificial intelligence application. However, no existing video annotation method preserves the spatiotemporal complexity of surgeon reasoning. Here we show that verbal reasoning and visual attention can be converted into structured, machine-actionable records of intraoperative behaviours. Our method decomposes transcribed verbal commentary into video-anchored semantic feedback chunks, which are classified via a large language model, with spatial grounding to surgical scenes via eyegaze or cursor tracking. We demonstrate method validity and scalability on structured and unstructured annotation tasks. For quality feedback on full-length colorectal procedures, the method reached near-human fidelity for chunking (cosine similarity: 0.95±0.01) and semantic classification across observations (Cohen’s κ: 0.71±0.07) and evaluative triggers (Cohen’s κ: 0.67±0.14), with excellent usability ratings. For structured critical view of safety assessment in laparoscopic cholecystectomy, implicit annotation yielded excellent agreement with explicit reviewer ratings (Cohen’s κ: 0.83, 0.49 and 0.81 across three criteria). We anticipate this method will advance surgical data science by enabling scalable construction of meaningfully annotated surgical video datasets.

Importance: Artificial Intelligence-driven analysis of laparoscopic video holds potential to increase the safety and precision of minimally invasive surgery. Vision-language models are particularly promising for video-based surgical decision support due to their capabilities to comprehend complex temporospatial (video) data. However, the same multimodal interfaces that enable such capabilities also introduce new vulnerabilities to manipulations through embedded deceptive text or images (prompt injection attacks). Objective: To systematically evaluate how susceptible state-of-the-art video-capable vision-language models are to textual and visual prompt injection attacks in the context of clinically relevant surgical decision support tasks. Design, Setting, and Participants: In this observational study, we systematically evaluated four state-of-the-art vision-language models, Gemini 1.5 Pro, Gemini 2.5 Pro, GPT-o4-mini-high, and Qwen 2.5-VL, across eleven surgical decision support tasks: detection of bleeding events, foreign objects, image distortions, critical view of safety assessment, and surgical skill assessment. Prompt injection scenarios involved misleading textual prompts and visual perturbations, displayed as white text overlay, applied at varying durations. Main Outcomes and Measures: The primary measure was model accuracy, contrasted between baseline performance and each prompt injection condition. Results: All vision-language models demonstrated good baseline accuracy, with Gemini 2.5 Pro generally achieving the highest mean [standard deviation] accuracy across all tasks (0.82 [0.01]), compared to Gemini 1.5 Pro (0.70 [0.03]) and GPT-o4 mini-high (0.67 [0.06]). Across tasks, Qwen 2.5-VL censored most outputs and achieved an accuracy of (0.58 [0.03]) on non-censored outputs. Textual and temporally-varying visual prompt injections reduced the accuracy for all models. Prolonged visual prompt injections were generally more harmful than single-frame injections. Gemini 2.5 Pro showed the greatest robustness and maintained stable performance for several tasks despite prompt injections, whereas GPT-o4-mini-high exhibited the highest vulnerability, with mean (standard deviation) accuracy across all tasks declining from 0.67 (0.06) at baseline to 0.24 (0.04) under full-duration visual prompt injection ( P < .001). Conclusion and Relevance: These findings indicate the critical need for robust temporal reasoning capabilities and specialized guardrails before vision-language models can be safely deployed for real-time surgical decision support. Key Points: Question: Are video vision-language models (VLMs) susceptible to textual and visual prompt injection attacks when used for surgical decision support tasks? Finding: Textual and visual prompt injection attacks consistently degraded the performance of four state-of-the-art VLMs across eleven surgical tasks. Gemini 2.5 Pro was most robust to textual and visual prompt injection attacks, whereas GPT-o4-mini-high was most vulnerable. Prolonged visual injections had a greater negative impact than single-frame injections. Meaning: Present-generation video VLMs are highly vulnerable to textual and visual prompt injection attacks. This critical safety vulnerability must be addressed before their integration into surgical decision support systems.

Based on the Fukuoka and Kyoto international consensus guidelines, the current clinical management of intraductal papillary mucinous neoplasm (IPMN) largely depends on imaging features. While these criteria are highly sensitive in detecting high-risk IPMN, they lack specificity, resulting in surgical overtreatment. Artificial Intelligence (AI)-based medical image analysis has the potential to augment the clinical management of IPMNs by improving diagnostic accuracy. Based on a systematic review of the academic literature on AI in IPMN imaging, 1041 publications were identified of which 25 published studies were included in the analysis. The studies were stratified based on prediction target, underlying data type and imaging modality, patient cohort size, and stage of clinical translation and were subsequently analyzed to identify trends and gaps in the field. Research on AI in IPMN imaging has been increasing in recent years. The majority of studies utilized CT imaging to train computational models. Most studies presented computational models developed on single-center datasets (n = 11,44%) and included less than 250 patients (n = 18,72%). Methodologically, convolutional neural network (CNN)-based algorithms were most commonly used. Thematically, most studies reported models augmenting differential diagnosis (n = 9,36%) or risk stratification (n = 10,40%) rather than IPMN detection (n = 5,20%) or IPMN segmentation (n = 2,8%). This systematic review provides a comprehensive overview of the research landscape of AI in IPMN imaging. Computational models have potential to enhance the accurate and precise stratification of patients with IPMN. Multicenter collaboration and datasets comprising various modalities are necessary to fully utilize this potential, alongside concerted efforts towards clinical translation.

Background Based on the Fukuoka and Kyoto international consensus guidelines, the current clinical management of intraductal papillary mucinous neoplasm (IPMN) largely depends on imaging features. While these criteria are highly sensitive in detecting high-risk IPMN, they lack specificity, resulting in surgical overtreatment. Artificial Intelligence (AI)-based medical image analysis has the potential to augment the clinical management of IPMNs by improving diagnostic accuracy. Methods Based on a systematic review of the academic literature on AI in IPMN imaging, 1041 publications were identified of which 25 published studies were included in the analysis. The studies were stratified based on prediction target, underlying data type and imaging modality, patient cohort size, and stage of clinical translation and were subsequently analyzed to identify trends and gaps in the field. Results Research on AI in IPMN imaging has been increasing in recent years. The majority of studies utilized CT imaging to train computational models. Most studies presented computational models developed on single-center datasets (n=11,44%) and included less than 250 patients (n=18,72%). Methodologically, convolutional neural network (CNN)-based algorithms were most commonly used. Thematically, most studies reported models augmenting differential diagnosis (n=9,36%) or risk stratification (n=10,40%) rather than IPMN detection (n=5,20%) or IPMN segmentation (n=2,8%). Conclusion This systematic review provides a comprehensive overview of the research landscape of AI in IPMN imaging. Computational models have potential to enhance the accurate and precise stratification of patients with IPMN. Multicenter collaboration and datasets comprising various modalities are necessary to fully utilize this potential, alongside concerted efforts towards clinical translation. Highlights Artificial Intelligence holds promise in the field of IPMN by augmenting IPMN detection, differentiation of different types of pancreatic cysts, stratifying malignant progression risk, and automating the analysis of IPMN imaging through computational cyst segmentation. The majority of studies related to AI-based analysis of IPMN imaging use single-center patient cohorts of less than 250 patients to develop and validate computational models and consider imaging as the only data modality. Reporting transparency of existing studies on AI in IPMN imaging is limited and there remains a scarcity of comprehensive, multimodal approaches as well as clinical translation.

Deteriorating water quality poses a substantial risk to human health, with billions at risk of waterborne diseases due to contamination. Insufficient water quality data augment risks as conventional monitoring methods lack comprehensive coverage. Technologies like the Internet of Things and machine learning offer real-time water quality monitoring and classification. IoT nodes often provide point data insufficient for monitoring the quality of entire water bodies. Remote sensing, though useful, has limitations such as measuring only optically active parameters and being affected by climate and resolution issues. To address these challenges, an unmanned surface vehicle named `AquaDrone' has been developed. AquaDrone traverses water bodies, collecting data of four key parameters (pH, dissolved oxygen, electrical conductivity, and temperature) along with GPS coordinates. The data is transmitted to a web portal via LoRa communication and Wi-Fi, where visualizations like data tables, trendlines and color-coded heatmaps are generated. A multilayer perceptron classifies water quality into five categories, aiding in real-time classification. A comparative analysis of various oversampling techniques has been conducted in the context of water quality classification. The AquaDrone offers a feasible solution for monitoring quality of small to medium-sized water bodies, crucial for safeguarding public health.