Abhinav Suri
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Abhinav Suri has a total of 132 co-authors. Here are the top co-authors by number of shared publications:
- Ronald M. Summers
Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Clinical Center, National Institutes of Health, Bethesda, USA
4 shared publications Ashley Terry
From the Department of Radiology and Orthopedic Surgery, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pa
3 shared publications- David A. Zarrin
David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, United States
3 shared publications Grace Choi
From the Department of Radiology and Orthopedic Surgery, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pa
3 shared publications- Pritam Mukherjee
National Institutes of Health
3 shared publications Sisi Tang
From the Department of Radiology and Orthopedic Surgery, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pa
3 shared publications
Latest Publications
- Open Access
- 10 November 2026
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 28, e81708
Labeling Clinical Notes With Large Language Models: Retrospective Benchmarking Study
Background: Clinical notes are rich sources of clinical information and are routinely used in clinical practice to infer past diagnoses and other comorbidities. Large language models (LLMs) may simplify the task by extracting this information from clinical notes. Objective: The study aims to evaluate the performance of state-of-the-art LLMs on the task of labeling English clinical notes. Methods: This institutional review board–approved retrospective study comprised 380 clinical notes obtained from 2 institutions in the United States. The notes were manually labeled for the presence or absence of 29 conditions (simplified Elixhauser Comorbidity Index) by 2 human readers—a radiology resident and a medical student under the supervision of a senior radiologist. Forty-six LLMs, including 4 closed-source commercial models from the GPT and Claude families, and 42 open-weights LLMs from the Llama (13 models) and the Qwen (29 models) families were used to label each note. The GPT-o3-mini-high reasoning model was used to parse the free-text model responses to binary yes/no answers. Using the manual labels as the reference standard, the recall, precision, and F 1 -scores for the models were evaluated. The effect of factors such as the length of the notes and the family, size, and release recency of the LLMs was analyzed using generalized linear models. Results: The top-performing LLMs included the GPT and Claude models, and the larger open-weights models included Llama 3 70B, Llama 3.1 70B, Llama 3.1 405B, Llama 3.3 70B, Qwen 1.5 72B, Qwen 1.5 110B, Qwen 2 72B, and Qwen 2 VL 72B, which achieved F 1 -scores above 70%. The most recent and higher-capacity releases of the LLMs have been improving mainly in their precision. The evaluation of 29 conditions showed a large variance in performance depending on the condition. Conclusions: In this evaluation with 46 LLMs, larger open-weights LLMs such as the Llama 3.3 70B performed on par with or better than the GPT and Claude models, achieving F 1 -scores above 70%. The performance of open-weights models improved significantly with model size and recency. These results suggest that locally run open-weights LLMs can be potentially used to label clinical notes.
- 10 November 2026
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 28, e81708
Open AccessLabeling Clinical Notes With Large Language Models: Retrospective Benchmarking Study
Background: Clinical notes are rich sources of clinical information and are routinely used in clinical practice to infer past diagnoses and other comorbidities. Large language models (LLMs) may simplify the task by extracting this information from clinical notes. Objective: The study aims to evaluate the performance of state-of-the-art LLMs on the task of labeling English clinical notes. Methods: This institutional review board–approved retrospective study comprised 380 clinical notes obtained from 2 institutions in the United States. The notes were manually labeled for the presence or absence of 29 conditions (simplified Elixhauser Comorbidity Index) by 2 human readers—a radiology resident and a medical student under the supervision of a senior radiologist. Forty-six LLMs, including 4 closed-source commercial models from the GPT and Claude families, and 42 open-weights LLMs from the Llama (13 models) and the Qwen (29 models) families were used to label each note. The GPT-o3-mini-high reasoning model was used to parse the free-text model responses to binary yes/no answers. Using the manual labels as the reference standard, the recall, precision, and F 1 -scores for the models were evaluated. The effect of factors such as the length of the notes and the family, size, and release recency of the LLMs was analyzed using generalized linear models. Results: The top-performing LLMs included the GPT and Claude models, and the larger open-weights models included Llama 3 70B, Llama 3.1 70B, Llama 3.1 405B, Llama 3.3 70B, Qwen 1.5 72B, Qwen 1.5 110B, Qwen 2 72B, and Qwen 2 VL 72B, which achieved F 1 -scores above 70%. The most recent and higher-capacity releases of the LLMs have been improving mainly in their precision. The evaluation of 29 conditions showed a large variance in performance depending on the condition. Conclusions: In this evaluation with 46 LLMs, larger open-weights LLMs such as the Llama 3.3 70B performed on par with or better than the GPT and Claude models, achieving F 1 -scores above 70%. The performance of open-weights models improved significantly with model size and recency. These results suggest that locally run open-weights LLMs can be potentially used to label clinical notes.
- 1 November 2026
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- Vol. 8 (6)
False Reassurances from False Negatives: AI Stroke Algorithms and Their Anatomical Blind Spots
- 1 November 2026
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- Vol. 8 (6)
False Reassurances from False Negatives: AI Stroke Algorithms and Their Anatomical Blind Spots
- 21 October 2026
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
TRAM-AI: A Framework for Transparency in Reporting of Medical Imaging Datasets for Artificial Intelligence Development
Purpose To develop and evaluate TRAM-AI, a consensus-based checklist for verifying whether medical imaging datasets described as publicly accessible are functionally usable for AI development. Materials and Methods A modified Delphi study was conducted. A steering committee generated candidate checklist items through a targeted literature review. An international expert panel ( n = 33) rated items on criticality and feasibility across two Delphi rounds. Consensus was defined as ≥ 75% agreement on both dimensions. The checklist was pilot-validated in 10 publicly available datasets. An advisory group of four experts from Latin America and sub-Saharan Africa evaluated cross-regional applicability. Results Of 16 candidate items, 14 were retained in the final checklist; one item was removed because of feasibility concerns, and another was merged with an existing item because of conceptual redundancy. All 14 final items achieved consensus for both criticality and feasibility. The final TRAM-AI checklist comprises 14 items across four domains: Dataset Access and Persistence (3 items), Technical Interoperability (4 items), Metadata and Reproducibility (4 items), and Ethical, Legal, and Maintenance considerations (3 items). In pilot validation, all items were applicable to all 10 datasets, compliance was 90% for trusted repository, de-dentification, and subject-level mapping, 10% for archive validation, and 30% for partition keys. All four advisory group members endorsed the checklist. Conclusion TRAM-AI provides a practical framework for verifying the functional accessibility of medical imaging datasets, complementing existing methodological reporting standards. Further evaluation and broader validation across diverse geographic settings are warranted. ©RSNA, 2026
- 21 October 2026
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
TRAM-AI: A Framework for Transparency in Reporting of Medical Imaging Datasets for Artificial Intelligence Development
Purpose To develop and evaluate TRAM-AI, a consensus-based checklist for verifying whether medical imaging datasets described as publicly accessible are functionally usable for AI development. Materials and Methods A modified Delphi study was conducted. A steering committee generated candidate checklist items through a targeted literature review. An international expert panel ( n = 33) rated items on criticality and feasibility across two Delphi rounds. Consensus was defined as ≥ 75% agreement on both dimensions. The checklist was pilot-validated in 10 publicly available datasets. An advisory group of four experts from Latin America and sub-Saharan Africa evaluated cross-regional applicability. Results Of 16 candidate items, 14 were retained in the final checklist; one item was removed because of feasibility concerns, and another was merged with an existing item because of conceptual redundancy. All 14 final items achieved consensus for both criticality and feasibility. The final TRAM-AI checklist comprises 14 items across four domains: Dataset Access and Persistence (3 items), Technical Interoperability (4 items), Metadata and Reproducibility (4 items), and Ethical, Legal, and Maintenance considerations (3 items). In pilot validation, all items were applicable to all 10 datasets, compliance was 90% for trusted repository, de-dentification, and subject-level mapping, 10% for archive validation, and 30% for partition keys. All four advisory group members endorsed the checklist. Conclusion TRAM-AI provides a practical framework for verifying the functional accessibility of medical imaging datasets, complementing existing methodological reporting standards. Further evaluation and broader validation across diverse geographic settings are warranted. ©RSNA, 2026
- 4 December 2025
- Published by Massachusetts Medical Society in New England Journal of Medicine
- Vol. 393 (22) , 2268-2270
Simplify or Stratify? The Debate over Medical School Grading Systems
- 4 December 2025
- Published by Massachusetts Medical Society in New England Journal of Medicine
- Vol. 393 (22) , 2268-2270
Simplify or Stratify? The Debate over Medical School Grading Systems
- Open Access
- 4 December 2025
- Published by JMIR Publications Inc. in JMIR AI
- Vol. 4, e72535
The Perceived Roles of AI in Clinical Practice: National Survey of 941 Academic Physicians
Background: Artificial intelligence (AI) and machine learning models are frequently developed in medical research to optimize patient care, yet they remain rarely used in clinical practice. Objective: This study aims to understand the disconnect between model development and implementation by surveying physicians of all specialties across the United States. Methods: The present survey was distributed to residency coordinators at Accreditation Council for Graduate Medical Education–accredited residency programs to disseminate among attending physicians and resident physicians affiliated with their academic institution. Respondents were asked to identify and quantify the extent of their training and specialization, as well as the type and location of their practice. Physicians were then asked follow-up questions regarding AI in their practice, including whether its use is permitted, whether they would use it if made available, primary reasons for using or not using AI, elements that would encourage its use, and ethical concerns. Results: Of the 941 physicians who responded to the survey, 384 (40.8%) were attending physicians and 557 (59.2%) were resident physicians. The majority of the physicians (651/795, 81.9%) indicated that they would adopt AI in clinical practice if given the opportunity. The most cited intended uses for AI were risk stratification, image analysis or segmentation, and disease prognosis. The most common reservations were concerns about clinical errors made by AI and the potential to replicate human biases. Conclusions: To date, this study comprises the largest and most diverse dataset of physician perspectives on AI. Our results emphasize that most academic physicians in the United States are open to adopting AI in their clinical practice. However, for AI to become clinically relevant, developers and physicians must work synergistically to design models that are accurate, accessible, and intuitive while thoroughly addressing ethical concerns associated with the implementation of AI in medicine.
- 4 December 2025
- Published by JMIR Publications Inc. in JMIR AI
- Vol. 4, e72535
Open AccessThe Perceived Roles of AI in Clinical Practice: National Survey of 941 Academic Physicians
Background: Artificial intelligence (AI) and machine learning models are frequently developed in medical research to optimize patient care, yet they remain rarely used in clinical practice. Objective: This study aims to understand the disconnect between model development and implementation by surveying physicians of all specialties across the United States. Methods: The present survey was distributed to residency coordinators at Accreditation Council for Graduate Medical Education–accredited residency programs to disseminate among attending physicians and resident physicians affiliated with their academic institution. Respondents were asked to identify and quantify the extent of their training and specialization, as well as the type and location of their practice. Physicians were then asked follow-up questions regarding AI in their practice, including whether its use is permitted, whether they would use it if made available, primary reasons for using or not using AI, elements that would encourage its use, and ethical concerns. Results: Of the 941 physicians who responded to the survey, 384 (40.8%) were attending physicians and 557 (59.2%) were resident physicians. The majority of the physicians (651/795, 81.9%) indicated that they would adopt AI in clinical practice if given the opportunity. The most cited intended uses for AI were risk stratification, image analysis or segmentation, and disease prognosis. The most common reservations were concerns about clinical errors made by AI and the potential to replicate human biases. Conclusions: To date, this study comprises the largest and most diverse dataset of physician perspectives on AI. Our results emphasize that most academic physicians in the United States are open to adopting AI in their clinical practice. However, for AI to become clinically relevant, developers and physicians must work synergistically to design models that are accurate, accessible, and intuitive while thoroughly addressing ethical concerns associated with the implementation of AI in medicine.
- 1 May 2025
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- Vol. 7 (3) , e250081
Artificial Intelligence Is Brittle: We Need to Do Better
- 1 May 2025
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- Vol. 7 (3) , e250081
Artificial Intelligence Is Brittle: We Need to Do Better
- 1 April 2025
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 315 (1) , e241979
MRISegmenter: A Fully Accurate and Robust Automated Multiorgan and Structure Segmentation Tool for T1-weighted Abdominal MRI
MRISegmenter provided accurate and robust segmentation of 62 organs and structures, including 15 main organs, seven vessels, eight muscles, and 32 bones at T1-weighted abdominal MRI.
- 1 April 2025
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 315 (1) , e241979
MRISegmenter: A Fully Accurate and Robust Automated Multiorgan and Structure Segmentation Tool for T1-weighted Abdominal MRI
MRISegmenter provided accurate and robust segmentation of 62 organs and structures, including 15 main organs, seven vessels, eight muscles, and 32 bones at T1-weighted abdominal MRI.
- Open Access
- 11 February 2025
- Published by JMIR Publications Inc. in JMIR Preprints
The Perceived Roles of AI in Clinical Practice: National Survey of 941 Academic Physicians (Preprint)
BACKGROUND Artificial intelligence (AI) and machine learning models are frequently developed in medical research to optimize patient care, yet they remain rarely used in clinical practice. OBJECTIVE This study aims to understand the disconnect between model development and implementation by surveying physicians of all specialties across the United States. METHODS The present survey was distributed to residency coordinators at Accreditation Council for Graduate Medical Education–accredited residency programs to disseminate among attending physicians and resident physicians affiliated with their academic institution. Respondents were asked to identify and quantify the extent of their training and specialization, as well as the type and location of their practice. Physicians were then asked follow-up questions regarding AI in their practice, including whether its use is permitted, whether they would use it if made available, primary reasons for using or not using AI, elements that would encourage its use, and ethical concerns. RESULTS Of the 941 physicians who responded to the survey, 384 (40.8%) were attending physicians and 557 (59.2%) were resident physicians. The majority of the physicians (651/795, 81.9%) indicated that they would adopt AI in clinical practice if given the opportunity. The most cited intended uses for AI were risk stratification, image analysis or segmentation, and disease prognosis. The most common reservations were concerns about clinical errors made by AI and the potential to replicate human biases. CONCLUSIONS To date, this study comprises the largest and most diverse dataset of physician perspectives on AI. Our results emphasize that most academic physicians in the United States are open to adopting AI in their clinical practice. However, for AI to become clinically relevant, developers and physicians must work synergistically to design models that are accurate, accessible, and intuitive while thoroughly addressing ethical concerns associated with the implementation of AI in medicine. CLINICALTRIAL
- 11 February 2025
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessThe Perceived Roles of AI in Clinical Practice: National Survey of 941 Academic Physicians (Preprint)
BACKGROUND Artificial intelligence (AI) and machine learning models are frequently developed in medical research to optimize patient care, yet they remain rarely used in clinical practice. OBJECTIVE This study aims to understand the disconnect between model development and implementation by surveying physicians of all specialties across the United States. METHODS The present survey was distributed to residency coordinators at Accreditation Council for Graduate Medical Education–accredited residency programs to disseminate among attending physicians and resident physicians affiliated with their academic institution. Respondents were asked to identify and quantify the extent of their training and specialization, as well as the type and location of their practice. Physicians were then asked follow-up questions regarding AI in their practice, including whether its use is permitted, whether they would use it if made available, primary reasons for using or not using AI, elements that would encourage its use, and ethical concerns. RESULTS Of the 941 physicians who responded to the survey, 384 (40.8%) were attending physicians and 557 (59.2%) were resident physicians. The majority of the physicians (651/795, 81.9%) indicated that they would adopt AI in clinical practice if given the opportunity. The most cited intended uses for AI were risk stratification, image analysis or segmentation, and disease prognosis. The most common reservations were concerns about clinical errors made by AI and the potential to replicate human biases. CONCLUSIONS To date, this study comprises the largest and most diverse dataset of physician perspectives on AI. Our results emphasize that most academic physicians in the United States are open to adopting AI in their clinical practice. However, for AI to become clinically relevant, developers and physicians must work synergistically to design models that are accurate, accessible, and intuitive while thoroughly addressing ethical concerns associated with the implementation of AI in medicine. CLINICALTRIAL
- 1 November 2024
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- Vol. 6 (6) , e240624
AI as a Second Reader Can Reduce Radiologists’ Workload and Increase Accuracy in Screening Mammography
- 1 November 2024
- Published by Radiological Society of North America (RSNA) in Radiology: Artificial Intelligence
- Vol. 6 (6) , e240624
AI as a Second Reader Can Reduce Radiologists’ Workload and Increase Accuracy in Screening Mammography
- 1 October 2024
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 313 (1) , e240609
Evaluation of GPT Large Language Model Performance on RSNA 2023 Case of the Day Questions
Human raters did not significantly outperform OpenAI’s GPT-4V in assessing RSNA 2023 Case of the Day questions nor did their accuracy improve with the assistance of GPT-4V.
- 1 October 2024
- Published by Radiological Society of North America (RSNA) in Radiology
- Vol. 313 (1) , e240609
Evaluation of GPT Large Language Model Performance on RSNA 2023 Case of the Day Questions
Human raters did not significantly outperform OpenAI’s GPT-4V in assessing RSNA 2023 Case of the Day questions nor did their accuracy improve with the assistance of GPT-4V.