Yu Gan
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Yu Gan has a total of 176 co-authors. Here are the top co-authors by number of shared publications:
- Christine P. Hendon
Department of Electrical Engineering Columbia University New York New York USA
16 shared publications - Xinwen Yao
Nanyang Technological University
13 shared publications - Yuye Ling
University of Oklahoma
11 shared publications Christine P. Hendon
Department of Electrical Engineering, Columbia University, New York, New York
10 shared publicationsXueshen Li
Department of Biomedical Engineering and Centers for Health Innovation, Stevens Institute of Technology, Hoboken, NJ, USA
10 shared publicationsCharles C. Marboe
Department of Pathology and Cell Biology Columbia University Medical Center New York New York
8 shared publications
Latest Publications
- Open Access
- 16 November 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computing for Healthcare
- Vol. 8 (1) , 1-20
Diagnostic Report Generation via a ProActive Multimodal Agentic System
Automated diagnostic report generation lies at the core of clinical diagnosis and can alleviate clinician shortages. Existing diagnostic report generation methods have two major limitations: they rely on unimodal inputs (e.g., images), ignoring textual biomarkers like medical history, and lack proactive dialogue capabilities to elicit personalized clinical information. To address those issues, we propose a ProActive Multimodal Agentic (PAMA) system, which performs a comprehensive disease analysis by examining biomarkers in diverse sources, including multi-view medical images, medical histories, and diagnostic conversations. Built upon a knowledge graph and recommendation-based dialogue architecture, PAMA actively initiates adaptive, multi-turn conversations with patients, which is integrated with visual data for robust and reliable report generation. Specifically, PAMA actively generates adaptive multi-turn questions to collect clinically relevant background information and then fuses the resulting dialogue context with visual representations for robust diagnostic report generation. We validate our approach on two real-world benchmark datasets, MIMIC-CXR and IU-X-ray, through extensive quantitative evaluations and comparisons with state-of-the-art baselines. Furthermore, we conduct user studies to assess the realism and clinical quality of the generated reports. Finally, we present real-world case studies to examine the performance of our system across diverse scenarios, demonstrating its robustness on scalability, complexity, and data variability.
- 16 November 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computing for Healthcare
- Vol. 8 (1) , 1-20
Open AccessDiagnostic Report Generation via a ProActive Multimodal Agentic System
Automated diagnostic report generation lies at the core of clinical diagnosis and can alleviate clinician shortages. Existing diagnostic report generation methods have two major limitations: they rely on unimodal inputs (e.g., images), ignoring textual biomarkers like medical history, and lack proactive dialogue capabilities to elicit personalized clinical information. To address those issues, we propose a ProActive Multimodal Agentic (PAMA) system, which performs a comprehensive disease analysis by examining biomarkers in diverse sources, including multi-view medical images, medical histories, and diagnostic conversations. Built upon a knowledge graph and recommendation-based dialogue architecture, PAMA actively initiates adaptive, multi-turn conversations with patients, which is integrated with visual data for robust and reliable report generation. Specifically, PAMA actively generates adaptive multi-turn questions to collect clinically relevant background information and then fuses the resulting dialogue context with visual representations for robust diagnostic report generation. We validate our approach on two real-world benchmark datasets, MIMIC-CXR and IU-X-ray, through extensive quantitative evaluations and comparisons with state-of-the-art baselines. Furthermore, we conduct user studies to assess the realism and clinical quality of the generated reports. Finally, we present real-world case studies to examine the performance of our system across diverse scenarios, demonstrating its robustness on scalability, complexity, and data variability.
- Top Cited PapersOpen Access
- 18 August 2026
- Published by JMIR Publications Inc. in JMIR AI
- Vol. 5, e92528
Objective Assessment of Medication Adherence Through Automated Pill Detection Using a Computer Vision Framework: Development and Evaluation Study
Background: Medication adherence remains a significant concern in both clinical practice and public health. Nonadherence to prescribed medication regimens is associated with poorer health outcomes, higher rates of hospitalization, and increased financial burdens on health care systems. Despite its critical role in ensuring treatment efficacy, adherence assessment is still largely dependent on patient self-reports, pharmacy refill records, or caregiver observations—methods that are often subjective, inconsistent, and unreliable. Therefore, there is an urgent need for objective, automated solutions to accurately monitor medication, particularly the information on the pills taken and intake behavior. AI techniques, especially computer vision, are promising solutions; however, their application in assessing medication adherence has several key pitfalls in maintaining robustness under environmental variation that limit their reliability and real-world applicability. Objective: This study aimed to develop a reliable and accurate computer vision framework for pill detection and counting using mobile phone images captured under diverse real-world conditions to support medication adherence assessment. Methods: This study is carried out using a public dataset on pill detection (N=152) and a curated dataset (N=60) on medication adherence assessment. Inspired by mixture of experts (MoE), we propose and implement a robust pill detection framework that integrates two expert-level object detection models, You Only Look Once (YOLOv12) and Faster Region-based Convolutional Neural Network (Faster R-CNN), with a Segment Anything Model (SAM). The pill counting performance was evaluated using mean absolute error (MAE) and accuracy. We further conduct a pilot study on a curated dataset that mimics a subject taking medicine for a period of 20 days. We evaluate sensitivity and specificity in medication adherence assessment. Results: The proposed multimodel framework substantially reduces both error types, achieving a low MAE of 0.167 on the Kaggle test set in pill counting and an accuracy of 0.933 in pill detection. In medication adherence simulation, the proposed framework achieves high accuracy (0.917), sensitivity (0.933), and specificity (0.900), significantly outperforming the evaluated YOLOv12 and Faster R-CNN baselines ( P <.05). Performance remained robust on the self-collected dataset in medication-taking scenarios in the settings of household, restaurant, office, and outdoor environments. Conclusions: The proposed framework achieved high performance in pill counting, detection, and simulated medication-adherence classification. This proof-of-concept study has demonstrated the potential to enhance medication adherence among patients and support more reliable treatment monitoring.
- 18 August 2026
- Published by JMIR Publications Inc. in JMIR AI
- Vol. 5, e92528
Top Cited PapersOpen AccessObjective Assessment of Medication Adherence Through Automated Pill Detection Using a Computer Vision Framework: Development and Evaluation Study
Background: Medication adherence remains a significant concern in both clinical practice and public health. Nonadherence to prescribed medication regimens is associated with poorer health outcomes, higher rates of hospitalization, and increased financial burdens on health care systems. Despite its critical role in ensuring treatment efficacy, adherence assessment is still largely dependent on patient self-reports, pharmacy refill records, or caregiver observations—methods that are often subjective, inconsistent, and unreliable. Therefore, there is an urgent need for objective, automated solutions to accurately monitor medication, particularly the information on the pills taken and intake behavior. AI techniques, especially computer vision, are promising solutions; however, their application in assessing medication adherence has several key pitfalls in maintaining robustness under environmental variation that limit their reliability and real-world applicability. Objective: This study aimed to develop a reliable and accurate computer vision framework for pill detection and counting using mobile phone images captured under diverse real-world conditions to support medication adherence assessment. Methods: This study is carried out using a public dataset on pill detection (N=152) and a curated dataset (N=60) on medication adherence assessment. Inspired by mixture of experts (MoE), we propose and implement a robust pill detection framework that integrates two expert-level object detection models, You Only Look Once (YOLOv12) and Faster Region-based Convolutional Neural Network (Faster R-CNN), with a Segment Anything Model (SAM). The pill counting performance was evaluated using mean absolute error (MAE) and accuracy. We further conduct a pilot study on a curated dataset that mimics a subject taking medicine for a period of 20 days. We evaluate sensitivity and specificity in medication adherence assessment. Results: The proposed multimodel framework substantially reduces both error types, achieving a low MAE of 0.167 on the Kaggle test set in pill counting and an accuracy of 0.933 in pill detection. In medication adherence simulation, the proposed framework achieves high accuracy (0.917), sensitivity (0.933), and specificity (0.900), significantly outperforming the evaluated YOLOv12 and Faster R-CNN baselines ( P <.05). Performance remained robust on the self-collected dataset in medication-taking scenarios in the settings of household, restaurant, office, and outdoor environments. Conclusions: The proposed framework achieved high performance in pill counting, detection, and simulated medication-adherence classification. This proof-of-concept study has demonstrated the potential to enhance medication adherence among patients and support more reliable treatment monitoring.
- Open Access
- 3 August 2026
- Published by Elsevier in Journal of Agriculture and Food Research
- Vol. 30
Non-invasive microwave imaging for metallic contaminant detection in packaged peanut butter
- 3 August 2026
- Published by Elsevier in Journal of Agriculture and Food Research
- Vol. 30
Open AccessNon-invasive microwave imaging for metallic contaminant detection in packaged peanut butter
- Open Access
- 27 June 2026
- Published by Elsevier in Future Foods
- Vol. 14
Hybrid loss for robust ripeness classification under noisy labels
- 27 June 2026
- Published by Elsevier in Future Foods
- Vol. 14
Open AccessHybrid loss for robust ripeness classification under noisy labels
- 10 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 4823-4834
Screen Detection From Egocentric Image Streams Leveraging Multi-View Vision Language Model
Accurately monitoring the screen exposure of young children is important for research related to screen use such as childhood obesity, physical activity, and social interaction. Most existing studies rely upon self-report or manual measures from bulky wearable sensors, thus lacking efficiency and accuracy in capturing quantitative screen exposure data. In this work, we developed a novel screen detection framework that utilizes egocentric images from a wearable sensor, named the screen time tracker (STT), and a vision language model (VLM). In particular, we devised a multi-view VLM that takes multiple views from egocentric image streams and interprets screen exposure dynamically. We validated our approach by using a dataset of children's free-living activities, demonstrating significant improvement over existing methods in conventional vision language models and object detection models. The combination of vision language model and lightweight hardware design provides a novel solution in screen detection for children. The proposed framework has great potential to benefit children's behavioral study.
- 10 February 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 28 (15209210) , 4823-4834
Screen Detection From Egocentric Image Streams Leveraging Multi-View Vision Language Model
Accurately monitoring the screen exposure of young children is important for research related to screen use such as childhood obesity, physical activity, and social interaction. Most existing studies rely upon self-report or manual measures from bulky wearable sensors, thus lacking efficiency and accuracy in capturing quantitative screen exposure data. In this work, we developed a novel screen detection framework that utilizes egocentric images from a wearable sensor, named the screen time tracker (STT), and a vision language model (VLM). In particular, we devised a multi-view VLM that takes multiple views from egocentric image streams and interprets screen exposure dynamically. We validated our approach by using a dataset of children's free-living activities, demonstrating significant improvement over existing methods in conventional vision language models and object detection models. The combination of vision language model and lightweight hardware design provides a novel solution in screen detection for children. The proposed framework has great potential to benefit children's behavioral study.
- Open Access
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 137588-137599
Objective Assessment of Media Screens Effect on Physical Activity in Preschool Children
- 1 January 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 137588-137599
Open AccessObjective Assessment of Media Screens Effect on Physical Activity in Preschool Children
- 28 April 2025
- Published by Optica Publishing Group in Biomedical Optics Express
- Vol. 16 (5) , 2061-2076
Automated cell properties toolbox from 3D bioprinted hydrogel scaffolds via deep learning and optical coherence tomography
Accurately assessing cell viability and morphological properties within 3D bioprinted hydrogel scaffolds is essential for tissue engineering but remains challenging due to the limitations of existing invasive and threshold-based methods. We present a computational toolbox that automates cell viability analysis and quantifies key properties such as elongation, flatness, and surface roughness. This framework integrates optical coherence tomography (OCT) with deep learning-based segmentation, achieving a mean segmentation precision of 88.96%. By leveraging OCT’s high-resolution imaging with deep learning-based segmentation, our novel approach enables non-invasive, quantitative analysis, which can advance rapid monitoring of 3D cell cultures for regenerative medicine and biomaterial research.
- 28 April 2025
- Published by Optica Publishing Group in Biomedical Optics Express
- Vol. 16 (5) , 2061-2076
Automated cell properties toolbox from 3D bioprinted hydrogel scaffolds via deep learning and optical coherence tomography
Accurately assessing cell viability and morphological properties within 3D bioprinted hydrogel scaffolds is essential for tissue engineering but remains challenging due to the limitations of existing invasive and threshold-based methods. We present a computational toolbox that automates cell viability analysis and quantifies key properties such as elongation, flatness, and surface roughness. This framework integrates optical coherence tomography (OCT) with deep learning-based segmentation, achieving a mean segmentation precision of 88.96%. By leveraging OCT’s high-resolution imaging with deep learning-based segmentation, our novel approach enables non-invasive, quantitative analysis, which can advance rapid monitoring of 3D cell cultures for regenerative medicine and biomaterial research.