Steven Lin
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Steven Lin has a total of 140 co-authors. Here are the top co-authors by number of shared publications:
Amelia Sattler
Stanford Healthcare AI Applied Research Team, Division of Primary Care and Population Health, Department of Medicine, Stanford University School of Medicine, Stanford, CA
8 shared publicationsCathina Nguyen
Stanford Medical Scribe Fellowship, Division of Primary Care and Population Health, Department of Medicine, Stanford University School of Medicine, Stanford, CA
6 shared publicationsErika Schillinger
Division of Primary Care, Department of Medicine, Stanford University School of Medicine, Stanford, California, USA
6 shared publications- Anna Devon-Sand
Division of Primary Care and Population Health, Stanford School of Medicine, Stanford University, Palo Alto, California, USA
3 shared publications Dale R. Webster
Google Research, Mountain View, CA, USA
3 shared publicationsGrace Hong
3 shared publications
Latest Publications
- 23 June 2026
- Published by Massachusetts Medical Society in NEJM AI
- Vol. 3 (7)
SmartAlert — Implementing Machine Learning–Driven Clinical Decision Support for Inpatient Laboratory Utilization Reduction
Repetitive laboratory testing that is unlikely to yield clinically useful information is a common practice that burdens patients and increases health care costs. Education and feedback interventions have limited success, while general test ordering restrictions and electronic alerts impede appropriate clinical care. We introduce and evaluate SmartAlert, a machine learning–driven clinical decision support (CDS) system integrated into the electronic health record that predicts stable laboratory results to reduce unnecessary repeat testing. This Case Study describes the implementation process, challenges, and lessons learned from deploying SmartAlert targeting complete blood count (CBC) utilization in a randomized controlled pilot across 9270 admissions in eight acute care units across two hospitals between August 15, 2024, and March 15, 2025. The results show a significant decrease in the number of CBC results within 52 hours of SmartAlert display (1.54 vs. 1.82; P<0.01) without adverse effect on secondary safety outcomes, representing a 15% relative reduction in repetitive testing. Implementation lessons learned include interpretation of probabilistic model predictions in clinical contexts, stakeholder engagement to define acceptable model behavior, governance processes for deploying a complex model in a clinical environment, user interface design considerations, alignment with clinical operational priorities, and the value of qualitative feedback from end users. In conclusion, a machine learning–driven CDS system backed by a deliberate implementation and governance process can provide precision guidance on inpatient laboratory testing to safely reduce unnecessary repetitive testing. (Funded by the Agency for Science, Technology, and Research and others.)
- 23 June 2026
- Published by Massachusetts Medical Society in NEJM AI
- Vol. 3 (7)
SmartAlert — Implementing Machine Learning–Driven Clinical Decision Support for Inpatient Laboratory Utilization Reduction
Repetitive laboratory testing that is unlikely to yield clinically useful information is a common practice that burdens patients and increases health care costs. Education and feedback interventions have limited success, while general test ordering restrictions and electronic alerts impede appropriate clinical care. We introduce and evaluate SmartAlert, a machine learning–driven clinical decision support (CDS) system integrated into the electronic health record that predicts stable laboratory results to reduce unnecessary repeat testing. This Case Study describes the implementation process, challenges, and lessons learned from deploying SmartAlert targeting complete blood count (CBC) utilization in a randomized controlled pilot across 9270 admissions in eight acute care units across two hospitals between August 15, 2024, and March 15, 2025. The results show a significant decrease in the number of CBC results within 52 hours of SmartAlert display (1.54 vs. 1.82; P<0.01) without adverse effect on secondary safety outcomes, representing a 15% relative reduction in repetitive testing. Implementation lessons learned include interpretation of probabilistic model predictions in clinical contexts, stakeholder engagement to define acceptable model behavior, governance processes for deploying a complex model in a clinical environment, user interface design considerations, alignment with clinical operational priorities, and the value of qualitative feedback from end users. In conclusion, a machine learning–driven CDS system backed by a deliberate implementation and governance process can provide precision guidance on inpatient laboratory testing to safely reduce unnecessary repetitive testing. (Funded by the Agency for Science, Technology, and Research and others.)
- Open Access
- 15 May 2026
- Published by Society of Teachers of Family Medicine in Family Medicine
- Vol. 58 (5) , 397-398
Irreplaceable: Five Enduring Roles of the Physician in the AI Era
- 15 May 2026
- Published by Society of Teachers of Family Medicine in Family Medicine
- Vol. 58 (5) , 397-398
Open AccessIrreplaceable: Five Enduring Roles of the Physician in the AI Era
- Open Access
- 5 March 2026
- Published by Society of Teachers of Family Medicine in Family Medicine
- Vol. 58 (3) , 253-254
The Power of Enough
- 5 March 2026
- Published by Society of Teachers of Family Medicine in Family Medicine
- Vol. 58 (3) , 253-254
Open AccessThe Power of Enough
- Open Access
- 2 June 2025
- Published by Elsevier in EBioMedicine
- Vol. 116, 105766
Closing the AI generalisation gap by adjusting for dermatology condition distribution differences across clinical settings
- 2 June 2025
- Published by Elsevier in EBioMedicine
- Vol. 116, 105766
Open AccessClosing the AI generalisation gap by adjusting for dermatology condition distribution differences across clinical settings
- Open Access
Crowdsourcing Dermatology Images with Google Search Ads: Creating a Real-World Skin Condition Dataset
Background: Health datasets from clinical sources do not reflect the breadth and diversity of disease in the real world, impacting research, medical education, and artificial intelligence (AI) tool development. Dermatology is a suitable area to develop and test a new and scalable method to create representative health datasets. Methods: We used Google Search advertisements to invite contributions to an open access dataset of images of dermatology conditions, demographic and symptom information. With informed contributor consent, we describe and release this dataset containing 10,408 images from 5,033 contributions from internet users in the United States over 8 months starting March 2023. The dataset includes dermatologist condition labels as well as estimated Fitzpatrick Skin Type (eFST) and Monk Skin Tone (eMST) labels for the images. Results: We received a median of 22 submissions/day (IQR 14-30). Female (66.72%) and younger (52% < age 40) contributors had a higher representation in the dataset compared to the US population, and 32.6% of contributors reported a non-White racial or ethnic identity. Over 97.5% of contributions were genuine images of skin conditions. Dermatologist confidence in assigning a differential diagnosis increased with the number of available variables, and showed a weaker correlation with image sharpness (Spearman's P values <0.001 and 0.01 respectively). Most contributions were short-duration (54% with onset < 7 days ago ) and 89% were allergic, infectious, or inflammatory conditions. eFST and eMST distributions reflected the geographical origin of the dataset. The dataset is available at github.com/google-research-datasets/scin . Conclusion: Search ads are effective at crowdsourcing images of health conditions. The SCIN dataset bridges important gaps in the availability of representative images of common skin conditions.
Open AccessCrowdsourcing Dermatology Images with Google Search Ads: Creating a Real-World Skin Condition Dataset
Background: Health datasets from clinical sources do not reflect the breadth and diversity of disease in the real world, impacting research, medical education, and artificial intelligence (AI) tool development. Dermatology is a suitable area to develop and test a new and scalable method to create representative health datasets. Methods: We used Google Search advertisements to invite contributions to an open access dataset of images of dermatology conditions, demographic and symptom information. With informed contributor consent, we describe and release this dataset containing 10,408 images from 5,033 contributions from internet users in the United States over 8 months starting March 2023. The dataset includes dermatologist condition labels as well as estimated Fitzpatrick Skin Type (eFST) and Monk Skin Tone (eMST) labels for the images. Results: We received a median of 22 submissions/day (IQR 14-30). Female (66.72%) and younger (52% < age 40) contributors had a higher representation in the dataset compared to the US population, and 32.6% of contributors reported a non-White racial or ethnic identity. Over 97.5% of contributions were genuine images of skin conditions. Dermatologist confidence in assigning a differential diagnosis increased with the number of available variables, and showed a weaker correlation with image sharpness (Spearman's P values <0.001 and 0.01 respectively). Most contributions were short-duration (54% with onset < 7 days ago ) and 89% were allergic, infectious, or inflammatory conditions. eFST and eMST distributions reflected the geographical origin of the dataset. The dataset is available at github.com/google-research-datasets/scin . Conclusion: Search ads are effective at crowdsourcing images of health conditions. The SCIN dataset bridges important gaps in the availability of representative images of common skin conditions.
- Open Access
Closing the AI generalization gap by adjusting for dermatology condition distribution differences across clinical settings
Recently, there has been great progress in the ability of artificial intelligence (AI) algorithms to classify dermatological conditions from clinical photographs. However, little is known about the robustness of these algorithms in real-world settings where several factors can lead to a loss of generalizability. Understanding and overcoming these limitations will permit the development of generalizable AI that can aid in the diagnosis of skin conditions across a variety of clinical settings. In this retrospective study, we demonstrate that differences in skin condition distribution, rather than in demographics or image capture mode are the main source of errors when an AI algorithm is evaluated on data from a previously unseen source. We demonstrate a series of steps to close this generalization gap, requiring progressively more information about the new source, ranging from the condition distribution to training data enriched for data less frequently seen during training. Our results also suggest comparable performance from end-to-end fine tuning versus fine tuning solely the classification layer on top of a frozen embedding model. Our approach can inform the adaptation of AI algorithms to new settings, based on the information and resources available.
Open AccessClosing the AI generalization gap by adjusting for dermatology condition distribution differences across clinical settings
Recently, there has been great progress in the ability of artificial intelligence (AI) algorithms to classify dermatological conditions from clinical photographs. However, little is known about the robustness of these algorithms in real-world settings where several factors can lead to a loss of generalizability. Understanding and overcoming these limitations will permit the development of generalizable AI that can aid in the diagnosis of skin conditions across a variety of clinical settings. In this retrospective study, we demonstrate that differences in skin condition distribution, rather than in demographics or image capture mode are the main source of errors when an AI algorithm is evaluated on data from a previously unseen source. We demonstrate a series of steps to close this generalization gap, requiring progressively more information about the new source, ranging from the condition distribution to training data enriched for data less frequently seen during training. Our results also suggest comparable performance from end-to-end fine tuning versus fine tuning solely the classification layer on top of a frozen embedding model. Our approach can inform the adaptation of AI algorithms to new settings, based on the information and resources available.
- Open Access
- 23 February 2024
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 26, e47667
Building Pandemic-Resilient Primary Care Systems: Lessons Learned From COVID-19
On January 30, 2023, the Biden Administration announced its intention to end the existing COVID-19 public health emergency declaration. The transition to a “postpandemic” landscape presents a unique opportunity to sustain and strengthen pandemic-era changes in care delivery. With this in mind, we present 3 critical lessons learned from a primary care perspective during the COVID-19 pandemic. First, clinical workflows must support both in-person and internet-based care delivery. Second, the integration of asynchronous care delivery is critical. Third, planning for the future means planning for everyone, including those with potentially limited access to health care due to barriers in technology and communication. While these lessons are neither unique to primary care settings nor all-encompassing, they establish a grounded foundation on which to construct higher-quality, more resilient, and more equitable health systems.
- 23 February 2024
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 26, e47667
Open AccessBuilding Pandemic-Resilient Primary Care Systems: Lessons Learned From COVID-19
On January 30, 2023, the Biden Administration announced its intention to end the existing COVID-19 public health emergency declaration. The transition to a “postpandemic” landscape presents a unique opportunity to sustain and strengthen pandemic-era changes in care delivery. With this in mind, we present 3 critical lessons learned from a primary care perspective during the COVID-19 pandemic. First, clinical workflows must support both in-person and internet-based care delivery. Second, the integration of asynchronous care delivery is critical. Third, planning for the future means planning for everyone, including those with potentially limited access to health care due to barriers in technology and communication. While these lessons are neither unique to primary care settings nor all-encompassing, they establish a grounded foundation on which to construct higher-quality, more resilient, and more equitable health systems.
- Open Access
- 2 June 2023
- Published by Society of Teachers of Family Medicine in Family Medicine
- Vol. 55 (6) , 389-393
Impact of a Medical Scribe Program on the Educational Trajectory of Prehealth and Underrepresented in Medicine Students
Background and Objectives: Sparse research exists on evaluating the effects of medical scribing programs on the educational trajectory of prehealth students. This study assesses the impact of the Stanford Medical Scribe Fellowship (COMET) on its prehealth participants’ educational goals, preparation for graduate training, and acceptance into health professional schools. Methods: We distributed a 31-question survey with both closed- and open-ended questions to 96 alumni. The survey collected participant demographics, self-reported underrepresented in medicine (URM) status, pre-COMET clinical experiences and educational goals, application to and acceptance at health professional schools, and perceived impact of COMET on their educational trajectory. SPSS was used to complete the analyses. Results: The survey had a 97% (93/96) completion rate. Among all respondents, 69% (64/93) applied to a health professional school and 70% (45/64) were accepted. Among URM respondents, 68% (23/34) applied to a health professional school and 70% (16/23) were accepted. Overall acceptance rates for MD/DO and PA/NP programs were 51% (24/47) and 61% (11/18), respectively. URM acceptance rates for MD/DO and PA/NP programs were 43% (3/7) and 58% (7/12), respectively. For current or recently graduated health professional school respondents, 97% (37/38) “strongly agreed” or “agreed” that COMET helped them succeed in their training. Conclusions: COMET is associated with a positive impact on the educational trajectory of its prehealth participants and a higher acceptance rate into health professional schools than the national rates for both overall and URM applicants. Scribing programs may serve as pipeline development and help increase the diversity of the future health care workforce.
- 2 June 2023
- Published by Society of Teachers of Family Medicine in Family Medicine
- Vol. 55 (6) , 389-393
Open AccessImpact of a Medical Scribe Program on the Educational Trajectory of Prehealth and Underrepresented in Medicine Students
Background and Objectives: Sparse research exists on evaluating the effects of medical scribing programs on the educational trajectory of prehealth students. This study assesses the impact of the Stanford Medical Scribe Fellowship (COMET) on its prehealth participants’ educational goals, preparation for graduate training, and acceptance into health professional schools. Methods: We distributed a 31-question survey with both closed- and open-ended questions to 96 alumni. The survey collected participant demographics, self-reported underrepresented in medicine (URM) status, pre-COMET clinical experiences and educational goals, application to and acceptance at health professional schools, and perceived impact of COMET on their educational trajectory. SPSS was used to complete the analyses. Results: The survey had a 97% (93/96) completion rate. Among all respondents, 69% (64/93) applied to a health professional school and 70% (45/64) were accepted. Among URM respondents, 68% (23/34) applied to a health professional school and 70% (16/23) were accepted. Overall acceptance rates for MD/DO and PA/NP programs were 51% (24/47) and 61% (11/18), respectively. URM acceptance rates for MD/DO and PA/NP programs were 43% (3/7) and 58% (7/12), respectively. For current or recently graduated health professional school respondents, 97% (37/38) “strongly agreed” or “agreed” that COMET helped them succeed in their training. Conclusions: COMET is associated with a positive impact on the educational trajectory of its prehealth participants and a higher acceptance rate into health professional schools than the national rates for both overall and URM applicants. Scribing programs may serve as pipeline development and help increase the diversity of the future health care workforce.
- Open Access
- 8 May 2023
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 9 (1) , e43190
Feasibility and Acceptability of a US National Telemedicine Curriculum for Medical Students and Residents: Multi-institutional Cross-sectional Study
Background: Telemedicine use increased as a response to health care delivery changes necessitated by the COVID-19 pandemic. However, lack of standardized curricular content creates gaps and inconsistencies in effectively integrating telemedicine training at both the undergraduate medical education and graduate medical education levels. Objective: This study evaluated the feasibility and acceptability of a web-based national telemedicine curriculum developed by the Society of Teachers of Family Medicine for medical students and family medicine (FM) residents. Based on the Association of American Medical Colleges telehealth competencies, the asynchronous curriculum featured 5 self-paced modules; covered topics include evidence-based telehealth uses, best practices in communication and remote physical examinations, technology requirements and documentation, access and equity in telehealth delivery, and the promise and potential perils of emerging technologies. Methods: A total of 17 medical schools and 17 FM residency programs implemented the curriculum between September 1 and December 31, 2021. Participating sites represented 25 states in all 4 US census regions with balanced urban, suburban, and rural settings. A total of 1203 learners, including 844 (70%) medical students and 359 (30%) FM residents, participated. Outcomes were measured through self-reported 5-point Likert scale responses. Results: A total of 92% (1101/1203) of learners completed the entire curriculum. Across the modules, 78% (SD 3%) of participants agreed or strongly agreed that they gained new knowledge, skills, or attitudes that will help them in their training or career; 87% (SD 4%) reported that the information presented was at the right level for them; 80% (SD 2%) reported that the structure of the modules was effective; and 78% (SD 3%) agreed or strongly agreed that they were satisfied. Overall experience using the national telemedicine curriculum did not differ significantly between medical students and FM residents on binary analysis. No consistent statistically significant relationships were found between participants’ responses and their institution’s geographic region, setting, or previous experience with a telemedicine curriculum. Conclusions: Both undergraduate medical education and graduate medical education learners, represented by diverse geographic regions and institutions, indicated that the curriculum was broadly acceptable and effective.
- 8 May 2023
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 9 (1) , e43190
Open AccessFeasibility and Acceptability of a US National Telemedicine Curriculum for Medical Students and Residents: Multi-institutional Cross-sectional Study
Background: Telemedicine use increased as a response to health care delivery changes necessitated by the COVID-19 pandemic. However, lack of standardized curricular content creates gaps and inconsistencies in effectively integrating telemedicine training at both the undergraduate medical education and graduate medical education levels. Objective: This study evaluated the feasibility and acceptability of a web-based national telemedicine curriculum developed by the Society of Teachers of Family Medicine for medical students and family medicine (FM) residents. Based on the Association of American Medical Colleges telehealth competencies, the asynchronous curriculum featured 5 self-paced modules; covered topics include evidence-based telehealth uses, best practices in communication and remote physical examinations, technology requirements and documentation, access and equity in telehealth delivery, and the promise and potential perils of emerging technologies. Methods: A total of 17 medical schools and 17 FM residency programs implemented the curriculum between September 1 and December 31, 2021. Participating sites represented 25 states in all 4 US census regions with balanced urban, suburban, and rural settings. A total of 1203 learners, including 844 (70%) medical students and 359 (30%) FM residents, participated. Outcomes were measured through self-reported 5-point Likert scale responses. Results: A total of 92% (1101/1203) of learners completed the entire curriculum. Across the modules, 78% (SD 3%) of participants agreed or strongly agreed that they gained new knowledge, skills, or attitudes that will help them in their training or career; 87% (SD 4%) reported that the information presented was at the right level for them; 80% (SD 2%) reported that the structure of the modules was effective; and 78% (SD 3%) agreed or strongly agreed that they were satisfied. Overall experience using the national telemedicine curriculum did not differ significantly between medical students and FM residents on binary analysis. No consistent statistically significant relationships were found between participants’ responses and their institution’s geographic region, setting, or previous experience with a telemedicine curriculum. Conclusions: Both undergraduate medical education and graduate medical education learners, represented by diverse geographic regions and institutions, indicated that the curriculum was broadly acceptable and effective.
- Open Access
- 30 March 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Building Pandemic-Resilient Primary Care Systems: Lessons Learned From COVID-19 (Preprint)
UNSTRUCTURED On January 30, 2023, the Biden Administration announced its intention to end the existing COVID-19 public health emergency declaration. The transition to a “postpandemic” landscape presents a unique opportunity to sustain and strengthen pandemic-era changes in care delivery. With this in mind, we present 3 critical lessons learned from a primary care perspective during the COVID-19 pandemic. First, clinical workflows must support both in-person and internet-based care delivery. Second, the integration of asynchronous care delivery is critical. Third, planning for the future means planning for everyone, including those with potentially limited access to health care due to barriers in technology and communication. While these lessons are neither unique to primary care settings nor all-encompassing, they establish a grounded foundation on which to construct higher-quality, more resilient, and more equitable health systems.
- 30 March 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessBuilding Pandemic-Resilient Primary Care Systems: Lessons Learned From COVID-19 (Preprint)
UNSTRUCTURED On January 30, 2023, the Biden Administration announced its intention to end the existing COVID-19 public health emergency declaration. The transition to a “postpandemic” landscape presents a unique opportunity to sustain and strengthen pandemic-era changes in care delivery. With this in mind, we present 3 critical lessons learned from a primary care perspective during the COVID-19 pandemic. First, clinical workflows must support both in-person and internet-based care delivery. Second, the integration of asynchronous care delivery is critical. Third, planning for the future means planning for everyone, including those with potentially limited access to health care due to barriers in technology and communication. While these lessons are neither unique to primary care settings nor all-encompassing, they establish a grounded foundation on which to construct higher-quality, more resilient, and more equitable health systems.