Eta S. Berner

Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: The Health Information Technology for Economic & Clinical Health (HITECH) Act of 2009 was legislated to reduce healthcare costs, improve quality, and increase patient safety. Providers and organizations were incentivized for exhibiting meaningful use of certified Electronic Health Record (EHR) systems in order to achieve this objective. EHR adoption is an expensive investment, given the resources and capital that are invested. Due to the cost of the investment, a return on the EHR adoption investment (ROI) is expected. Objective: This study performed a value analysis of EHRS. The objective of this study was to investigate the relationship between EHR adoption levels and financial and clinical outcomes by combining both financial and clinical outcomes into one conceptual model. Methods: Multivariate relationships between different levels of EHR adoption and financial and clinical outcomes along with the time variant control variables were examined using moderation analysis with a longitudinal fixed effects model. Since it is unknown as to when hospitals begin experiencing improvements in financial outcomes, additional analysis was conducted using a 1- or 2-year lag for profit margin ratios. Results: 5,768 hospital year observations were analyzed over the course of four years. According to the results of the moderation analysis, as the readmission rate increases by one unit, the effect of a one-unit increase in EHR adoption level on the operating margin decreases by 5.38%. Hospitals with higher readmission payment adjustment factors have lower penalties. Conclusions: This study fills the gap in the literature by evaluating individual relationships between EHR adoption levels and financial and clinical outcomes, in addition to evaluating the relationship between EHR adoption level and financial outcomes, with clinical outcomes as moderators. This study provided statistically significant evidence, indicating that there is a relationship between EHR adoption level and operating margins when this relationship is moderated by readmission rates. This finding could further be supported by evaluating more recent data to analyze whether hospitals increasing their level of EHR adoption level would decrease readmission rates, resulting in an increase in operating margins. Hospitals would incur lower penalties as a result of improved readmission rates, which would contribute toward improved operating margins.

Objective: Developing targeted, culturally competent educational materials is critical for participant understanding of engagement in a large genomic study that uses computational pipelines to produce genome-informed risk assessments. Materials and Methods: Guided by the Smerecnik framework that theorizes understanding of multifactorial genetic disease through 3 knowledge types, we developed English and Spanish infographics for individuals enrolled in the Electronic Medical Records and Genomics Network. Infographics were developed to explain concepts in lay language and visualizations. We conducted iterative sessions using a modified “think-aloud” process with 10 participants (6 English, 4 Spanish-speaking) to explore comprehension of and attitudes towards the infographics. Results: We found that all but one participant had “awareness knowledge” of genetic disease risk factors upon viewing the infographics. Many participants had difficulty with “how-to” knowledge of applying genetic risk factors to specific monogenic and polygenic risks. Participant attitudes towards the iteratively-refined infographics indicated that design saturation was reached. Discussion: There were several elements that contributed to the participants’ comprehension (or misunderstanding) of the infographics. Visualization and iconography techniques best resonated with those who could draw on prior experiences or knowledge and were absent in those without. Limited graphicacy interfered with the understanding of absolute and relative risks when presented in graph format. Notably, narrative and storytelling theory that informed the creation of a vignette infographic was most accessible to all participants. Conclusion: Engagement with the intended audience who can identify strengths and points for improvement of the intervention is necessary to the development of effective infographics.

Apply natural language processing (NLP) to Amazon consumer reviews to identify adverse events (AEs) associated with unapproved over the counter (OTC) homeopathic drugs and compare findings with reports to the US Food and Drug Administration Adverse Event Reporting System (FAERS). Data were extracted from publicly available Amazon reviews and analyzed using JMP 16 Pro Text Explorer. Topic modeling identified themes. Sentiment analysis (SA) explored consumer perceptions. A machine learning model optimized prediction of AEs in reviews. Reports for the same time interval and product class were obtained from the FAERS public dashboard and analyzed. Homeopathic cough/cold products were the largest category common to both data sources (Amazon = 616, FAERS = 445) and were analyzed further. Oral symptoms and unpleasant taste were described in both datasets. Amazon reviews describing an AE had lower Amazon ratings (X² = 224.28, P < .0001). The optimal model for predicting AEs was Neural Boosted 5-fold combining topic modeling and Amazon ratings as predictors (mean AUC = 0.927). Topic modeling and SA of Amazon reviews provided information about consumers’ perceptions and opinions of homeopathic OTC cough and cold products. Amazon ratings appear to be a good indicator of the presence or absence of AEs, and identified events were similar to FAERS. Amazon reviews may complement traditional data sources to identify AEs associated with unapproved OTC homeopathic products. This study is the first to use NLP in this context and lays the groundwork for future larger scale efforts.

BACKGROUND The Health Information Technology for Economic and Clinical Health Act of 2009 was legislated to reduce health care costs, improve quality, and increase patient safety. Providers and organizations were incentivized to exhibit meaningful use of certified electronic health record (EHR) systems in order to achieve this objective. EHR adoption is an expensive investment, given the resources and capital that are invested. Due to the cost of the investment, a return on the EHR adoption investment is expected. OBJECTIVE This study performed a value analysis of EHRs. The objective of this study was to investigate the relationship between EHR adoption levels and financial and clinical outcomes by combining both financial and clinical outcomes into one conceptual model. METHODS We examined the multivariate relationships between different levels of EHR adoption and financial and clinical outcomes, along with the time variant control variables, using moderation analysis with a longitudinal fixed effects model. Since it is unknown as to when hospitals begin experiencing improvements in financial outcomes, additional analysis was conducted using a 1- or 2-year lag for profit margin ratios. RESULTS A total of 5768 hospital-year observations were analyzed over the course of 4 years. According to the results of the moderation analysis, as the readmission rate increases by 1 unit, the effect of a 1-unit increase in EHR adoption level on the operating margin decreases by 5.38%. Hospitals with higher readmission payment adjustment factors have lower penalties. CONCLUSIONS This study fills the gap in the literature by evaluating individual relationships between EHR adoption levels and financial and clinical outcomes, in addition to evaluating the relationship between EHR adoption level and financial outcomes, with clinical outcomes as moderators. This study provided statistically significant evidence (P<.05), indicating that there is a relationship between EHR adoption level and operating margins when this relationship is moderated by readmission rates, meaning hospitals that have adopted EHRs could see a reduction in their readmission rates and an increase in operating margins. This finding could further be supported by evaluating more recent data to analyze whether hospitals increasing their level of EHR adoption would decrease readmission rates, resulting in an increase in operating margins. Hospitals would incur lower penalties as a result of improved readmission rates, which would contribute toward improved operating margins.

Objective To evaluate the ability of case vignettes to assess the performance of symptom checker applications and to suggest refinements of the methods of case vignette-based audit studies. Methods We re-analyzed the publicly available data of two prominent case vignette-based symptom checker audit studies by calculating common metrics of test theory. Furthermore, we developed a new metric, the Capability Comparison Score (CCS), which controls for different degrees of item difficulty of the set of cases each symptom checker evaluated. We then scrutinized whether applying test theory and the CCS alter the performance ranking of the investigated symptom checkers. Results In both studies most symptom checkers changed their rank order of triage capability when adjusting for item difficulty with the CCS. The previously reported triage accuracies commonly overestimated the capability of symptom checkers because they do not account for the fact that symptom checkers tend to selectively appraise easier cases. Many case vignettes in both studies showed insufficient (very low and even negative) values of item-total correlation. Conclusions A test theoretic perspective helps identify previously unreported limitations to the validity of case vignette-based symptom checker assessments and provides guidance on how to improve the quality of case vignettes and metrics for appraising their quality. A more elaborate metric, which accounts for item difficulty of vignettes an app did (not) evaluate, might prove more meaningful than accuracy alone for competitive assessment of symptom checkers. Our approach contributes to standardizing the methods to appraise symptom checker capability to yield more generalizable results.

Background: Clinician notes are structured in a variety of ways. This research pilot tested an innovative study design and explored the impact of note formats on diagnostic accuracy and documentation review time. Objective: To compare two formats for clinical documentation (narrative format vs. list of findings) on clinician diagnostic accuracy and documentation review time. Method: Participants diagnosed written clinical cases, half in narrative format, and half in list format. Diagnostic accuracy (defined as including correct case diagnosis among top three diagnoses) and time spent processing the case scenario were measured for each format. Generalised linear mixed regression models and bias-corrected bootstrap percentile confidence intervals for mean paired differences were used to analyse the primary research questions. Results: Odds of correctly diagnosing list format notes were 26% greater than with narrative notes. However, there is insufficient evidence that this difference is significant (75% CI 0.8–1.99). On average the list format notes required 85.6 more seconds to process and arrive at a diagnosis compared to narrative notes (95% CI -162.3, −2.77). Of cases where participants included the correct diagnosis, on average the list format notes required 94.17 more seconds compared to narrative notes (75% CI -195.9, −8.83). Conclusion: This study offers note format considerations for those interested in improving clinical documentation and suggests directions for future research. Balancing the priority of clinician preference with value of structured data may be necessary. Implications: This study provides a method and suggestive results for further investigation in usability of electronic documentation formats.

Objective:To identify the informatics educational needs of clinical and translational research professionals whose primary focus is not informatics. Introduction:Informatics and data science skills are essential for the full spectrum of translational research, and an increased understanding of informatics issues on the part of translational researchers can alleviate the demand for informaticians and enable more productive collaborations when informaticians are involved. Identifying the level of interest in different topics among various types of of translational researchers will help set priorities for development and dissemination of informatics education. Methods:We surveyed clinical and translational science researchers in Clinical and Translational Science Award (CTSA) programs about their educational needs and preferences. Results:Researchers from 23 out of the 62 CTSA hubs responded to the survey. 67% of respondents across roles and topics expressed interest in learning about informatics topics. There was high interest in all 30 topics included in the survey, with some variation in interest depending on the role of the respondents. Discussion:Our data support the need to advance training in clinical and biomedical informatics. As the complexity and use of information technology and data science in research studies grows, informaticians will continue to be a limited resource for research collaboration, education, and training. An increased understanding of informatics issues across translational research teams can alleviate this burden and allow for more productive collaborations. To inform a roadmap for informatics education for research professionals, we suggest strategies to use the results of this needs assessment to develop future informatics education.

Editors: Kate Fultz Hollis Oregon Health & Science University Portland, OR, USA fultzhol@ohsu.edu Fleur Mougin University of Bordeaux & BPH UMR_S 1219 Bordeaux, France Fleur.Mougin@u-bordeaux.fr Lina F. Soualmia Normandie University, UNIROUEN, LITIS Rouen, France Lina.Soualmia@chu-rouen.fr Editorial Assistant: Martina Hutter Heidelberg University Hospital Heidelberg, Germany yearbook@imia-services.org Best Papers Editor: Adrien Ugon ESIEE Paris & LIP6 UMR_7606 adrien.ugon@esiee.fr Advisory Board: Reinhold Haux Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School Braunschweig, Germany reinhold.haux@plri.de Victor Maojo Universidad Politécnica de Madrid Departamento de Inteligencia Artificial Facultad de Informatica Madrid, Spain vmaojo@fi.upm.es George Mihalas University of Medicine and Pharmacy Dept. of Medical Informatics Timisoara, Romania mihalas@EFMI.info Section Editors: Christian Baumgartner Graz University of Technology Graz, Austria christian.baumgartner@tugraz.at Mary Lauren Benton Baylor University Waco, TX, USA marylauren_benton@baylor.edu Eta S. Berner Department of Health Services Administration University of Alabama at Birmingham Birmingham, AL, USA eberner@uab.edu Meryl Bloomrosen mbloomrosen@google.com Damian Borbolla Department of Biomedical Informatics University of Utah Salt Lake City, UT, USA damian.borbolla@utah.edu Georgeta Bordea University of Bordeaux & BPH UMR_S 1219 Bordeaux, France georgeta.bordea@u-bordeaux.fr Jean Charlet DRCI/AP-HP & LIMICS UMR_S 1142 Paris, France jean.charlet@sorbonne-universite.fr Tiago Colicchio University of Alabama at Birmingham, AL, USA tiagokc@uab.edu Licong Cui The University of Texas Health Science Center at Houston Houston, TX, USA Licong.Cui@uth.tmc.edu Christel Daniel WIND-DSI/AP-HP & LIMICS UMR_S 1142 Paris, France christel.daniel@aphp.fr Thomas M. Deserno Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School Braunschweig, Germany thomas.deserno@plri.de Ferdinand Dhombres Sorbonne University & LIMICS UMR_S 1142 Trousseau Hospital, AP-HP Paris, France ferdinand.dhombres@aphp.fr Gayo Diallo University of Bordeaux & BPH UMR_S 1219 Bordeaux, France gayo.diallo@u-bordeaux.fr Brian E. Dixon Indiana University Fairbanks School of Public Health & Regenstrief Institute Indianapolis, IN, USA bedixon@regenstrief.org Natalia Grabar CNRS, Univ. Lille, UMR 8163 STL - Savoirs Textes Langage Lille, France natalia.grabar@univ-lille.fr Cyril Grouin Université Paris-Saclay, CNRS, LISN Orsay, France cyril.grouin@lisn.upsaclay.fr Werner Hackl UMIT TIROL - Private University for Health Sciences and Health Technology Hall in Tirol, Austria werner.hackl@umit-tirol.at Alexander Hoerbst MCI - The Entrepreneurial School Innsbruck, Austria alexander.hoerbst@mci.edu John H. Holmes University of Pennsylvania PerelmanSchool of Medicine Philadelphia, PA, USA jhholmes@pennmedicine.upenn.edu Dipak Kalra The University of Gent Gent, Belgium dipak.kalra@i-hd.eu Annie Y. S. Lau Centre for Health Informatics Australian Institute of Health Innovation Macquarie University, Australia annie.lau@mq.edu.au Scott McGrath University of California Berkeley Missoula, USA smcgrath@berkeley.edu Debra Patt Texas Oncology Austin, TX, USA debra.patt@usoncology.com Michael Rooney Department of Radiation Oncology MD Anderson Cancer Center Houston, TX, USA mkrooney@mdanderson.org Yalini Senathirajah University of Pittsburgh School of Medicine Pittsburgh, USA yalini@pitt.edu Anthony Solomonides NorthShore University HealthSystem Evanston, Illinois, USA tony.solomonides@gmail.com Pascal Staccini IRIS Department, Lab RETINES Faculté de Médecine, Université Côte d’Azur Nice, France pascal.staccini@univ-cotedazur.fr X. Tannier Sorbonne Université Université Sorbonne Paris Nord, INSERM, LIMICS Paris, France xavier.tannier@sorbonne-universite.fr Jeremy Warner Lifespan Cancer Institute Providence, RI, USA jeremy_warner@brown.edu Regional Editors: Naoki Nakashima APAMI nnaoki@med.kyushu-u.ac.jp Ghislain B. Kouematchoua Tchuitcheu HELINA gkouema@gmail.com Marcelo Silva IMIA-LAC marcelo.silva@sbis.org.br Najeeb Al-Shorbaji MENAHIA shorbajin@gmail.com James J. Cimino AMIA Birmingham, AL, USA ciminojj@gmail.com Andre Kushniruk Digital Health Canada: Canada’s Health Informatics Association Toronto, ON, Canada andrek@uvic.ca Lacramioara Stoicu-Tivadar EFMI lacramioara.stoicu-tivadar@upt.ro IMIA Executive Director: Elaine Huesing imia@imia-services.org Article published online: 04 December 2022 © 2022. IMIA and Thieme. This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/) Georg Thieme Verlag KG Rüdigerstraße 14, 70469 Stuttgart, Germany

Learning Health Systems (LHS) require a workforce with specific knowledge and skills to identify and address healthcare quality issues, develop solutions to address those issues, and sustain and spread improvements within and outside the organization. Educational programs are tasked with designing learning opportunities that can meet these organizational needs. This manuscript explores different mechanisms for addressing challenges to creating educational programs to prepare individuals who can work in and lead LHS. Strategies and recommendations for educational programs to support the LHS include the creation of a new program, collaborating across existing programs, and producing a set of instructional materials.

A discussion and debate on the American Medical Informatics Association’s (AMIA) Ethical, Legal, and Social Issues (ELSI) Working Group listserv in 2021 raised important issues related to a forthcoming conference in Texas. Texas had recently enacted a restrictive abortion law and restricted voting rights. Several AMIA members advocated for a boycott of the state and the scheduled conference. The discussion led the AMIA Board of Directors to request that the organization’s Ethics Committee provide general guidance for principle-based venue selection. This document recommends overarching principles for the venue selection for future AMIA events and conferences. Discussions by the AMIA Board, the Ethics Committee, and the ELSI Working Group informed these recommendations, and this document on guiding principles was approved by the AMIA Board of Directors in April 2022.