Charles P. Friedman
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Charles P. Friedman has a total of 33 co-authors. Here are the top co-authors by number of shared publications:
A Elstein
2 shared publicationsF Wolf
2 shared publicationsG Murphy
2 shared publicationsJudy G. Ozbolt
Affiliations of the authors: University of Pittsburgh, Pittsburgh, PA (CPF, MCS); Stanford University, Stanford, CA (RBA); Harvard Medical School and Boston Children's Hospital, Boston, MA (ISK); SRA International, Inc., Rockville, MD (KAM); Yale University, New Haven, CT (PLM, DT); Vanderbilt University, Nashville, TN (JGO); Columbia University, New York, NY (EHS); Washington University, St. Louis, MO (GDS); American Medical Informatics Association, Bethesda, MD (JW)
2 shared publicationsT Franz
2 shared publicationsA J Krumrey
1 shared publication
Latest Publications
- Open Access
- 1 May 2004
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 11 (3) , 167-172
Training the Next Generation of Informaticians: The Impact of "BISTI" and Bioinformatics--A Report from the American College of Medical Informatics
In 2002–2003, the American College of Medical Informatics (ACMI) undertook a study of the future of informatics training. This project capitalized on
- 1 May 2004
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 11 (3) , 167-172
Open AccessTraining the Next Generation of Informaticians: The Impact of "BISTI" and Bioinformatics--A Report from the American College of Medical Informatics
In 2002–2003, the American College of Medical Informatics (ACMI) undertook a study of the future of informatics training. This project capitalized on
- 1 March 2003
- Published by Elsevier in International Journal of Medical Informatics
- Vol. 69 (2-3) , 261-272
Is medical informatics a mature science? A review of measurement practice in outcome studies of clinical systems
- 1 March 2003
- Published by Elsevier in International Journal of Medical Informatics
- Vol. 69 (2-3) , 261-272
Is medical informatics a mature science? A review of measurement practice in outcome studies of clinical systems
- Open Access
- 1 November 2001
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 8 (6) , 519-526
Toward a New Culture for Biomedical Informatics: Report of the 2001 ACMI Symposium
For those of us who have dedicated our careers to medical informatics, it is easy to feel over-stimulated in the current times. Moore's Law has seemingly been generalized, beyond the hardware we use, to embrace every aspect of our professional lives. Opportunities to apply our science seem to double annually, in ways unforeseeable as recently as five years ago. A field that once was clearly focused on systems to support the care of hospitalized and clinic patients has extended its reach to health information resources for consumers, systems to enhance and protect public health, and systems that support research in genomics and proteomics. We have spawned subfields denoted by prefixes or qualifying phrases, such as “public health informatics,” to mark this trend. Moreover, a field that was solidly rooted in academic medical centers now finds professional representation in for-profit corporations both large and small, in government agencies, and in foundations and professional societies. The field has acquired a distinct entrepreneurial spirit, not at all unwelcome but somehow new and unfamiliar.
- 1 November 2001
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 8 (6) , 519-526
Open AccessToward a New Culture for Biomedical Informatics: Report of the 2001 ACMI Symposium
For those of us who have dedicated our careers to medical informatics, it is easy to feel over-stimulated in the current times. Moore's Law has seemingly been generalized, beyond the hardware we use, to embrace every aspect of our professional lives. Opportunities to apply our science seem to double annually, in ways unforeseeable as recently as five years ago. A field that once was clearly focused on systems to support the care of hospitalized and clinic patients has extended its reach to health information resources for consumers, systems to enhance and protect public health, and systems that support research in genomics and proteomics. We have spawned subfields denoted by prefixes or qualifying phrases, such as “public health informatics,” to mark this trend. Moreover, a field that was solidly rooted in academic medical centers now finds professional representation in for-profit corporations both large and small, in government agencies, and in foundations and professional societies. The field has acquired a distinct entrepreneurial spirit, not at all unwelcome but somehow new and unfamiliar.
- 1 January 2001
- Vol. 84, 454-8
Are clinicians correct when they believe they are correct? Implications for medical decision support.
The process of clinical decision support is linked to the validity of clinicians' confidence in their judgments. Clinicians who are appropriately confident-highly confident when they are correct and less confident when they are incorrect-will access computer-based and other information resources only when they are needed. Clinicians who are consistently underconfident will rely on external resources when they are not needed. Those who are overconfident, who believe they are correct when in fact they are not, will be prone to medical errors. An extensive literature indicates a general tendency toward overconfidence in human judgment. This study explores the relationship between confidence and "correctness", across three levels of clinical experience, in the task domain of diagnosis in internal medicine. We created detailed synopses of 36 diagnostically challenging cases and divided them into four equivalent sets of nine cases each. We asked 216 subjects at three experience levels (72 senior medical students, 72 senior medical residents, and 72 faculty attendings) to generate a differential diagnosis for each of the nine cases in one randomly-assigned set, and simultaneously to indicate their level of confidence in each of their diagnoses. We then examined the relationship between the correctness of these diagnoses (the appearance of the correct diagnosis anywhere in the hypothesis list) and these confidence judgments, for all subjects and separately for subjects at each experience level. Results indicate a small but statistically significant relationship associating correctness with higher confidence for all subjects (Kendall's tau b =.-106;p <.0001). This statistical relationship is strongest for the students ( tau b =.-121;p <.001), somewhat lesser but still significant for the faculty-level attendings ( tau b =.-103;p <.005), and non-significant ( tau b =.-041 ) for the residents. (The negative correlations are a coding artifact.) Subjects in this study showed a tendency toward underconfidence: they had low confidence in correct diagnoses more often than they had high confidence when wrong. Nonetheless, they were overconfident and thus "error prone" for 17% of cases overall. The medical students were possibly overmatched by the difficulty of the cases, so their concordance between confidence and correctness may have resulted from an awareness that they were often guessing. The relatively low concordance seen in the residents and attendings makes a strong argument that decision support systems to reduce medical errors should include both "push" and "pull" models. In sum, these results indicate that medical decision support systems cannot rely exclusively on clinicians' perceptions of their information needs, as such perceptions will frequently be incorrect.
- 1 January 2001
- Vol. 84, 454-8
Are clinicians correct when they believe they are correct? Implications for medical decision support.
The process of clinical decision support is linked to the validity of clinicians' confidence in their judgments. Clinicians who are appropriately confident-highly confident when they are correct and less confident when they are incorrect-will access computer-based and other information resources only when they are needed. Clinicians who are consistently underconfident will rely on external resources when they are not needed. Those who are overconfident, who believe they are correct when in fact they are not, will be prone to medical errors. An extensive literature indicates a general tendency toward overconfidence in human judgment. This study explores the relationship between confidence and "correctness", across three levels of clinical experience, in the task domain of diagnosis in internal medicine. We created detailed synopses of 36 diagnostically challenging cases and divided them into four equivalent sets of nine cases each. We asked 216 subjects at three experience levels (72 senior medical students, 72 senior medical residents, and 72 faculty attendings) to generate a differential diagnosis for each of the nine cases in one randomly-assigned set, and simultaneously to indicate their level of confidence in each of their diagnoses. We then examined the relationship between the correctness of these diagnoses (the appearance of the correct diagnosis anywhere in the hypothesis list) and these confidence judgments, for all subjects and separately for subjects at each experience level. Results indicate a small but statistically significant relationship associating correctness with higher confidence for all subjects (Kendall's tau b =.-106;p <.0001). This statistical relationship is strongest for the students ( tau b =.-121;p <.001), somewhat lesser but still significant for the faculty-level attendings ( tau b =.-103;p <.005), and non-significant ( tau b =.-041 ) for the residents. (The negative correlations are a coding artifact.) Subjects in this study showed a tendency toward underconfidence: they had low confidence in correct diagnoses more often than they had high confidence when wrong. Nonetheless, they were overconfident and thus "error prone" for 17% of cases overall. The medical students were possibly overmatched by the difficulty of the cases, so their concordance between confidence and correctness may have resulted from an awareness that they were often guessing. The relatively low concordance seen in the residents and attendings makes a strong argument that decision support systems to reduce medical errors should include both "push" and "pull" models. In sum, these results indicate that medical decision support systems cannot rely exclusively on clinicians' perceptions of their information needs, as such perceptions will frequently be incorrect.
- 1 October 2000
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 75 (Supplement) , S137-S142
The Marvelous Medical Education Machine or How Medical Education Can Be Unstuck in Time
- 1 October 2000
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 75 (Supplement) , S137-S142
The Marvelous Medical Education Machine or How Medical Education Can Be Unstuck in Time
- 17 November 1999
- Published by American Medical Association (AMA) in JAMA
- Vol. 282 (19) , 1851-1856
Enhancement of Clinicians' Diagnostic Reasoning by Computer-Based Consultation
Research from JAMA — Enhancement of Clinicians' Diagnostic Reasoning by Computer-Based Consultation — A Multisite Study of 2 Systems — ContextComputer-based diagnostic decision support systems (DSSs) were developed to improve health care quality by providing accurate, useful, and timely diagnostic information to clinicians. However, most studies have emphasized the accuracy of the computer system alone, without placing clinicians in the role of direct users.ObjectiveTo explore the extent to which consultations with DSSs improve clinicians' diagnostic hypotheses in a set of diagnostically challenging cases.DesignPartially randomized controlled trial conducted in a laboratory setting, using a prospective balanced experimental design in 1995-1998.SettingThree academic medical centers, none of which were involved in the development of the DSSs.ParticipantsA total of 216 physicians: 72 at each site, including 24 internal medicine faculty members, 24 senior residents, and 24 fourth-year medical students. One physician's data were lost to analysis.InterventionTwo DSSs, ILIAD (version 4.2) and Quick Medical Reference (QMR; version 3.7.1), were used by participants for diagnostic evaluation of a total of 36 cases based on actual patients. After training, each subject evaluated 9 of the 36 cases, first without and then using a DSS, and suggested an ordered list of diagnostic hypotheses after each evaluation.Main Outcome MeasureDiagnostic accuracy, measured as the presence of the correct diagnosis on the hypothesis list and also using a derived diagnostic quality score, before and after consultation with the DSSs.ResultsCorrect diagnoses appeared in subjects' hypothesis lists for 39.5% of cases prior to consultation and 45.4% of cases after consultation. Subjects' mean diagnostic quality scores increased from 5.7 (95% confidence interval [CI], 5.5-5.9) to 6.1 (95% CI, 5.9-6.3) (effect size: Cohen d = 0.32; 95% CI, 0.23-0.41; P<.001). Larger increases (P = .048) were observed for students than for residents and faculty. Effect size varied significantly (P<.02) by DSS (Cohen d = 0.20; 95% CI, 0.08-0.32 for ILIAD vs Cohen d = 0.45; 95% CI, 0.31-0.59 for QMR).ConclusionsOur study supports the idea that "hands-on" use of diagnostic DSSs can influence diagnostic reasoning of clinicians. The larger effect for students suggests a possible educational role for these systems.
- 17 November 1999
- Published by American Medical Association (AMA) in JAMA
- Vol. 282 (19) , 1851-1856
Enhancement of Clinicians' Diagnostic Reasoning by Computer-Based Consultation
Research from JAMA — Enhancement of Clinicians' Diagnostic Reasoning by Computer-Based Consultation — A Multisite Study of 2 Systems — ContextComputer-based diagnostic decision support systems (DSSs) were developed to improve health care quality by providing accurate, useful, and timely diagnostic information to clinicians. However, most studies have emphasized the accuracy of the computer system alone, without placing clinicians in the role of direct users.ObjectiveTo explore the extent to which consultations with DSSs improve clinicians' diagnostic hypotheses in a set of diagnostically challenging cases.DesignPartially randomized controlled trial conducted in a laboratory setting, using a prospective balanced experimental design in 1995-1998.SettingThree academic medical centers, none of which were involved in the development of the DSSs.ParticipantsA total of 216 physicians: 72 at each site, including 24 internal medicine faculty members, 24 senior residents, and 24 fourth-year medical students. One physician's data were lost to analysis.InterventionTwo DSSs, ILIAD (version 4.2) and Quick Medical Reference (QMR; version 3.7.1), were used by participants for diagnostic evaluation of a total of 36 cases based on actual patients. After training, each subject evaluated 9 of the 36 cases, first without and then using a DSS, and suggested an ordered list of diagnostic hypotheses after each evaluation.Main Outcome MeasureDiagnostic accuracy, measured as the presence of the correct diagnosis on the hypothesis list and also using a derived diagnostic quality score, before and after consultation with the DSSs.ResultsCorrect diagnoses appeared in subjects' hypothesis lists for 39.5% of cases prior to consultation and 45.4% of cases after consultation. Subjects' mean diagnostic quality scores increased from 5.7 (95% confidence interval [CI], 5.5-5.9) to 6.1 (95% CI, 5.9-6.3) (effect size: Cohen d = 0.32; 95% CI, 0.23-0.41; P<.001). Larger increases (P = .048) were observed for students than for residents and faculty. Effect size varied significantly (P<.02) by DSS (Cohen d = 0.20; 95% CI, 0.08-0.32 for ILIAD vs Cohen d = 0.45; 95% CI, 0.31-0.59 for QMR).ConclusionsOur study supports the idea that "hands-on" use of diagnostic DSSs can influence diagnostic reasoning of clinicians. The larger effect for students suggests a possible educational role for these systems.
- 1 July 1999
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 74 (7) , 795-9
Information technology leadership in academic medical centers
Persons and groups within academic medical centers bring consistent and predictable viewpoints to planning and decision making . The varied professional and academic cultures of these individuals appear to account primarily for the diversity of their viewpoints . Understanding these professional cultures can help leaders achieve some predictability in the complex environments for which they are responsible . Leaders in information technology in particular , in order to be successful , must become part-time anthropologists , immersing themselves in the varied workplaces of their constituents to understand the work they do and the cultures that have grown up around this work . Only in this way will they be able to manage the challenges that arise continuously as the technology and the needs it can address change over time . In this article , the author briefly describes the concept of culture , portrays four specific professional cultures that typically coexist in academic medical centers , and argues that understanding these cultures is absolutely critical to effective management and use of information resources .
- 1 July 1999
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 74 (7) , 795-9
Information technology leadership in academic medical centers
Persons and groups within academic medical centers bring consistent and predictable viewpoints to planning and decision making . The varied professional and academic cultures of these individuals appear to account primarily for the diversity of their viewpoints . Understanding these professional cultures can help leaders achieve some predictability in the complex environments for which they are responsible . Leaders in information technology in particular , in order to be successful , must become part-time anthropologists , immersing themselves in the varied workplaces of their constituents to understand the work they do and the cultures that have grown up around this work . Only in this way will they be able to manage the challenges that arise continuously as the technology and the needs it can address change over time . In this article , the author briefly describes the concept of culture , portrays four specific professional cultures that typically coexist in academic medical centers , and argues that understanding these cultures is absolutely critical to effective management and use of information resources .
- 1 September 1998
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 73 (9) , 975-9
Managing information technology in academic medical centers
Based on a session at the 1997 conference on Information Resources and Academic Medicine sponsored by the Association of American Medical Colleges, this article illustrates how the beliefs and concerns of academic medicine's diverse professional cultures affect the management of information technology. Two scenarios--one dealing with the standardization of desktop PCs, the other with publication of syllabi on an institutional intranet--form the basis of this exercise. Four prototypical members of a hypothetical medical center community--the chairman of surgery, a senior basic scientist, the chief information officer of an affiliated hospital, and the chief administrative officer--offer their perspectives on each scenario. Their statements illustrate many of the challenges of planning, deploying, and maintaining effective information technology in the “multicultural” environment of academic medical centers.
- 1 September 1998
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 73 (9) , 975-9
Managing information technology in academic medical centers
Based on a session at the 1997 conference on Information Resources and Academic Medicine sponsored by the Association of American Medical Colleges, this article illustrates how the beliefs and concerns of academic medicine's diverse professional cultures affect the management of information technology. Two scenarios--one dealing with the standardization of desktop PCs, the other with publication of syllabi on an institutional intranet--form the basis of this exercise. Four prototypical members of a hypothetical medical center community--the chairman of surgery, a senior basic scientist, the chief information officer of an affiliated hospital, and the chief administrative officer--offer their perspectives on each scenario. Their statements illustrate many of the challenges of planning, deploying, and maintaining effective information technology in the “multicultural” environment of academic medical centers.
- 1 January 1998
- p. 864-8
Measuring the quality of diagnostic hypothesis sets for studies of decision support.
Within medical informatics there is widespread interest in computer-based decision support and the evaluation of its impact. It is widely recognized that the measurement of dependent variables, or outcomes, represents the most challenging aspect of this work. This paper describes and reports the reliability and validity of an outcome metric for studies of diagnostic decision support. The results of this study will guide the analytic methods used in our ongoing multi-site study of the effects of decision support on diagnostic reasoning. Our measurement approach conceptualizes the quality of a diagnostic hypothesis set as having two components summed to generate a composite index: a Plausibility Component derived from ratings of each hypothesis in the set, whether correct or incorrect; and a Location Component derived from the location of the correct diagnosis if it appears in the set. The reliability of this metric is determined by the extent of interrater agreement on the plausibility of diagnostic hypotheses. Validity is determined by the extent to which the index generates scores that make sense on inspection (face validity), as well as the extent to which the component scores are non-redundant and discriminate the performance of novices and experts (construct validity). Using data from the pilot and main phases of our ongoing study (n = 124 subjects working 1116 cases), the reliability of our diagnostic quality metric was found to be 0.85-0.88. The metric was found to generate, on inspection, no clearly counterintuitive scores. Using data from the pilot phase of our study (n = 12 subjects working 108 cases), the component scores were moderately correlated (r = 0.68). The composite index, computed by equally weighting both components, was found to discriminate the hypotheses of medical students and attending physicians by 0.97 standard deviation units. Based on these findings, we have adopted this metric for use in our further research exploring the impact of decision support systems on diagnostic reasoning and will make it available to the informatics research community.
- 1 January 1998
- p. 864-8
Measuring the quality of diagnostic hypothesis sets for studies of decision support.
Within medical informatics there is widespread interest in computer-based decision support and the evaluation of its impact. It is widely recognized that the measurement of dependent variables, or outcomes, represents the most challenging aspect of this work. This paper describes and reports the reliability and validity of an outcome metric for studies of diagnostic decision support. The results of this study will guide the analytic methods used in our ongoing multi-site study of the effects of decision support on diagnostic reasoning. Our measurement approach conceptualizes the quality of a diagnostic hypothesis set as having two components summed to generate a composite index: a Plausibility Component derived from ratings of each hypothesis in the set, whether correct or incorrect; and a Location Component derived from the location of the correct diagnosis if it appears in the set. The reliability of this metric is determined by the extent of interrater agreement on the plausibility of diagnostic hypotheses. Validity is determined by the extent to which the index generates scores that make sense on inspection (face validity), as well as the extent to which the component scores are non-redundant and discriminate the performance of novices and experts (construct validity). Using data from the pilot and main phases of our ongoing study (n = 124 subjects working 1116 cases), the reliability of our diagnostic quality metric was found to be 0.85-0.88. The metric was found to generate, on inspection, no clearly counterintuitive scores. Using data from the pilot phase of our study (n = 12 subjects working 108 cases), the component scores were moderately correlated (r = 0.68). The composite index, computed by equally weighting both components, was found to discriminate the hypotheses of medical students and attending physicians by 0.97 standard deviation units. Based on these findings, we have adopted this metric for use in our further research exploring the impact of decision support systems on diagnostic reasoning and will make it available to the informatics research community.
- 1 June 1997
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 72 (6) , 511
Perspectives on computing and medical education. Introduction
- 1 June 1997
- Published by Oxford University Press (OUP) in Academic Medicine
- Vol. 72 (6) , 511
Perspectives on computing and medical education. Introduction