Clinical measurements obtained from point-of-care ultrasound images to assess acquisition skills
Open Access
- 22 February 2019
- journal article
- research article
- Published by Mattioli1885 in The Ultrasound Journal
- Vol. 11 (1) , 4
Abstract
Current methods of assessing competence in acquiring point-of-care ultrasound images are inadequate. They rely upon cumbersome rating systems that do not depend on the actual outcome measured and lack evidence of validity. We describe a new method that uses a rigorous statistical model to assess performance of individual trainees based on the actual task, image acquisition. Measurements obtained from the images acquired (the actual desired outcome) are themselves used to validate effective training and competence acquiring ultrasound images. We enrolled a convenience sample of 21 spontaneously breathing adults from a general medicine ward. In random order, two trainees (A and B) and an instructor contemporaneously acquired point-of-care ultrasound images of the inferior vena cava and the right internal jugular vein from the same patients. Blinded diameter measurements from each ultrasound were analyzed quantitatively using a multilevel model. Consistent mean differences between each trainee’s and the instructor’s images were ascribed to systematic acquisition errors, indicative of poor measurement technique and a need for further training. Wider variances were attributed to sporadic errors, indicative of inconsistent application of measurement technique across patients. In addition, the instructor recorded qualitative observations of each trainee’s performance during image acquisition. For all four diameters, the means and variances of measurements from trainee A’s images differed significantly from the instructor’s, whereas those from trainee B’s images were comparable. Techniques directly observed by the instructor supported these model-derived findings. For example, mean anteroposterior diameters of the internal jugular vein obtained from trainee A’s images were 3.8 mm (90% CI 2.3–5.4) smaller than from the instructor’s; this model-derived finding matched the instructor’s observation that trainee A compressed the vein during acquisition. Instructor summative assessments agreed with model-derived findings, providing internal validation of the descriptive and quantitative assessments of competence acquiring ultrasound images. Clinical measurements obtained from point-of-care ultrasound images acquired contemporaneously by trainees and an instructor can be used to quantitatively assess the image acquisition competence of specific trainees. This method may obviate resource-intensive qualitative rating systems that are based on ultrasound image quality and direct observation, while also helping instructors guide remediation.Keywords
Funding Information
- Center for Clinical and Translational Sciences, University of Texas Health Science Center at Houston (UL1TR001086)
This publication has 33 references indexed in Scilit:
- Estimation of the diameter and cross-sectional area of the internal jugular veins in adult patientsCritical Care, 2009
- Diagnostic accuracy of hospitalist‐performed hand‐carried ultrasound echocardiography after a brief training programJournal of Hospital Medicine, 2009
- Diagnostic accuracy of a simple ultrasound measurement to estimate central venous pressure in spontaneously breathing, critically ill patientsJournal of Hospital Medicine, 2009
- Reliability, repeatability and reproducibility: analysis of measurement errors in continuous variablesUltrasound in Obstetrics & Gynecology, 2008
- A comparison of the hierarchical likelihood and Bayesian approaches to spatial epidemiological modellingEnvironmetrics, 2007
- An Overview on Assessing Agreement with Continuous MeasurementsJournal of Biopharmaceutical Statistics, 2007
- Estimating Reliability and Generalizability from Hierarchical Biomedical DataJournal of Biopharmaceutical Statistics, 2007
- A comparison of Bayesian and likelihood-based methods for fitting multilevel modelsBayesian Analysis, 2006
- Forming inferences about some intraclass correlation coefficients.Psychological Methods, 1996
- Validating Interpretive Arguments for Licensure and Certification ExaminationsEvaluation & the Health Professions, 1994