Tools for the direct observation and assessment of psychomotor skills in medical trainees: a systematic review
Authors:
- 9 June 2013
- journal article
- review article
- Published by Wiley in Medical Education
- Vol. 47 (7) , 650-673
Abstract
Context: The A ccreditation C ouncil for G raduate M edical E ducation ( ACGME ) Milestone Project mandates programmes to assess the attainment of training outcomes, including the psychomotor (surgical or procedural) skills of medical trainees. The objectives of this study were to determine which tools exist to directly assess psychomotor skills in medical trainees on live patients and to identify the data indicating their psychometric and edumetric properties. Methods: An electronic search was conducted for papers published from January 1948 to May 2011 using the PubMed, E ducation R esource I nformation C enter ( ERIC ), C umulative I ndex to N ursing and A llied H ealth L iterature ( CINAHL ) and Web of Science electronic databases and the review of references in article bibliographies. A study was included if it described a tool or instrument designed for the direct observation of psychomotor skills in patient care settings by supervisors. Studies were excluded if they referred to tools that assessed only clinical or non‐technical skills, involved non‐medical health professionals, or assessed skills performed on a simulator. Overall, 4114 citations were screened, 168 (4.1%) articles were reviewed for eligibility and 51 (1.2%) manuscripts were identified as meeting the study inclusion criteria. Three authors abstracted and reviewed studies using a standardised form for the presence of key psychometric and edumetric elements as per ACGME and American Psychological Association (APA) recommendations, and also assigned an overall grade based on the ACGME Committee on Educational Outcome Assessment grading system. Results: A total of 30 tools were identified. Construct validity based on associations between scores and training level was identified in 24 tools, internal consistency in 14, test–retest reliability in five and inter‐rater reliability in 20. The modification of attitudes, knowledge or skills was reported using five tools. The seven‐item Global Rating Scale and the Procedure‐Based Assessment received an overall Class 1 ACGME grade and are recommended based on Level A ACGME evidence. Conclusions: Numerous tools are available for the assessment of psychomotor skills in medical trainees, but evidence supporting their psychometric and edumetric properties is limited.Keywords
This publication has 67 references indexed in Scilit:
- In‐training assessment and predictors of competency in endoscopic sinus surgeryThe Laryngoscope, 2010
- Development and pilot testing of an operative competency assessment tool for pediatric direct laryngoscopy and rigid bronchoscopyThe Laryngoscope, 2010
- Global Assessment of Gastrointestinal Endoscopic Skills (GAGES): a valid measurement tool for technical skills in flexible endoscopySurgical Endoscopy, 2010
- Advancing Resident Assessment in Graduate Medical EducationJournal of Graduate Medical Education, 2009
- Refining the Evaluation of Operating Room PerformanceJournal of Surgical Education, 2009
- Continuous, Data-Rich Appraisal of Surgical Trainees' Operative Abilities: A Novel Approach for Measuring Performance and Providing FeedbackJournal of Surgical Education, 2009
- Undertaking Procedure-Based Assessment is Feasible in Clinical PracticeThe Annals of The Royal College of Surgeons of England, 2009
- Value and face validity of objective structured assessment of technical skills (OSATS) for work based assessment of surgical skills in obstetrics and gynaecologyMedical Teacher, 2008
- An Evaluation of the Feasibility, Validity, and Reliability of Laparoscopic Skills Assessment in the Operating RoomAnnals of Surgery, 2007
- The assessment of professional competence: Developments, research and practical implicationsAdvances in Health Sciences Education, 1996