Marcel E. Reinders

The effect of patient feedback interventions as a method of improving physicians' consultation (i.e., communication, interpersonal) skills is equivocal; research is scarce, and methods and rigor vary. The authors conducted this systematic review to analyze the educational effect of feedback from real patients on physicians' consultation skills at the four Kirkpatrick levels. The authors searched five databases (PubMed, EMBASE, Cochrane, PsycInfo, ERIC; April 2010). They included empirical studies of all designs (randomized controlled, quasi-experimental, cross-sectional, and qualitative) if the studies concerned physicians in general health care who received formal feedback regarding their consultation skills from real patients. The authors have briefly described aspects of the included studies, analyzed their quality, and examined their results by Kirkpatrick educational effect level. The authors identified 15 studies (10 studies in primary care; 5 in other specialties) in which physicians received feedback in various ways (e.g., aggregated patient reports or educator-mediated coaching sessions), conducted in the United States, the Netherlands, the United Kingdom, Australia, and Canada. All studies that assessed level 1 (valuation), level 2 (learning), and level 3 (intended behavior) demonstrated positive results; however, only four of the seven studies that assessed level 4 (change in actual performance or results) found a beneficial effect. Some evidence for the effectiveness of using feedback from real patients to improve knowledge and behavior exists; however, before implementing patient feedback into training programs, educators and policy makers should realize that the evidence for effecting actual improvement in physicians' consulting skills is rather limited.

Medical Education 2011; 45 : 578–584 Objectives  Training in and assessment of consultation skills are high on the agenda of vocational training institutes for postgraduate training. There is a need to establish valid and reliable instruments to assess consultation skills in authentic settings. We investigated the number of assessors and observations needed to achieve reliable assessments of the consultation skills of general practice trainees (GPTs) using a communication instrument (MAAS‐Global) and either standardised patient (SP) encounters or videotaped real patient (RP) encounters. Methods  Eight teachers at the Vrije Universiteit (VU) University Medical Centre in Amsterdam attended a training course on the use of the MAAS‐Global instrument, which they subsequently used to assess the consultation skills of 53 GPTs in 176 videotaped consultations (102 with SPs, 74 with RPs). All consultations were randomly allocated and assessed by two teachers independently. The reliability of the ratings was estimated using generalisability theory. Results  It was easier to obtain acceptable reliability using RP consultations than SP consultations. Two assessors and five consultations were required to achieve minimal reliability (generalisability coefficient 0.7) with RPs, whereas three assessors and 30 consultations were needed to achieve minimal reliability with SPs. Conclusions  Inter‐observer and context variability in the assessment of the consultation skills of GPTs remains high. To achieve acceptable levels of reliability, large samples of observations are required in both formats, but, interestingly, RP encounters require a smaller sample than SP encounters.

Medical Education 2010: 44 : 156–164 Context  This study aimed to assess if an additional patient feedback training programme leads to better consultation skills in general practice trainees (GPTs) than regular communication skills training, and whether process measurements (intensity of participation in the programme) predict the effect of the intervention. Methods  We carried out a controlled trial in which two sub‐cohorts of GPTs were allocated to an intervention group ( n = 23) or a control group ( n = 30), respectively. In 2006, allocated first‐year GPTs in the VU University Medical Centre attended a patient feedback training programme in addition to the regular communication skills training. The control group attended only regular communication skills training. Trainees were assessed by simulated patients who visited the practices and videotaped the consultations at baseline and after 3 months. The videotapes were randomly assigned to eight trained staff members. The MAAS‐Global Instrument (range 0–6) was used to assess (a change in) trainee consultation skills. Results were analysed using a multi‐level, linear mixed‐model analysis. Results  Data on 50 GPTs were available for the follow‐up analysis. Both intervention group and control group GPTs improved their consultation skills: mean MAAS‐Global scores for all participants were 3.29 (standard deviation [SD] 0.75) at baseline and 3.54 (SD 0.66) at follow‐up ( P  = 0.047). The improvement in MAAS‐Global scores in the intervention group did not differ significantly from the improvement in the control group. The analysis showed a trend for intensity of participation in the patient feedback programme to predict greater improvement in MAAS‐Global scores. Discussion  Although the baseline scores were already in the high range, consultation skills in both groups improved significantly. This is reassuring for current teaching methods. The patient feedback programme did not improve consultation skills more than regular communication skills training. However, a subgroup of GPTs who participated intensively in the programme did improve their consultation skills further in comparison with the less motivated subgroup.