Jill M. Huber

Introduction: Learners may subconsciously change their behavior once they know they are being observed, and this Hawthorne effect should be considered when designing assessments of learner behavior. While there is a growing body of literature to suggest direct observation is the ideal standard for formative assessment, the best method to directly observe learners is unknown. We explored scheduled and unscheduled methods of direct observation among internal medicine residents in the outpatient continuity clinic to advance the understanding of both observation methods. Methods: We conducted a thematic analysis of faculty and internal medicine residents in an outpatient clinic setting. A semi-structured interview guide for focus group sessions was created. Focus groups were used to explore the internal medicine resident and core teaching faculty perceptions of the scheduled and unscheduled direct observation methods in the outpatient clinc. An experienced qualitative research interviewer external to the internal medicine residency was moderating the sessions. Eight peer focus groups were held. Abstraction of themes from focus group transcripts identified resident and faculty perceptions of the different observation methods. Results: Focus groups had 14 resident participants and 14 faculty participants. Unscheduled observations were felt to be more authentic than scheduled observations since residents perceived their behavior to be unmodified. Unscheduled observations allowed for increased numbers of observations per resident, which permitted more frequent formative assessments. Residents and faculty preferred remote video observation compared to in-room observation. Participants found direct observation a useful learning tool for high-yield, specific feedback. Conclusions: Unscheduled remote direct observation captures authentic clinical encounters while minimizing learner behavior modification. An unscheduled observation approach results in more frequent formative assessment and therefore in more instances of valuable feedback compared to scheduled observations. These findings can help guide the best practice approaches to direct clinical observation in order to enhance residents learning and experience.

Obesity is a leading preventable cause of death and disability and is associated with a lower health-related quality of life. We evaluated the impact of telecoaching conducted by a counselor trained in motivational interviewing paired with a portion control plate for obese patients in a primary care setting. We conducted a randomized, clinical trial among patients in a primary care practice in the midwestern United States. Patients were randomized to either usual care or an intervention including telecoaching with a portion control plate. The intervention was provided during a 3-month period with follow-up of all patients through 6 months after randomization. The primary outcomes were weight, body mass index (BMI),waist circumference, and waist to hip ratio measured at baseline, 6, 12, 18, and 24 weeks. Secondary outcomes included measures assessing eating behaviors, self-efficacy, and physical activity at baseline and at 12 and 24 weeks. A total of 1,101 subjects were pre-screened, and 90 were randomly assigned to telecoaching plus portion control plate (n = 45) or usual care (n = 45). Using last-value carried forward without adjustment for baseline demographics, significant reductions in BMI (estimated treatment effect -0.4 kg/m², P = .038) and waist to hip ratio (estimated treatment effect -.02, P = .037) at 3 months were observed in the telecoaching plus portion control plate group compared to usual care. These differences were not statistically significant at 6 months. In females, the telecoaching plus portion control plate intervention was associated with significant reductions in weight and BMI at both 3 months (estimated treatment effect -1.6 kg, P = .016 and -0.6 kg/m², P = .020) and 6 months (estimated treatment effect -2.3 kg, P = .013 and -0.8 kg/m², P = .025). In males, the telecoaching plus portion control intervention was associated with a significant reduction in waist to hip ratio at 3 months (estimated treatment effect -0.04, P = .017), but failed to show a significant difference in weight and BMI. Telecoaching with a portion control plate can produce positive change in body habitus among obese primary care patients; however, changes depend upon sex. ClinicalTrials.gov NCT02373878, 13 February 2015. https://clinicaltrials.gov/ct2/show/NCT02373878 .