Effects of Questionnaire-Based Diagnosis and Training on Inter-Rater Reliability Among Practitioners of Traditional Chinese Medicine
- 1 July 2009
- journal article
- research article
- Published by SAGE Publications in The Journal of Alternative and Complementary Medicine
- Vol. 15 (7) , 703-709
Abstract
Objective: To investigate whether a training process that focused on a questionnaire-based diagnosis in Traditional Chinese Medicine (TCM), and developing diagnostic consensus, would improve the agreement of TCM diagnoses among 10 TCM practitioners evaluating patients with temporomandibular joint disorder (TMJD). Design and setting: Evaluation of a diagnostic training program at the Department of Family and Community Medicine, University of Arizona, Tucson Arizona, and the Oregon College of Oriental Medicine, Portland, Oregon. Subjects: Screened participants for a study of TCM for TMJD. Practitioners: Ten (10) licensed acupuncturists with a minimum of 5 years licensure and education in Chinese herbs. Methods: A training session using a questionnaire-based diagnostic form was conducted, followed by waves of diagnostic sessions. Between sessions, practitioners discussed the results of the previous round of participants with a focus on reducing variability in primary diagnosis and severity rating of each diagnosis: 3 waves of 5 patients were assessed by 4 practitioner pairs for a total of 120 diagnoses. At 18 months, practitioners completed a recalibration exercise with a similar format with a total of 32 diagnoses. These diagnoses were then examined with respect to the rate of agreement among the 10 practitioners using inter-rater correlations and kappas. Results: The inter-rater correlation with respect to the TCM diagnoses among the 10 practitioners increased from 0.112 to 0.618 with training. Statistically significant improvements were found between the baseline and 18 month exercises (p < 0.01). Conclusions: Inter-rater reliability of TCM diagnosis may be improved through a training process and a questionnaire-based diagnosis process. The improvements varied by diagnosis, with the greatest congruence among primary and more severe diagnoses. Future TCM studies should consider including calibration training to improve the validity of results.Keywords
This publication has 25 references indexed in Scilit:
- A Pilot Whole Systems Clinical Trial of Traditional Chinese Medicine and Naturopathic Medicine for the Treatment of Temporomandibular DisordersThe Journal of Alternative and Complementary Medicine, 2008
- Traditional Chinese Medicine Tongue Inspection: An Examination of the Inter- and Intrapractitioner Reliability for Specific Tongue CharacteristicsThe Journal of Alternative and Complementary Medicine, 2008
- The Bipolar Depression Rating Scale (BDRS): its development, validation and utilityBipolar Disorders, 2007
- Reliability and validity of the scale for the assessment and rating of ataxia: A study in 64 ataxia patientsMovement Disorders, 2007
- Proposal for an Integrated Evaluation Model for the Study of Whole Systems Health Care in CancerIntegrative Cancer Therapies, 2006
- Methodological Challenges in Whole Systems ResearchThe Journal of Alternative and Complementary Medicine, 2006
- Complementary and alternative medicine whole systems research: Beyond identification of inadequacies of the RCTComplementary Therapies in Medicine, 2005
- Agreements among traditional Chinese medicine practitioners in the diagnosis and treatment of irritable bowel syndromeAlimentary Pharmacology & Therapeutics, 2004
- Addressing Spirituality as a Key Element of Health Care and Healing: An Interview with Rosemary AlteaAlternative and Complementary Therapies, 2003
- Measuring nominal scale agreement among many raters.Psychological Bulletin, 1971