A combined Fuzzy and Naïve Bayesian strategy can be used to assign event codes to injury narratives
- 11 April 2011
- journal article
- Published by BMJ in Injury Prevention
- Vol. 17 (6) , 407-414
Abstract
Background Bayesian methods show promise for classifying injury narratives from large administrative datasets into cause groups. This study examined a combined approach where two Bayesian models (Fuzzy and Naïve) were used to either classify a narrative or select it for manual review. Methods Injury narratives were extracted from claims filed with a worker's compensation insurance provider between January 2002 and December 2004. Narratives were separated into a training set (n=11,000) and prediction set (n=3,000). Expert coders assigned two-digit Bureau of Labor Statistics Occupational Injury and Illness Classification event codes to each narrative. Fuzzy and Naïve Bayesian models were developed using manually classified cases in the training set. Two semi-automatic machine coding strategies were evaluated. The first strategy assigned cases for manual review if the Fuzzy and Naïve models disagreed on the classification. The second strategy selected additional cases for manual review from the Agree dataset using prediction strength to reach a level of 50% computer coding and 50% manual coding. Results When agreement alone was used as the filtering strategy, the majority were coded by the computer (n=1,928, 64%) leaving 36% for manual review. The overall combined (human plus computer) sensitivity was 0.90 and positive predictive value (PPV) was >0.90 for 11 of 18 2-digit event categories. Implementing the 2nd strategy improved results with an overall sensitivity of 0.95 and PPV >0.90 for 17 of 18 categories. Conclusions A combined Naïve-Fuzzy Bayesian approach can classify some narratives with high accuracy and identify others most beneficial for manual review, reducing the burden on human coders.Keywords
This publication has 10 references indexed in Scilit:
- The use of narrative text for injury surveillance research: A systematic reviewAccident Analysis & Prevention, 2010
- Bayesian methods: a useful tool for classifying injury narratives into cause groupsInjury Prevention, 2009
- Completeness and accuracy of International Classification of Disease (ICD) external cause of injury codes in emergency department electronic dataInjury Prevention, 2007
- Unspecified falls among youth: predictors of coding specificity in the emergency departmentInjury Prevention, 2006
- Hybrid singular value decomposition; a model of human text classificationInternational Journal of Human Factors Modelling and Simulation, 2006
- Using narrative text and coded data to develop hazard scenarios for occupational injury interventionsInjury Prevention, 2004
- Computerized coding of injury narrative data from the National Health Interview SurveyAccident Analysis & Prevention, 2004
- Public health approaches to occupational injury prevention: do they work?: Figure 1Injury Prevention, 2001
- Three perspectives on work-related injury surveillance systemsAmerican Journal of Industrial Medicine, 1997
- Motor vehicle crashes in roadway construction workzones: An analysis using narrative text from insurance claimsAccident Analysis & Prevention, 1996