Computer-based coding of free-text job descriptions to efficiently identify occupations in epidemiological studies
- 21 April 2016
- journal article
- research article
- Published by BMJ in Occupational and Environmental Medicine
- Vol. 73 (6) , 417-424
Abstract
Background Mapping job titles to standardised occupation classification (SOC) codes is an important step in identifying occupational risk factors in epidemiological studies. Because manual coding is time-consuming and has moderate reliability, we developed an algorithm called SOCcer (Standardized Occupation Coding for Computer-assisted Epidemiologic Research) to assign SOC-2010 codes based on free-text job description components. Methods Job title and task-based classifiers were developed by comparing job descriptions to multiple sources linking job and task descriptions to SOC codes. An industry-based classifier was developed based on the SOC prevalence within an industry. These classifiers were used in a logistic model trained using 14 983 jobs with expert-assigned SOC codes to obtain empirical weights for an algorithm that scored each SOC/job description. We assigned the highest scoring SOC code to each job. SOCcer was validated in 2 occupational data sources by comparing SOC codes obtained from SOCcer to expert assigned SOC codes and lead exposure estimates obtained by linking SOC codes to a job-exposure matrix. Results For 11 991 case–control study jobs, SOCcer-assigned codes agreed with 44.5% and 76.3% of manually assigned codes at the 6-digit and 2-digit level, respectively. Agreement increased with the score, providing a mechanism to identify assignments needing review. Good agreement was observed between lead estimates based on SOCcer and manual SOC assignments (κ 0.6–0.8). Poorer performance was observed for inspection job descriptions, which included abbreviations and worksite-specific terminology. Conclusions Although some manual coding will remain necessary, using SOCcer may improve the efficiency of incorporating occupation into large-scale epidemiological studies.Keywords
This publication has 17 references indexed in Scilit:
- Large Unbalanced Credit Scoring Using Lasso-Logistic Regression EnsemblePLOS ONE, 2015
- Methods and feasibility of collecting occupational data for a large population-based cohort study in the United States: the reasons for geographic and racial differences in stroke studyBMC Public Health, 2014
- Beyond Crosswalks: Reliability of Exposure Assessment Following Automated Coding of Free-Text Job Descriptions for Occupational EpidemiologyAnnals of Occupational Hygiene, 2014
- JEMs and Incompatible Occupational Coding Systems: Effect of Manual and Automatic Recoding of Job Codes on Exposure AssignmentAnnals of Occupational Hygiene, 2012
- Hypertension and Risk of Renal Cell Carcinoma Among White and Black AmericansEpidemiology, 2011
- A case–control study of reproductive factors and renal cell carcinoma among black and white women in the United StatesCancer Causes & Control, 2011
- Occupation and bladder cancer in a population-based case–control study in Northern New EnglandOccupational and Environmental Medicine, 2010
- A computer system for coding occupationAmerican Journal of Industrial Medicine, 2006
- Occupational Risk Factors for Cancer of the Gastric CardiaJournal of Occupational and Environmental Medicine, 1998
- Stacked generalizationNeural Networks, 1992