Susan Quach

Rates of Bordetella pertussis have been increasing in Alberta, Canada despite vaccination programs. Waning immunity from existing acellular component vaccines may be contributing to this. Vaccine effectiveness can be estimated using a variety of data sources including diagnostic codes from physician billing claims, public health records, reportable disease and laboratory databases. We sought to determine if diagnostic codes from billing claims (administrative data) are adequately sensitive and specific to identify pertussis cases among patients who had undergone disease-specific laboratory testing. Data were extracted for 2004–2014 from a public health communicable disease database that contained data on patients under investigation for B. pertussis (both those who had laboratory tests and those who were epidemiologically linked to laboratory-confirmed cases) in Alberta, Canada. These were deterministically linked using a unique lifetime person identifier to the provincial billing claims database, which contains International Classification of Disease version 9 (ICD-9) diagnostic codes for physician visits. We examined visits within 90 days of laboratory testing. ICD-9 codes 033 (whooping cough), 033.0 (Bordetella pertussis), 033.1 (B. parapertussis), 033.8 (whooping cough, other specified organism), and 033.9 (whooping cough, other unspecified organism) in any of the three diagnostic fields for a claim were classified as being pertussis-specific codes. We calculated sensitivity, specificity, positive (PPV) and negative (NPV) predictive values. We identified 22,883 unique patients under investigation for B. pertussis. Of these, 22,095 underwent laboratory testing. Among those who had a laboratory test, 2360 tested positive for pertussis. The sensitivity of a pertussis-specific ICD-9 code for identifying a laboratory-confirmed case was 38.6%, specificity was 76.9%, PPV was 16.0%, and NPV was 91.6%. ICD-9 codes from physician billing claims data have low sensitivity and moderate specificity to identify laboratory-confirmed pertussis among persons tested for pertussis.

Background: Evaluating the features and performance of health information systems can serve to strengthen the systems themselves as well as to guide other organizations in the process of designing and implementing surveillance tools. We adapted an evaluation framework in order to assess electronic immunization data collection systems, and applied it in two Ontario public health units. Methods: The Centers for Disease Control and Prevention’s Guidelines for Evaluating Public Health Surveillance Systems are broad in nature and serve as an organizational tool to guide the development of comprehensive evaluation materials. Based on these Guidelines, and informed by other evaluation resources and input from stakeholders in the public health community, we applied an evaluation framework to two examples of immunization data collection and examined several system attributes: simplicity, flexibility, data quality, timeliness, and acceptability. Data collection approaches included key informant interviews, logic and completeness assessments, client surveys, and on-site observations. Results: Both evaluated systems allow high-quality immunization data to be collected, analyzed, and applied in a rapid fashion. However, neither system is currently able to link to other providers’ immunization data or provincial data sources, limiting the comprehensiveness of coverage assessments. We recommended that both organizations explore possibilities for external data linkage and collaborate with other jurisdictions to promote a provincial immunization repository or data sharing platform. Conclusions: Electronic systems such as the ones described in this paper allow immunization data to be collected, analyzed, and applied in a rapid fashion, and represent the infostructure required to establish a population-based immunization registry, critical for comprehensively assessing vaccine coverage.

Although many studies have demonstrated the benefits of reminder/recall (RR) measures to address patient under-immunization and improve immunization coverage, they are not widely implemented by healthcare providers. We identified providers’ perceived barriers to their use from existing literature.

Background: During the pandemic (H1N1) 2009 influenza vaccination campaign, health regions in Canada collected client-level immunization data using fully electronic or hybrid systems, with the latter comprising both electronic and paper-based elements. The objective of our evaluation was to compare projected five-year costs associated with implementing these systems in Ontario public health units (PHUs) during pandemic and seasonal influenza vaccination campaigns. Methods: Six PHUs provided equipment and staffing costs during the pandemic (H1N1) 2009 influenza vaccination campaign and staffing algorithms for seasonal campaigns. We standardized resources to population sizes 100,000, 500,000 and 1,000,000, assuming equipment lifetime of five years and public health vaccine administration rates of 18% and 2.5% for H1N1 and seasonal campaigns, respectively. Two scenarios were considered: Year 1 pandemic and Year 1 seasonal campaigns, each followed by four regular influenza seasons. Costs were discounted at 5%. Results: Assuming a Year 1 pandemic, the five-year costs per capita for the electronic system decrease as PHU population size increases, becoming increasingly less costly than hybrid systems ($4.33 vs. $4.34 [100,000], $4.17 vs. $4.34 [500,000], $4.12 vs. $4.34 [1,000, 000]). The same trend is observed for the scenario reflecting five seasonal campaigns, with the electronic system being less expensive per capita than the hybrid system for all population sizes ($1.93 vs. $1.95 [100,000], $1.91 vs. $1.94 [500,000], $1.87 vs. $1.94 [1,000, 000]). Sensitivity analyses identified factors related to nurse hours as affecting the direction and magnitude of the results. Conclusions: Five-year cost projections for electronic systems were comparable or less expensive than for hybrid systems, at all PHU population sizes. An intangible benefit of the electronic system is having data rapidly available for reporting.

During the 2009 H1N1 immunization campaign, electronic and hybrid (comprising both electronic and paper components) systems were employed to collect client-level vaccination data in clinics across Canada. Because different systems were used across the country, the 2009 immunization campaign offered an opportunity to study the usability of the various data collection methods. A convenience sample of clinic staff working in public health agencies and hospitals in 9 provinces/territories across Canada completed a questionnaire in which they indicated their level of agreement with seven statements regarding the usability of the data collection system employed at their vaccination clinic. Questions included overall ease of use, effectiveness of the method utilized, efficiency at completing tasks, comfort using the method, ability to recover from mistakes, ease of learning the method and overall satisfaction with the method. A 5-point Likert-type scale was used to measure responses. Most respondents (96%) were employed in sites run by public health. Respondents included 186 nurses and 114 administrative staff, among whom 90% and 47%, respectively, used a paper-based method for data collection. Approximately half the respondents had a year or less of experience with immunization-related tasks during seasonal influenza campaigns. Over 90% of all frontline staff found their data collection method easy to use, perceived it to be effective in helping them complete their tasks, felt quick and comfortable using the method, and found the method easy to learn, regardless of whether a hybrid or electronic system was used. This study demonstrates that there may be a greater willingness of frontline immunization staff to adapt to new technologies than previously perceived by decision-makers. The public health community should recognize that usability may not be a barrier to implementing electronic methods for collecting individual-level immunization data.

The collection of individual-level pandemic (H1N1) 2009 influenza immunization data was considered important to facilitate optimal vaccine delivery and accurate assessment of vaccine coverage. These data are also critical for research aimed at evaluating the new vaccine's safety and effectiveness. Systems used to collect immunization data include manual approaches in which data are collected and retained on paper, electronic systems in which data are captured on computer at the point of vaccination and hybrid systems which are comprised of both computerized and manual data collection components. This study's objective was to compare the efficiencies and perceptions of data collection methods employed during Canada's pandemic (H1N1) 2009 influenza vaccination campaign.