Melissa Ann Marx

Estimates of excess mortality can provide insight into direct and indirect impacts of the coronavirus disease 2019 (COVID-19) pandemic beyond deaths specifically attributed to COVID-19. We analyzed death certificate data from Baltimore City, Maryland, from March 1, 2020, to March 31, 2021, and found that 1,725 individuals (95% confidence interval: 1,495, 1,954) died in excess of what was expected from all-cause mortality trends in 2016–2019; 1,050 (61%) excess deaths were attributed to COVID-19. Observed mortality was 23%–32% higher than expected among individuals aged 50 years and older. Non-White residents of Baltimore City also experienced 2 to 3 times higher rates of excess mortality than White residents (e.g., 37.4 vs. 10.7 excess deaths per 10,000 population among Black residents vs. White residents). There was little to no observed excess mortality among residents of hospice, long-term care, and nursing home facilities, despite accounting for nearly 30% (312/1,050) of recorded COVID-19 deaths. There was significant geographic variation in excess mortality within the city, largely following racial population distributions. These results demonstrate the substantial and unequal impact of the COVID-19 pandemic on Baltimore City residents and the importance of building robust, timely surveillance systems to track disparities and inform targeted strategies to remediate the impact of future epidemics.

Background: Data that capture implementation strength can be combined in multiple ways across content and health system levels to create a summary measure that can help us to explore and compare program implementation across facility catchment areas. Summary indices can make it easier for national policymakers to understand and address variation in strength of program implementation across jurisdictions. In this paper, we describe the development of an index that we used to describe the district-level strength of implementation of Malawi’s national family planning program. Methods: To develop the index, we used data collected during a 2017 national, health facility and community health worker Implementation Strength Assessment survey in Malawi to test different methods to combine indicators within and then across domains (4 methods—simple additive, weighted additive, principal components analysis, exploratory factor analysis) and combine scores across health facility and community health worker levels (2 methods—simple average and mixed effects model) to create a catchment area-level summary score for each health facility in Malawi. We explored how well each model captures variation and predicts couple-years protection and how feasible it is to conduct each type of analysis and the resulting interpretability. Results: We found little difference in how the four methods combined indicator data at the individual and combined levels of the health system. However, there were major differences when combining scores across health system levels to obtain a score at the health facility catchment area level. The scores resulting from the mixed effects model were able to better discriminate differences between catchment area scores compared to the simple average method. The scores using the mixed effects combination method also demonstrated more of a dose–response relationship with couple-years protection. Conclusions: The summary measure that was calculated from the mixed effects combination method captured the variation of strength of implementation of Malawi’s national family planning program at the health facility catchment area level. However, the best method for creating an index should be based on the pros and cons listed, not least, analyst capacity and ease of interpretability of findings. Ultimately, the resulting summary measure can aid decision-makers in understanding the combined effect of multiple aspects of programs being implemented in their health system and comparing the strengths of programs across geographies.

Many countries have adopted integrated community case management (iCCM) to reduce mortality among children under five years from common childhood illnesses. The 2016–2020 Malian Red Cross iCCM prog...

Increasing coverage of evidence-based maternal, neonatal, child, reproductive health and nutrition (MNCRHN) programs in low- and middle-income countries has coincided with dramatic improvements in health despite variable quality of implementation. Comprehensive evaluation to inform program improvement requires standardized but adaptable tools, which the Real Accountability, Data Analysis for Results (RADAR) project has developed. To inform selection of tools and methods packages (‘packages’) to measure program quality of care (QoC), we documented experiences testing the packages, which were developed and adapted based on global and local expertise, and pre- and pilot-testing. We conducted cross-sectional studies in 2018–2019 on the quality of 1) integrated community case management, 2) counseling on maternal, infant, and young child feeding, 3) intrapartum care, and 4) family planning counseling in Mali, Mozambique, Tanzania, and Malawi. Herein we describe package performance and highlight experiences that inform their selection and use. Direct observation packages provided high-quality, immediately applicable results but they required specialized expertise, in-person collection, adequate patient volume, reasonable wait times, and unambiguously ‘correct’ provision of care. General satisfaction questions from exit interview packages produced unvaryingly positive responses despite variable observed quality of care. Variation increased when questions were more targeted, but findings on caregiver and client’s recall of recommendations were more actionable. When interactive, clinical vignettes can capture knowledge of clinical care. But for conditions that can be simulated, like provision of family planning counseling, we could capture provider practice from simulated clients. Clinicians could more easily demonstrate tactile aspects of intrapartum care using observed structured clinical examinations, but this method required storage and transport of the required mannequins. Based on our findings we recommend ten questions upon which evaluators can base package selection. Findings from these packages inform programs and, in the context of comprehensive program evaluation enable us to link programs with impact.

Background: As the global burden of malaria decreases, routine health information systems (RHIS) have become invaluable for monitoring progress towards elimination. The District Health Information System, version 2 (DHIS2) has been widely adopted across countries and is expected to increase the quality of reporting of RHIS. In this study, we evaluated the quality of reporting of key indicators of childhood malaria from January 2014 through December 2017, the first 4 years of DHIS2 implementation in Senegal. Methods: Monthly data on the number of confirmed and suspected malaria cases as well as tests done were extracted from the Senegal DHIS2. Reporting completeness was measured as the number of monthly reports received divided by the expected number of reports in a given year. Completeness of indicator data was measured as the percentage of non-missing indicator values. We used a quasi-Poisson model with natural cubic spline terms of month of reporting to impute values missing at the facility level. We used the imputed values to take into account the percentage of malaria cases that were missed due to lack of reporting. Consistency was measured as the absence of moderate and extreme outliers, internal consistency between related indicators, and consistency of indicators over time. Results: In contrast to public facilities of which 92.7% reported data in the DHIS2 system during the study period, only 15.3% of the private facilities used the reporting system. At the national level, completeness of facility reporting increased from 84.5% in 2014 to 97.5% in 2017. The percentage of expected malaria cases reported increased from 76.5% in 2014 to 94.7% in 2017. Over the study period, the percentage of malaria cases reported across all districts was on average 7.5% higher (P < 0.01) during the rainy season relative to the dry season. Reporting completeness rates were lower among hospitals compared to health centers and health posts. The incidence of moderate and extreme outlier values was 5.2 and 2.3%, respectively. The number of confirmed malaria cases increased by 15% whereas the numbers of suspected cases and tests conducted more than doubled from 2014 to 2017 likely due to a policy shift towards universal testing of pediatric febrile cases. Conclusions: The quality of reporting for malaria indicators in the Senegal DHIS2 has improved over time and the data are suitable for use to monitor progress in malaria programs, with an understanding of their limitations. Senegalese health authorities should maintain the focus on broader adoption of DHIS2 reporting by private facilities, the sustainability of district-level data quality reviews, facility-level supervision and feedback mechanisms at all levels of the health system.

Background: To explore the association between the strength of implementation of family planning (FP) programs on the use of modern contraceptives. Specifically, how strongly these programs are being implemented across a health facility’s catchment area in Malawi and the odds of a woman in that catchment area is using modern contraceptives. This information can be used to assess whether the combined impact of multiple large-scale FP programs is leading to change in the health outcomes they aim to improve. Methods and findings: We used data from the 2017 Implementation Strength Assessment (ISA) that quantified how much of family planning programs at the health facility and community health worker levels were being implemented across every district of Malawi. We used a summary measure developed in a previous study that employs quantitative methods to combine data across FP domains and health system levels. We tested the association of this summary measure for implementation strength with household data from the 2015 Malawi Demographic Health Survey (DHS). We found that areas with stronger implementation of FP programs had higher odds of women using modern contraceptives compared with areas with weaker implementation. The association of ISA with use of modern contraception was different by education, marital status, and geography. After controlling for these factors, we found that the adjusted odds of using a modern contraceptive was three times higher in catchment areas with high implementation strength compared to those with lower strength. Conclusion: Metrics that summarize how strongly FP programs are being implemented were used to show a statistically significantly positive relationship between increasing implementation strength and higher rates of modern contraceptive use. Decisionmakers at the various levels of health authority can use this type of summary measure to better understand the combined impact of their diverse FP programming and inform future programmatic and policy decisions. The findings also reinforce the idea that having a well-supported and supplied cadre of community health workers supplementing FP provision at the health facility can be an important health systems mechanism, especially in rural settings and to target youth populations.

Background: To effectively deliver on proposed objectives, it is vital that practitioners, policymakers, and other stakeholders are able to clearly understand how strongly their large-scale program is being implemented. This study sought to test the feasibility, cost-effectiveness, and validity of a phone-based method as an innovative and cost-efficient approach to assessing program implementation strength (through an Implementation Strength Assessment - ISA), alternative to the traditional in-person field methods. Methods: We conducted 701 mobile phone and 356 in-person interviews with facility in-Charges and two types of community health workers who provide family planning services in the Dowa and Ntcheu districts in Malawi. Responses received via the phone interview were validated through in-person review of records and inspections. Sensitivity and specificity were calculated to determine validity. Results: Most indicators at the health facility and community health worker levels were above a 70% threshold for sensitivity. However, there were fewer indicators that met this threshold for specificity. The primary reason for lower specificity was due to poor recordkeeping. Collecting data via mobile phone was found to be feasible and twice as cost-efficient as collecting the same data via in-person inspections. Conclusions: The rapid increase in mobile phone ownership and network availability in lower income countries could offer an alternative, cost-effective avenue to collect data for a better understanding of program implementation. Through rigorous assessment, this study found that using mobile phones could be a low-cost alternative to collect data on health system delivery of services, especially in places where routine data quality is poor and traditional, in-person methods are costly.

To assess the strength of implementation of family planning programs targeting youth (15-24) in Malawi with a specific focus on youth and the Youth-Friendly Health Services program We conducted 9781 mobile phone interviews with facility in-Charge Nurses and health workers (health facility workers, health surveillance assistants [HSAs] and community-based distribution agent [CBDAs]) who provide family planning (FP) services across the 28 districts. Responses were entered in tablet using Open Data Kit. They were summarized and presented using R, Stata (College Station, TX, USA, StatsReport, JHU, Baltimore, MD, USA) and ArcView GIS (ESRI, Redlands, CA, USA). Availability of key products was a challenge across all health worker types as only 39% of health facilities, 29% of HSAs and 45% of CBDAs had all the FP methods they are supposed to provide on the day of the interview. About 50% of health workers were supervised within past 90 days preceding the study. Despite most facilities saying that they provide youth friendly health services, youth-specific FP guidelines or protocols were not available in 43% of facilities that provide these services and only 33% of facilities had special rooms and 58% have special days for youth. The commodity supply system needs to ensure that all facilities and workers have a consistent supply of all contraceptive methods. Government and program implementers should ensure availability of all FP guidelines and information, education, communication materials at all service delivery points and facilitate creation of special rooms or days for youth.

Malawi has made progress in increasing its overall modern contraceptive prevalence rate since 2000, resulting in a dramatic reduction in its total fertility rate. However, youth, 15–24 years, have not had the same successes. Teenage pregnancies are on the rise and little progress has been made in reducing unmet need for family planning among youth. With two-thirds of the population under the age of 25 and with Malawi’s rapid population growth, reducing unmet need for family planning among youth remains a priority for the government’s reproductive health agenda. To further explore this situation, we conducted a qualitative study to explore the perspectives of youth and adults about the drivers and barriers to youth accessing family planning in Malawi and their ideas to improve services. We conducted 34 focus group discussions with youth aged 15–24 and parents or legal guardians of female youth in 3 districts in Malawi. Focus groups were translated and transcribed. Data was input into Dedoose and analyzed using a thematic framework to identify broader patterns and themes. Youth participants felt motivated to use family planning to protect themselves from sexually transmitted diseases and to prevent unwanted pregnancies. Females focused on the consequences of unplanned pregnancies and believed family planning services were targeted primarily at them, while males thought family planning services targeted males and females equally. Barriers to youth accessing family planning included contraception misconceptions, the costs of family planning services, and negative attitudes. Parents had mixed views on family planning. While many parents acknowledged they could play a role in supporting youth, most said they are reluctant to support youth using family planning. Participants said improving counseling services, integrating family planning services and education within school curricula, and utilizing youth clubs could improve family planning services for youth. Policy makers and program implementers should consider the diverse preferences among youth and parents and continue seeking their input when designing policies and programs. Youth clubs and school-based services were among the most common suggestions. However, the effectiveness of youth clubs and school-based initiatives to increase contraceptive use among youth in Malawi is not clear.

Background: Routine health data can guide health systems improvements, but poor quality of these data hinders use. To address concerns about data quality in Malawi, the Ministry of Health and National Statistical Office conducted a data quality assessment (DQA) in July 2016 to identify systems-level factors that could be improved. Methods: We used 2-stage stratified random sampling methods to select health centers and hospitals under Ministry of Health auspices, included those managed by faith-based entities, for this DQA. Dispensaries, village clinics, police and military facilities, tertiary-level hospitals, and private facilities were excluded. We reviewed client registers and monthly reports to verify availability, completeness, and accuracy of data in 4 service areas: antenatal care (ANC), family planning, HIV testing and counseling, and acute respiratory infection (ARI). We also conducted interviews with facility and district personnel to assess health management information system (HMIS) functioning and systems-level factors that may be associated with data quality. We compared systems and quality factors by facility characteristics using 2-sample t tests with Welch9s approximation, and calculated verification ratios comparing total entries in registers to totals from summarized reports. Results: We selected 16 hospitals (of 113 total in Malawi), 90 health centers (of 466), and 16 district health offices (of 28) in 16 of Malawi9s 28 districts. Nearly all registers were available and complete in health centers and district hospitals, but data quality varied across service areas; median verification ratios comparing register and report totals at health centers ranged from 0.78 (interquartile range [IQR]: 0.25, 1.07) for ARI and 0.99 (IQR: 0.82, 1.36) for family planning to 1.00 (IQR: 0.96, 1.00) for HIV testing and counseling and 1.00 (IQR: 0.80, 1.23) for ANC. More than half (60%) of facilities reported receiving a documented supervisory visit for HMIS in the prior 6 months. A recent supervision visit was associated with better availability of data (P=.05), but regular district- or central-level supervision was not. Use of data by the facility to track performance toward targets was associated with both improved availability (P=.04) and completeness of data (P=.02). Half of facilities had a full-time statistical clerk, but their presence did not improve the availability or completeness of data (P=.39 and P=.69, respectively). Conclusion: Findings indicate both strengths and weaknesses in Malawi9s HMIS performance, with key weaknesses including infrequent data quality checks and unreliable supervision. Efforts to strengthen HMIS in low- and middle-income countries should be informed by similar assessments. Nearly all facility registers were available and complete. But accuracy varied, with antenatal care and HIV testing and counseling performing the best and family planning and acute respiratory infections data less well. Most facilities visibly displayed routine health data and most hospitals and district health offices had staff trained in health management information systems, but training was lacking at the facility level as were routine data quality checks and regular supervision.