Can surveys of women accurately track indicators of maternal and newborn care? A validity and reliability study in Kenya
Authors:
Open Access
- 10 September 2016
- journal article
- research article
- Published by International Society of Global Health in Journal of Global Health
- Vol. 6 (2) , 020502
Abstract
Tracking progress on maternal and newborn survival requires accurate information on the coverage of essential interventions. Despite widespread use, most indicators measuring maternal and newborn intervention coverage have not been validated. This study assessed the ability of women delivering in two Kenyan hospitals to recall critical elements of care received during the intrapartum and immediate postnatal period at two time points: hospital discharge and 13-15 months following delivery. Women's reports of received care were compared against observations by trained third party observers. Indicators selected for validation were either currently in use or have the potential to be included in population-based surveys. We used a mixed-methods approach to validate women's reporting ability. We calculated individual-reporting accuracy using the area under the receiver operating curve (AUC), population-level accuracy using the inflation factor (IF), and compared the accuracy of women's reporting at baseline and follow-up. We also assessed the consistency of women's reporting over time. We used in-depth interviews with a sub-set of women (n = 20) to assess their understanding of key survey terms. Of 606 women who participated at baseline and agreed to follow-up, 515 were re-interviewed. Thirty-eight indicators had sufficient sample size for validation analysis; ten met criteria for high or moderate reporting accuracy (0.60<AUC) alone and ten met criteria for low population-level bias alone (0.75<IF<1.25). There was a significant decline in the individual level reporting accuracy between baseline and follow-up for ten indicators. Seven indicators had moderate or higher (0.4≤rphi) consistency between self-reports at baseline and follow-up. Four indicators met all criteria at follow-up: support person was present during the birth, episiotomy, caesarean section, and low birthweight infant (<2500 g). The few indicators that women reported accurately at baseline were consistently recalled with accuracy at 13-15 months follow-up. Although there is deterioration in women's recall in some indicators over time, the extent of deterioration does not appreciably compromise reporting accuracy for indicators with high baseline validity. Indicators related to initial client assessment and the immediate postnatal period have generally low accuracy and poor reporting consistency over time.Keywords
This publication has 16 references indexed in Scilit:
- Global causes of maternal death: a WHO systematic analysisThe Lancet Global Health, 2014
- Measuring Coverage in MNCH: A Validation Study Linking Population Survey Derived Coverage to Maternal, Newborn, and Child Health Care Records in Rural ChinaPLOS ONE, 2013
- Measuring Coverage in MNCH: Validating Women’s Self-Report of Emergency Cesarean Sections in Ghana and the Dominican RepublicPLOS ONE, 2013
- Measuring Coverage in MNCH: Testing the Validity of Women's Self-Report of Key Maternal and Newborn Health Interventions during the Peripartum Period in MozambiquePLOS ONE, 2013
- Measuring Coverage in MNCH: New Findings, New Strategies, and Recommendations for ActionPLOS Medicine, 2013
- Skilled Birth Attendants: Who is Who? A Descriptive Study of Definitions and Roles from Nine Sub Saharan African CountriesPLOS ONE, 2012
- Validity and Reliability of Postpartum Morbidity Questionnaires in BeninMaternal and Child Health Journal, 2011
- Reducing Bias from Test Misclassification in Burden of Disease Studies: Use of Test to Actual Positive Ratio – New Test ParameterCroatian Medical Journal, 2008
- Statistical Methodology: I. Incorporating the Prevalence of Disease into the Sample Size Calculation for Sensitivity and SpecificityAcademic Emergency Medicine, 1996
- Predictive Value of a Single Diagnostic Test in Unselected PopulationsNew England Journal of Medicine, 1966