Policy Brief: Understanding key drivers of poor data quality, usability, and use in Kano, Yobe, Niger, and Lagos, Nigeria

BACKGROUND

Data quality and use play a fundamental role in the success or failure of an immunization program.[1] Unavailability of quality data and poor use of data can undermine the government’s and stakeholders’ decision making and action for improvement in the system.[2]

The World Health Organization’s (WHO) Immunization Agenda 2030 recognizes that reliable data will help countries continuously improve their performance and efficiency.[3] To address the gaps in data quality, usability, and use, the government of Nigeria and partners developed a Data Quality Improvement Plan in 2017 and piloted a variety of initiatives to address this issue. While this plan helped to drive some improvements in the data reporting quality, discrepancies in administrative data from the health facilities and household survey data from interviews with caregivers remain. For instance, differences in reporting on pentavalent vaccine administration fell from 72 percent in 2015 to 34 percent in 2017 but were still well above the acceptable threshold level of 10 percent.[4]

Many studies in low- and middle-income countries, including Nigeria, have shown that interventions that encourage the positive practice of health workers improves compliance and achievement of broader system health goals[5],[6]. Some studies in Nigeria provide evidence linking poor data quality and use to structural and organizational barriers, but do not emphasize the exact causes attributed to behavioral factors. Insights from behavioral science can help determine key drivers of data quality, usability, and use at the Local Government Area (LGA) and health facility (HF) levels as well as inform intervention design and policy changes. Potential interventions that may encourage certain behaviors or practices that enable better data collection and management can be designed by aligning concepts gleaned with behavioral science literature and qualitative analysis.

To ensure that potential interventions to address data quality and use gaps adopt a behavioral science perspective, CHAI conducted a study to determine key drivers of quality, usability, and use at service delivery and HF management levels.

OBJECTIVES

The study aimed to understand health worker and primary healthcare (PHC) manager perspectives and actions concerning data quality and use within their contexts.

Specific objectives included:

• Identifying key drivers of poor PHC data quality, the usability of data, and the use of data for action at the HF and LGA levels.
• Identifying motivators (non-financial) that reinforce positive health worker practices and barriers to PHC data quality.
• Identifying measures to strengthen accountability for data quality and use for action at the LGA and HF levels.

STUDY DESIGN AND APPROACH

The study adopted cross-sectional qualitative and quantitative methods targeting routine immunization, antenatal care, and family planning service providers, PHC managers, partners, and state officials. It was conducted over an eight-week period between August 31 to October 25, 2020, across Yobe, Kano, Niger, and Lagos in Nigeria and involved desk reviews, observation of PHC service provision, and focus group discussions (FGDs), and key informant interviews (KIIs). A total of 88 KIIs, 48 FGDs, 192 observations of PHC sessions, and 447 surveys were conducted. Interviews were audio-recorded, de-identified, transcribed, and then coded thematically. Ethical and administrative approval was obtained from Nigeria’s National Health Research Ethics Committee, National Primary Health Care Development Agency, and State Primary Health Care Development Agencies in Kano, Lagos, Niger, and Yobe states.

KEY RESULTS

A) Drivers of poor data quality, usability, and use

Health workers’ motivations, attitudes, and pressure from higher levels were identified as the key internal and external drivers for poor data quality in the states.

• 73 percent of health workers expressed that their superiors pushed them to submit data on short notice to meet up with the submission deadline on DHIS2. This serves as an external influence on poor data quality at the facilities.
• 68 percent of health workers in the state indicated that the time given was insufficient to complete numerous data tools, owing to the increased workload.
• 84 percent and 64 percent of health workers and LGA managers, respectively, reported feeling unappreciated and unacknowledged for good data quality by supervisors. Underpayment of salary is a high demotivating factor among healthcare workers.

B) Non-financial incentives to reinforce positive data management practices

Provision of feedback and review of the National Health Management Information System (NHMIS) tools were identified as some of the key factors to incentivize and improve health worker practices:

• 28 percent and 55 percent of interviewed health workers and LGA managers, respectively, expressed that quality supervision of work and provision of constructive feedback will motivate and encourage positive practices for improvement in data quality.
• Improving the quality of registers and reducing the number of variables to record on the NHMIS tools was also identified as an incentive to improve the data collection practices among health workers and LGA managers in 87 percent of respondents.

C) Measures to strengthen accountability for data quality and use

• Findings from the key informant interviews (KIIs) and focus group discussions (FGDs) indicated a poor understanding of some key data characteristics, particularly usability and use. This signifies the need for simple and clear standard operating procedures (SOPs) to guide health workers and LGA managers on data use for decision making and performance management, and institute measures to monitor adherence during supportive supervision.
• Unclear data accountability mechanisms within the states result in persistent data quality issues, as well as poor health worker attitudes and practices.

POLICY RECOMMENDATIONS

Results from the study highlight that strategies that focus on people, processes, and tools— with an emphasis on accountability— are recommended to improve data quality. See the table below for a summary of recommendations.

This study was funded by the Bill and Melinda Gates Foundation. We also acknowledge the successful collaboration with National Primary Health Care Development Agency, Kano State Primary Health Care Management Board, Lagos State Primary Health Care Board, Niger State Primary Health Care Development Agency, and Yobe State Primary Health Care Management Board. Data for this study was collected by Sydani Initiative in 2021.

Contributing Authors: Sebastian Ilomuanya, Manager, Immunization Systems & Capacity; Muhammad Sheriff, Assistant Program Officer, Vaccines Program; Isa Adamu, Senior Analyst, Vaccines Program; Lawan Lawu, Analyst, Vaccines Program; Rogers Kanee, Analyst, Vaccines Program and Dr. Owens Wiwa, EVP Global Resources for Health, West and Central Africa, and Country Director Nigeria

[1] Papania M, Rodewald L. For better immunization coverage, measure coverage better. Lancet. 2006; 367: 965–966

[2] ibid

[3] World Health Organization. Immunization Agenda 2030.

[4] Administrative data is sourced for children aged 0-11 months while survey data looks at the corresponding birth cohort aged 12-23 months. Administrative data was sourced from DVD-MT and DHIS2 in 2015 and 2017 respectively. Survey data from 2016/2017 MICS-NICS and 2018 NDHS.

[5]  MCHIP. (2012). Creating Stronger Incentives for High-Quality Health Care in Low- and Middle-Income Countries. Retrieved from https://www.mchip.net/sites/default/files/QoC and PBI_Full report_Final.pdf

[6] Ogundeji, Y. Can Performance Based Financing Improve Quality of Healthcare in Nigeria? http://afhea.org/docs/presetationspdfs/Yewande%20Ogundeji%20-%20Can%20RBF%20improve%20quality%20and%20access%20to%20care%20in%20Nigeria%20-%20Copy.pdf. 2016