Zimbabwe I

EVALUATION DESIGN

The Zimbabwe results-based financing (RBF) pilot program started in two districts, and in March 2012 was expanded to 16 additional pilot districts. In 2014, the Ministry of Health and Child Care (MOHCC), with the support of the Health Transition Fund administered by UNICEF, scaled up the RBF program to the remaining 44 rural districts in the country. The RBF pilot program aims to increase access to care by abolishing fees in pilot program districts. Furthermore, the project seeks to improve maternal and child health services by including three additional features: (i) results-based contracting; (ii) management and capacity building; and (iii) monitoring and documentation. Under the first component, health facilities received financial resources based on the quantity and quality of the services they provide, with a focus on maternal and child health.

To evaluate whether the pilot program improved the provision and quality of health care as intended, a quasi-experimental evaluation was designed. The evaluation was comprised of both quantitative and qualitative approaches. The quantitative evaluation compared 16 RBF districts to 16 counterfactual districts (control districts). These control districts were chosen to match RBF districts closely on various characteristics such as remoteness, type of constituent facilities, demographics, and rates of health care utilization. The impact of the RBF intervention was then estimated using a “differences-in-differences” estimator, which compares the change in the outcomes of interest for treated households or facilities between baseline and follow-up with the change over the same period in the outcomes for controls.

FINDINGS

The evaluation results show improvement from baseline to midline in both the RBF and control districts for the RBF coverage indicators. Key indicators like delivery by a skilled provider, in-facility delivery, and cesarean section deliveries improved faster in RBF districts than in control districts, although there was a general increase of these indicators across Zimbabwe. In addition, there was a mixed but positive message on the quality of care under the RBF program. For instance, within RBF districts, significant improvements were observed for selected measure of structural quality. For example, there was a higher incidence of biomedical waste disposal and increase availability of iron tablets, folic acid, and urine dipsticks. There was also increased availability of certain equipment such as electric autoclaves and refrigerators. Conversely, for both the RBF and control districts, the availability of the majority of medicines, supplies, and equipment remained largely unchanged, with minor fluctuations across products from baseline.

The evaluation results also found little justification for the risk of “task shifting” to incentivized (targeted) services at the expense of those that are not incentivized. None of the non-incentivized services investigated showed a decline in the number of cases treated, as would be expected if task shifting affected these services. For many of the non-incentivized services, there appears to be a slight increase in volume, suggesting that service coverage actually increased for a broader set of services than those directly incentivized.

Further, the findings also revealed that under the RBF program, there was improved autonomy and decentralized decision-making, strengthened facility-level management, and governance. For instance, health facilities experienced more autonomy; in particular, health staff in the RBF districts were more likely to allocate their facility budget according to their needs. RBF facilities also reported an increase in weekly operating hours for antenatal care and under-five clinic services, although these differences were no significant. There was also more meetings and greater participation of communities in decision making to prioritize resources to improve health facilities.

The results from the qualitative evaluation also highlighted important channels of influence. These include regular and structured supervision yielding feedback to improve performance, enhanced community participation, and team-based incentives facilitating teamwork. The qualitative findings also showed that health staff were strongly motivated by incentives. However, the nurses also experienced heavy workloads as they divided their attention between supervisory, administrative, facility-level management, and technical duties. The RBF-linked extra tasks in reporting, local procurement, and organizing logistics further aggravated the shortage and workload situation in health facilities. Further, the nurses expressed their dissatisfaction with reduced unity prices of services for RBF indicators that took place in 2013, the proportion of incentives relative to their tasks and those of their peers, the inadequate living accommodations, the limited capacity of their supervisions, and the limited leadership ability among the managers of health facilities.

Overall, there were several key insights that emerged from the evaluation. First, some of the coverage indicators that displayed the lowest degree of change under RBF also exhibited the highest baseline coverage rates. This suggests that incentivizing these indicators might not present a highly efficient leverage of program funds. Another insight is the importance of continued innovations on ways to incentivize quality measures of care, which are more complex than coverage indicators. Therefore, it might be judicious to start incentivizing structural quality indicators before progressively introducing process measures of clinical care. This is critical, as it will allow health providers to address less complex quality-of-care issues first, develop a better understanding of RBF programs and quality of care. Finally, RBF programs should not be isolated from broader health systems reforms and complementary interventions. Instead, it should be viewed as an entry point to tackling wider systemic issues that are brought to the fore when a RBF program is rolled out.