Zambia I

EVALUATION DESIGN

The Zambia Results-Based Financing (RBF) pilot project was implemented between 2008 and 2014 at district level in eleven rural districts.  In these districts, health centers were contracted to deliver a specified package of essential maternal and child health (MCH) services, for which the program paid fee-for-service claims. Further, additional financial resources were made available to health centers if the quality of services, measured by a quality score, surpassed an agreed threshold. The pilot project also included an impact evaluation, a cost-effectiveness analysis, a process evaluation, and a sub-analysis on the impact of RBF on human resources for health outcomes. In particular, the   impact evaluation was designed to measure the program’s effect on targeted health indicators and other population outcomes of interest (i.e. coverage of health services, quality of care, and health system functionality).

Ten of the eleven districts which were implementing the RBF were included in the impact evaluation. The impact evaluation was a three-armed experimental study design. Thirty districts were matched in groups of three based on baseline performance on key health systems and outcomes indicators. These districts were then randomly assigned to three equal groups (10 in each group), the intervention group and two comparison groups. The districts in the intervention group or RBF group received performance-based financial incentives and additional Emergency Obstetric Neonatal Care (EmONC) equipment. In the first comparison group or input financing group (C1), districts were provided with input-based financial grants, and EmONC equipment as in the intervention group. Financial incentives in the intervention (RBF) group were disbursed directly to health facilities while in the first comparison group the financial grants were disbursed to health facilities through the districts. In the second comparison group or pure comparison group (C2), nothing was provided.

FINDINGS

Health coverage. Results were comparable in the RBF and C1 groups. i.e. both the RBF and C1 groups contributed to some important health gains as compared to the C2 group. For instance, institutional deliveries and skilled birth attendance increased substantially in the RBF districts relative to the C2 group, but rose by even more margins in the input financing group relative to the C2 group. On the other hand, first antenatal care (ANC) visits were earlier by two weeks in the RBF districts as compared to the two comparison groups. In addition, while full vaccination coverage declined in the two comparison groups, it remained constant or slightly higher in RBF districts, suggesting that the RBF program was protective with respect to some measures of immunization coverage. In contrast, the rate of change for any postnatal care was better in the input financing group as compared to RBF districts.

Quality of health care. The evaluation also found some improvements in structural quality in the RBF group and mixed results on process quality. On perceived quality, patient counseling appeared to work better in the RBF group than the two comparison groups. In addition, health workers in RBF facilities spent significantly more time with their patients during consultations as compared to both other groups.

Human Resources for Health. Job satisfaction and retention of health workers increased in both the RBF and input financing groups but the gains were higher in the RBF group. However, both the RBF and input financing groups had no impact on the motivation of health workers. Nonetheless, patients trusted health workers working at RBF facilities more than those in the two comparison groups for both maternal and child health services.

Functionality of the health system. The RBF was purposefully implemented using a contracted-in model to increase institutional sustainability. Through direct disbursement of RBF funds to health facilities, the RBF arguably enhanced fiscal decentralization; and financial autonomy. Fundamentally, the disbursement mechanism of funds to RBF facilities was more efficient than the disbursement mechanism in the input financing group i.e. budget execution and/or burn rate of funds in the RBF districts was better than the input financing districts.  RBF districts also performed better than the two controls on health facility governance and managerial autonomy.  

On the negative side, use of the fee-for-service provider payment mechanism was disadvantageous to the RBF i.e. a target or coverage-based provider payment mechanism could perhaps have produced even greater results given that the health system already had relatively high coverage in some indicators when the RBF was introduced. Furthermore, the RBF funds which were disbursed to the RBF districts ended up playing a substitutional role instead of being additional (as intended).

Cost-Effectiveness. Compared to the two comparison groups, the RBF was highly cost-effective in terms of lives saved or QALYs gained relative to Zambia’s per capita GDP ($1,759 in 2013). However these gains were supplied at a higher unit cost. The input financing group was also highly cost-effective in comparison to the C2 group, with less health gains but at a lower unit cost. Notwithstanding the above, cost-effectiveness analysis does not explicitly account for health system strengthening investments – certain dimensions of effectiveness with regards to RBF take time to manifest and may have been missed by the analysis. Further, without an accompanying budget analysis, it is difficult to know whether cost-effective interventions are financially sustainable.


Download17.61 MB