Impact Evaluation: The Missing Link to Scaling-Up RBF in Zambia

Collins Chansa's picture
November 28, 2012

“It works.” “It is transformative.” “It’s a catalyst.” “It is a strategic tool which can be used to reform the entire health sector.” These are some of the several positive and encouraging statements coming from the ever-increasing body of evidence on Results Based Financing (RBF). In Zambia, some results from a World Bank-supported pre-pilot project in Katete district shows improvements in incentivized and non-incentivized indicators ranging from 6 percent to 54 percent. An external data verification exercise also shows improvements in data collection and accuracy, while an independent, qualitative review showed a wide range of innovations and increased motivation of staff at health facility level.

As the interest on RBF grows, the distinct task for all proponents and implementers is to continue gathering and showcasing attributable evidence on RBF. As the Zambian Health Minister, Dr. Joseph Kasonde comments, “Where are the results coming from? Is it results-based financing or finance-based results? So far the results we have seen are impressive, but we need to be certain that it is, indeed, RBF that is triggering the change. If it works, money will be found – no doubt about this.” Coming from a very high political figure, this statement suggests that there is strong political and management support for RBF in Zambia, but that more irrefutable evidence will be needed for the Zambian Government to fully scale-up RBF to all districts. As highlighted in the Zambia National Health Strategic Plan 2011-2015, the Zambian Government is committed to exploring, evaluating, and rolling-out RBF initiatives. Thus, the role of impact evaluation in influencing change and commitment to RBF should not be overlooked.

Zimbabwe RBF Project Implementation Team 2012

This then brings me to the current debate on the RBF Community of Practice. Bruno Meessen on 29th August 2012 wrote, “Let us not make the mistake to believe that conclusive evidence from the impact evaluations will naturally bring key actors to commit to RBF. The world is simply not that rational.” While to some extent I agree with this statement, I believe that with more definitive evidence we can convince far more people. As they say in Zambia, “Those who judge before they know the facts will learn to shed tears.” And in Uganda, “The best preparation for tomorrow is to make sure today’s work is superbly done.” As such, we have to use the results from the impact evaluation to accelerate our efforts to advocate for RBF. Mind you, a number of potential donors still have the “wait and see” attitude and as we try to engage and convert these donors, we have to have all the facts on the table.

In Zambia, there is a real opportunity to evaluate the impact of RBF. This will help to provide answers to some skeptics of RBF, who appeal for more case studies detailing field experiences, success factors, and the potential advantages and disadvantages of RBF. The impact evaluation design in Zambia is robust and realistic in its approach. Quantitative and qualitative information from health facilities and households has already been collected (at baseline), while process evaluation is being conducted as part of the implementation process to account for contextual factors and explain what changes actually made the difference. At the end of the project, more data will be collected and included in the analysis. With an intervention group of 10 districts, and two matching control groups of 20 districts, it will be possible to attribute changes, and to come up with conclusive results that will separate out the effects of the mere addition of financing and that of actually embarking on results-based financing.

Thus, it is Zambia’s expectation that the World Bank-supported RBF pilot project will provide evidence on the impact of RBF initiatives. The impact evaluation should respond to the extremely relevant question posed by the Minister of Health, namely that accepting or rejecting RBF as a modality for strengthening health systems and achieving results should not be based on personal preference or faith, but should be based on evidence rooted in rigorous evaluation.


Add comment

This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.