Modified from a presentation given at the Health Systems Research symposium in Liverpool, Oct 2018
Results-Based Financing (RBF) has demonstrated an ability to improve the coverage and quality of healthcare services in countries including Argentina, Cameroon, Democratic Republic of Congo, Nigeria, Rwanda, and Zimbabwe . Yet there is a residual concern: If RBF incentivizes some outcomes, won’t it crowd out others?
With a handful of HRITF impact evaluations now available, we can start to answer this question both from a theoretical and an empirical perspective.
A graphical analysis is useful to think through what may be happening when some outcomes are incentivized…and others are not. Borrowing from the basic economics model of the Production Possibility Frontier (PPF), there is a theoretical limit at which, for a given level of resources, a health facility (or hospital, or district, or health system) cannot produce more of one output without giving up producing some of another.
In a context where an outcome, such as institutional deliveries (represented along the horizontal axis), is incentivized and another outcome, such as the provision of oral rehydration solution (ORS) for diarrheic children under five (represented on the vertical axis), is not incentivized, then production that is at the boundary of the PPF curve is efficient. So, if production is already at Point A, then no more institutional deliveries (i.e. moving to C) can be performed without providing fewer ORS for diarrheic children under five. The same is true if we want to move in the opposite direction: If we start at the frontier, then we cannot provide more ORS (e.g., moving from A to B) without performing fewer institutional deliveries.
So if we are incentivizing the outcome along the X axis (say the number of antenatal care visits or institutional deliveries) but not the outcome represented along the Y axis (perhaps iron tables or time with patients), and if production starts at A, then it is efficient. That is, we cannot produce any more antenatal care (ANC) visits—moving to point C—without having fewer visits in which oral rehydration solution is given to diarrheic children under five (U5). The reverse is also true: we cannot give out more ORS—moving along the curve from A to B—without producing fewer ANC visits.
However, if current production is not fully efficient (e.g., starting at point D), then increases to institutional deliveries would be possible without affecting the provision of ORS for diarrheic children under five (moving from D to C). Likewise, a clinic that is not (yet) fully efficient could give more ORS for diarrheic children under five without affecting the number of institutional deliveries performed (e.g., moving from D to B). It would even be possible to get more of both ORS visits and institutional deliveries at the same time, moving from D to A.
This analysis and dynamic hold for both extra-marginal effects (such as coverage of population and type of services delivered) and infra-marginal effects (such as quality of care). Only if a health center or system is already fully efficient does it follow that increasing the quantity (or quality) of one service must necessarily reduce quality (or quantity) of that service or the quantity or quality of another service. Ultimately, then, the issue of crowding out services in RBF is an empirical question.
The evaluation portfolio of the Health Results Innovation Trust Fund at the World Bank is a great place to start to explore that question. The HRITF was established in large part to ascertain whether RBF can improve maternal and child health outcomes and is evaluating the effects of RBF in 30 countries. At least rigorous, causal estimation of the effects of RBF is expected from more than 2/3 of those countries through experimental or quasi-experimental impact evaluations.
To explore the question of potentially harmful unintended consequences of RBF, we use data from the completed impact evaluations conducted in Afghanistan and Rwanda. We also report on findings from previous evaluations in four additional HRITF countries.
The Afghanistan pay-for-performance project provided financial incentivizes to health facilities to increase service coverage and quality. Coverage incentives were provided through quarterly payments based on the number of ANC visits, postnatal care visits, births attended by a skilled attendant, Pentavalent3 vaccinations, tuberculosis cases detected, and contraceptive prevalence rates. Incentives for equity and for quality of care were disbursed based on an annual assessment of health facilities; quality was assessed using a balanced scorecard.
In Rwanda, RBF was used to incentivize community health workers (CHWs) to increase the utilization of targeted child health services, including growth monitoring, ANC, institutional deliveries and family planning consultations for new and regular users. The quality component was limited to the timely completion of quarterly data reports by the CHWs.
Interventions in Afghanistan and Rwanda were evaluated using randomized designs and their effects are summarized here and here, respectively. In the analysis below, we rely on that same identification strategy and use household data from treatment and control catchment areas to present Intent to Treat (ITT) effects of RBF on non-incentivized coverage (extra-marginal) and quality (infra-marginal) outcomes. In addition to the population-level effects, we also explore the equity dimension of unintended consequences by comparing the treatment effects of those above the median on an asset index to those below the median.
To assess the effects of RBF on unincentivized coverage outcomes in Afghanistan and Rwanda, we exploited the randomized design and evaluated unincentivized outcomes for which data had been collected. Because the surveys were not designed for this purpose, there were few available outcomes to test. The coverage outcomes susceptible to crowding out included tetanus vaccine during pregnancy, iron tablets during pregnancy, and at-home ORS administration for children under five. Afghanistan also included data on seeking treatment for any illness and for diarrhea and fever/cough separately and specifically, which we use as a proxy for coverage of those unincentivized outcomes.
Apart from a negative effect on tetanus, the RBF effects are dominated by null results in both countries—that is, there is not a statistically significant difference between the treatment and comparison groups for these unincentivized outcomes. Interestingly, it does not appear that the wealthy are benefitting differently from the poor in these unincentivized outcomes
Changes to unincentivized quality of care were measured through patient satisfaction surveys and observation in health facilities and of staff. Because quality was explicitly incentivized in Afghanistan, none of the quality indictors available in that country’s survey are used in this analysis. The results from Rwanda show a pattern of ubiquitous null results in unincentivized measures of quality, meaning that we cannot say that the RBF for community health workers induced changes in quality of care at facilities. For example, even though the RBF successfully incentivized CHWs to increase patient coverage, our results indicate that we cannot conclude the influx of those visits led to reductions in patient-perceived quality.
Of course, we are limited by the data available. The effect on incentivized outcomes is likely to be influenced by whether the incentivized and the non-incentivized outcome have competing or complementary inputs in their production functions. It would be useful to test unincentivized coverage outcomes that are unaligned with the MCH outcomes common to RBF.
Still, beyond the results from these two countries, effects on unincentivized outcomes have also been measured in four of the other five completed HRITF impact evaluations. For the most part, they follow a similar pattern. For instance, in Cameroon there were no negative effects on quality of ANC nor on the completeness of advice given during consultations for children under 5. Similarly, the evaluation of the pilot conducted in Haut Katanga in DRC indicated that the quality of care did not change despite increases in targeted preventive services. Results from Argentina were somewhat mixed: although there were notable gains in reducing low birthweight for the beneficiary population, there was not a statistically significant difference on birth outcomes for non-beneficiary populations in treatment facilities, and there was a small negative effect on prenatal care utilization for that group. Yet Zimbabwe saw positive effects on quality and quantity of non-incentivized services (ARI, diarrhea, skin disease and diabetes) without measurable changes in process quality.
As we saw in the graphical analysis, it is not necessarily the case that incentivizing more of one outcome will reduce output in another. That would only happen if a health facility were fully efficient. Of course, one of the underlying assumptions of RBF is that clinics are not yet operating at full efficiency, and that given additional autonomy and enhanced personnel incentives, the facility or system could produce more. And indeed, evidence from six rigorous impact evaluations seem to bear out that assumption.
Given the causal evidence available thus far—and there will be more in the coming years from the HRITF portfolio—we can surmise that although negative effects are possible and should be monitored, they are certainly not the norm. For now, it seems that with RBF, you largely get what you pay for—and you don’t get a result you didn’t pay for. In other words, there are not likely to be unintended effects—positive or negative.
 Note that in cases where RBF increases the total resource envelope rather than merely incentivizing within the current envelope, the intercepts would be increased and the PPF would be expanded and potentially change shape somewhat; that would not change the main results of the analysis here because even allocations that were previously efficient would no longer be efficient under an expanded budget envelope).