Some Days Are Better Than Others: Lessons Learned From Uganda’s First Results-Based Financing Pilot

In 2003, the Government of Uganda launched a pilot performance-based contracting scheme designed to improve the quality of and access to health services at private not-for-profit (PNFP) health facilities. In addition to performance incentives, facilities were given freedom to decide how to allocate resources. The latter innovation had a discernible positive impact on health facility performance, but the former—the incentives—did not. This brief explores the design and implementation issues behind the failure of the incentives and shows that while incentives matter, the success of RBF programs is not inevitable. They require significant investment (of time AND money) and careful design and implementation. The good news, though, is that enormous improvements can be had for free—by granting facilities the ability to choose how to spend their money.
 
 
Article: Lindsay Morgan [1]
 
The Decision to Try RBF in Uganda
In some ways, Uganda was a leader in health in Africa. The country had successfully reduced HIV prevalence from 18 percent in the early 1990s to 6 percent in 2003, and made many attempts at reforming the health system, including through decentralization of public service provision. In 2000, a basic minimum package of health services was instituted, and in 2001, user fees were abolished in public facilities.
The Ministry of Health also supported private not-for-profit (PNFP) facilities. Coordinated by medical bureaus of the country’s various religious denominations, in many areas, PNFPs were (and are) the only accessible providers for the poor, accounting for as much as half of all health services provided in the country, and an even larger share of certain services, such as childhood immunization.[2] Grants were provided to PNFPs (under the terms of a memorandum of understanding (MOU)) for the provision of specific services and the use of the grants was restricted to the purchase of specific inputs. Since 1997, the government periodically has increased public subsidies to PNFP providers with the aim of expanding health care in under-served parts of the country.
 
But despite attempts at reform, and some successes, serious health problems persisted. Infant and under-five mortality remained high at 88 and 152 per 1,000 live births, respectively, meaning that one in every seven Ugandan babies was not surviving to its fifth birthday. Infant mortality had actually increased by almost 10 percent from 1995 to 2000. [3] There had been little improvement in children’s nutritional status since 1995; the total fertility rate remained high at 6.9; life expectancy at birth was low, at 42.1 years in 2000[4]; and only about 38 percent of births were attended by trained health workers.[5]
Inequity in health care and outcomes was also a huge problem. Infant mortality among the poorest quintile, for example, was nearly double that for the richest. Children from better off families had a lower incidence of fever, acute respiratory infection and diarrhea compared to children from the poorest families. Northern and Western regions, which are poorer, had worse health outcomes. [6]
 
In April 2003, the Health Sector Strategic Plan mid-term review recommended that a study be conducted to gather evidence about a contractual arrangement that would improve access to the minimum health care package for the poor and most vulnerable, and, at the same time, increase the value of money invested in health. Two months later, the Ugandan Ministry of Health and the World Bank, with funding from the Canadian International Development Agency (CIDA), the U.S. Agency for International Development (USAID), and the Belgian Technical Cooperation, launched a results-based financing (RBF) pilot program to investigate whether PNFP providers would respond positively to a performance-based contract for health service provision.
 
Pilot Program Design and Results
 
The study had a prospective quasi-experimental design, with three arms: two intervention groups (study groups B and C) and one control group (A). PNFP facilities were randomly assigned to one of the three arms of the study. Only PNFP facilities were included in the two experimental arms; the control group was a mixture of public, private-for-profit, and PNFP facilities. The study included a total of 118 facilities (68 PNFPs) from five districts.
 
Control group A was subject to pre-existing financial arrangements. Treatment group B continued to receive the base grant from the government but was given freedom on how to spend it. And the main experimental arm, study group C, was also given freedom on how to spend the grant, but was also awarded bonuses if self-selected output targets were achieved. (All of the PNFP facilities in the study had been in the first wave of decentralization in the country, in which districts had been granted the authority to negotiate and manage the memoranda of understanding with the PNFPs.)
 
All providers (A, B, and C) could choose three out of six pre-established performance targets. For C, by meeting all the performance targets, facilities could obtain a maximum bonus of 11 percent of the block grant. Repeated surveys were conducted at the 118 health facilities. Staff surveys and exit polls at each facility, and interviews with a sample of households in the catchment areas of each facility were also conducted.
 
After 2½ years and three survey rounds, the study found no discernable impact of bonuses on the provision of health services by the PNFP providers (group C). Twenty-two out of 23 facilities receiving performance bonuses did reach at least one performance target, and 12 reached all three, but service levels at group B institutions similarly improved. If anything, facilities in the bonus group performed slightly worse than the facilities receiving only the untied base grant and about as well as the facilities in the control group.
 
Table 1. Design of Bonus Scheme Targets Goal Increase total outpatient visits 10 percent Increase treatment of malaria among children 10 percent Increase number of children immunized 10 percent Increase number of antenatal visits 10 percent Increase number of attended births 5 percent Increase uptake of modern family planning methods 5 percent Rewards 1% of base grant for each target met in each 6-month period 1% of base grant for each target met by end of year 1% if two targets met by end of year 1% if three targets met by end of year Total possible annual bonus payments = 11 percent (3 + 3 + 3 + 1 + 1)
 
In addition, data from the exit polls showed that the perceived availability of medicines, attitude of staff, and the prices charged by the facility worsened in the view of the respondents, more among the bonus group than among the control group, following the implementation of the bonus scheme. The wealth index of clients treated by the PNFP bonus group also increased relative to that of the PNFPs in the control group. This suggested that, rather than increasing their services to poorer segments of the population, the PNFP facilities in the bonus group were caring for clients who were wealthier relative to both the clients served by the PNFP control group and to the overall population.[7]
But the pilot had one positive and crucial outcome: granting autonomy in financial decision making appears to have had a positive impact on health service provision. “In my discussions with facility directors,” says Mattias Lundberg, Senior Economist with the Human Development Network at the World Bank, who led the project from Washington, “many of them said that they didn’t need more money, they simply needed to be able to spend the base grant in the way they saw fit, rather than according to health ministry mandates. The empirical results strongly confirmed the views of the facility directors.” And this practice, of granting autonomy in financial decision-making, has been retained by the Ministry of Health.
 
Why Didn’t the Performance Incentive Work?
 
Why did the bonuses fail to have an impact on performance while in other RBF schemes—Haiti, Afghanistan, Rwanda—they did? There are many possible explanations. They can be broken down into two broad (and familiar) categories: design and implementation.
 
Design  
First, the incentives may have been too small, and the bonus structure too complex. The bonuses were computed based on non-wage recurrent costs, which are relatively small—less than 20 percent of the total costs of running the facility. The maximum performance bonus a facility could receive was 11 percent of its base grant, or roughly between 5 to 7 percent of its total operating revenue. On average most health facilities received bonus payments of less than US$1,000 per year.
 
In other RBF schemes, bonuses have been either larger or similar but delivered through other means (performance grants managed through NGOs instead of through a centralized bureaucracy). In Afghanistan, NGOs contracted by three donors (European Union, USAID, and World Bank) to deliver a basic package of health services at primary care facility level could earn a bonus worth 10 percent of the World Bank contract value if they reached or exceeded targets outlined in the contract. In Haiti, NGOs could earn the final 5 percent of their fixed quarterly payment plus an additional 5 percent if all performance targets were achieved. In 2005, this was increased, from 10 to 12 percent. And in Rwanda, facilities could increase their budgets by a whopping 25 percent.
 
“The incentives provided probably need to be larger than the maximum 11 percent of the base grant that we provided,” says Mattias Lundberg. “[And] the bonus scheme…was most likely too complex to understand and implement. Had the bonus amount been larger, it might have been worth the effort to figure out these rules. Conversely, had the rules been simpler, the 11 percent might just have been worthwhile.” George Pariyo, the principal investigator in Kampala and Professor at Makerere School of Public Health, agrees: “The bonus payments were too little…to make an impact on their behavior or to make changes needed to improve services.”
 
Second, competing events occurred. Peter Okwero, a World Bank Senior Health Specialist based in Kampala, cites, for example, “the increase in salaries of government workers, which precipitated movement of PNFP workers to government employment. The bonus payments could not compensate for the salary variation.” Following the abolition of user fees in public facilities in 2000, the salaries of government health workers increased incrementally by 14-63 percent across different cadres of workers between 2001 and 2002. Management of health worker pay roll was also greatly improved, to ensure better human resources management.[8] Says Richard Alia, the national project coordinator on the Uganda RBF pilot: “there was an exodus of health workers from the PNFP/Mission sector to the public sector.”
 
The problem of poor information management systems at the facilities was also neglected. “Informal discussions with facility directors suggested that many—especially at the lower levels—were not used to keeping records of any kind, and certainly not for long periods of time.”[9] These details needed to be addressed before the pilot was launched, but they were not.
 
There was also the question of who received the bonuses—incentives were paid to facilities, not to individual providers. Some facilities distributed bonuses to staff, some organized parties for health staff but others used the funds for facility improvements (e.g., one health unit used the bonus to purchase new front gates for the compound). Says Okwero: “[T]his issue has remained controversial, especially among the PNFPs—who gets, who decides, etc. This was never discussed at the beginning. Some PNFPs had misgivings on the concept of bonuses, which in fact were never adequately taken up.”
 
Finally, the duration of the experiment (two years) may have been too short. The treatment facilities seemed to do better over time, suggesting that if the experiment had continued, facilities might have improved. Says Pariyo: “The whole thing was cut prematurely just when facilities were beginning to understand and show interest in the process. A longer implementation (given that things were slow to start) would have helped.”
 
In Rwanda, bonuses were larger, and paid regularly (once per month). Up to 80 percent of the earnings could be converted into health worker bonuses, and control facilities’ budgets were increased to the equivalent of the average RBF payment in phase one to compensate them for participating in the study. And whereas Uganda awarded bonuses based on percentage increases, Rwanda also used a fee-for-service mechanism. For example, about $5 was rewarded for every attended delivery, and $1.80 for a fully vaccinated child. This made the link between the bonus and the action more direct.
 
Implementation  
While design issues were an important factor contributing to the failure of the incentive scheme, and pre-existing constraints at the facilities probably compounded the problem, these things happen to one degree or another with almost all RBF schemes. No pilot is designed perfectly from the start, and plenty of facilities fall short in one or more areas. RBF schemes can work regardless if programs are flexible, and managers are given room to learn and adjust as they go.
 
But the Uganda pilot did not work and the most important reason why is problems with implementation. Meager investment and bureaucratic delays in the beginning sent shock waves throughout the program. Good will was squandered; data collection and analysis were insufficient; management and supervision were inadequate. Health worker staff who didn’t fully understand the program became frustrated and demotivated when bonus payments were delayed. And poor communication between Kampala and Washington, along with scarce resources, destroyed any chance of correcting these problems. Let’s start with the money.
 
The search for money, and bureaucratic delays  
Initial funds of about $250,000 came from the Canadian International Development Agency (CIDA). Nearly a year was spent looking for another funder to pay the bonuses (the Government of Tanzania had not factored this expense in its budget), and eventually the U.S. Agency for International Development (USAID) came in with $50,000.
 
The bureaucratic delay caused by the search for resources left many stakeholders with a bad taste in their mouths. “The commitments from the partners, government, World Bank, etc., took too long in coming,” says Pariyo. “MOH was initially interested, then they seemed to lose interest and never came up with their side of the funding. The project was rescued by USAID providing some $50,000 for use in paying the bonus [but]...the whole implementation process dragged on too long, people lost interest….” Okwero agrees: “By the time the study was resurrected…there was not much interest anymore from the PNFPs especially with a new problem of having to address the loss of staff.”
 
“The bureaucratic delay…cost us an enormous amount of goodwill,” Lundberg admits, “so we began with less-than-complete commitment from the medical providers as well as the research team.”
 
Problems with data collection 
Inadequate funding spread like a stain over the project, first making proper data collection nearly impossible. Baseline data were collected late, in April-May 2004, for the period dating back to January 2003. This was due, according to Lundberg, “to contracting problems internal to the Bank and only secondarily to funding issues. We did the preparatory work, designing the experiment, etc., and then the Bank just couldn’t get around to contracting the research team.” Not enough was budgeted to pay a sufficient number of research assistants or to facilitate their supervision. Once the research team was contracted, there was not enough money to pay a sufficient number of research assistants or to facilitate their supervision. Alia, the consultant hired to manage the process, says: “[The] data collection processes were done hurriedly and staff collecting these data were few.” When some households and facilities were not forthcoming, or when researcher’s were unable to access private facilities’ financial records, they were simply skipped by overworked and under-supervised research assistants.
 
By the time of Alia’s April 2005 quarterly report, the collection and analysis of baseline data and data for first wave was still not complete. At the same time, researchers were facing the next round of data analysis, and beginning to be overwhelmed. Three months later he noted again, “funding for data collection is inadequate hence research assistants spend fewer days in the districts than originally planned.” He also complained again about not being able to make trips to the field to supervise, highlighting in bold: “There is a big problem here that needs to be rectified. Richard has not been facilitated to go to the field (i.e., to the participating districts/health units) hence monitoring performance of the districts and health units by remote control are not effective at all. The last time I ever went to the field was in February 2005 (during the last bonus payment workshops and signing of the new MOU Addenda). Even the new MOU Addenda signed, have not been collected.” He also mentioned that some in charge of health units were locking up facility records with impunity, in an act of “total negligence.”[10] The frustration in the report is palpable.
 
Management, supervision, coordination  
I asked Lundberg what he thinks is at the heart of the data issue: “Supervision, supervision, supervision. I suppose funding was the main constraint. We needed a full-time supervisor. Peter [Okwero] has a day job and was not available to do it. I didn’t have the money to travel frequently enough to Uganda to do it, and we couldn’t afford to hire someone.”
 
Okwero says: “It was planned from the start to have a supervision team comprising various skills given the nature of the pilot study. No funding was provided for this activity. I was meant to provide oversight but not the technical supervision, which unfortunately did not take place.”
 
The lack of funds for travel created a huge disconnect between Washington, DC and Kampala. For example, MOU Addenda were signed for FY2004-05, which allowed some facilities to be eligible, after wave one, to more than double their bonuses. It also allowed facilities to receive performance payments in wave two when they did not achieve their goal for any indicator during the wave two period.[11] Mattias Lundberg, back at World Bank headquarters, did not know until I informed him that this had ever happened. Wave one bonuses had also been doubled: “It was noticed that the performance was generally poor, hence the amount of money that each health unit was going to receive would not make significant impacts. As a result, this time round (and only this time) the amount received by each health unit was doubled.”[12] “Well,” Lundberg said in an interview, “here’s the first example of poor coordination: I had no idea that the MOU had been changed to permit this. And here’s the second: I had no idea that the wave one bonuses were doubled. I was not involved in or aware of either points. Clearly I should have been.”
 
Payment delays  
The first bonus payment was made in January-February 2005. However, because of problems encountered by the research teams responsible for data collection and analysis (i.e., “budgetary constraints,” according to the April 2005 quarterly report), by November 2004, the research team lacked productivity data for the first six months of 2004, which the initial design of the study called for. But in order not to further delay bonus payments “and frustrate or de-motivate the beneficiaries,” bonuses were awarded based on data from three-month periods (January-March 2003 and 2004). Even then, however, some data from health units were missing. Payments were processed anyway, with the assurance that missing data would be verified during round two. But a July 2005 report notes that constraints in wave two data collection made verification from wave one impossible.
 
A September 2004 status report states: “participating health units expressed continuing frustration with the amount and timeliness of the grants they receive, the lack of an increase for the new fiscal year, and the increasing differential in salaries between public and private health workers. These are issues that need to be streamlined.”
 
Payment delays were related in part to poor data. The firm contracted by USAID was naturally reluctant to release funds when there were discrepancies in the numbers. In correspondence to Alia, the contractor wrote: “I found several errors and have a number of questions and concerns before [the organization] will be able to approve payments.” She cited two large payments to two hospitals that “may raise some questions for the study team about the accountability of these funds.” She also cited a number of instances where the figures appeared to be particularly low or high. For example, Namuyenje had a baseline of zero children immunized; Rwibale increased malaria treatment by 797 percent. “What is the means for ensuring the accuracy of data,” she asked?
 
Health workers not incentivized
 Though providers stood to receive an incentive payment, they were probably disincentivized due to the lack of supervision, delayed payments when they did perform, and—going back to the very beginning—poor sensitization. A sensitization workshop was held in Kampala prior to the launch. Attended by district health managers and representatives from the participating PNFP facilities, the workshop touched on the poor coverage of health services, the inability of the poor to access services, and the concept of performance contracting. But, “[a]t the end of the workshop,” says Alia, “not every participant understood the concept of the bonus payments. Follow-up workshops were therefore conducted in the districts.”
 
Correspondence between Alia and Pariyo states: “Following the recent mission of Robert Taylor [a U.S. consultant hired by the World Bank to provide technical assistance] to Uganda, and meetings with major stakeholders, it was agreed by consensus that another round of sensitization workshops be conducted in the participating districts ASAP in order to restore the confidence and morale of the participants. It would be even more motivating if bonus payments could be made in due course.” Alia also suggested that Lundberg delay coming to Uganda in order for these workshops to occur, so that participants would be in a mind frame of “interest and hope” when he launched round two of the study.
 
The Rwanda Model  
Though the Rwanda scheme had already benefited from the experience of three pilots, it is useful to compare key differences between the programs. First, there was a significant investment of time and money up front both from the government of Rwanda and donors. The government spent about 4.6 percent of its Ministry of Health budget on the output payments; scaling up RBF to the national level costs the government and development partners an estimated $1.8 per capita per year.
 
Rwanda held one-week trainings in each district, national workshops were organized, and user and training manuals were created. The significant fiscal investment, coupled with extensive consultation in the design phase and repeated sensitization of health workers helped to create a sense of shared commitment and team spirit among stakeholders.
 
Rwanda also had a more robust system to collect and verify data. Multiple levels of data validation enhanced the reliability and utility of the performance data. And finally, communication and coordination among stakeholders were frequent. An “extended team” was created in which district-based technical assistants from eleven non-governmental agencies; bilateral donor agencies; and the Rwandan Ministry of Health were tightly coordinated through monthly meetings.
 
Conclusion  
The results of Uganda’s first RBF pilot were disappointing in the sense that the incentives failed to spur systematic improvements in the performance of health workers. Understanding the issues behind the failure is important; incentives themselves, though powerful, are not enough to improve health service provision or health outcomes. RBF requires significant investment of time and money, and careful implementation—especially supervision and coordination. But the Uganda pilot was successful in at least one important respect: it verified the benefits of local autonomy in financial decision-making and showed that “enormous improvements in performance can be had for free.”[13]
 
_______________________________
[1] Lindsay Morgan is a policy analyst and freelance writer based in Dar es Salaam.
[2] Mattias Lundberg, Tonia Marek, Peter Okwero, Contracting Health Services in Uganda, Unpublished Report, World Bank (2007), p. 7.
[3] Uganda Demographic and Health Survey 2000-2001.
[4] Ibid.
[5] Maternal Health Care in Rural Uganda: Leveraging Traditional and Modern Knowledge Systems, World Bank Brief, 233313, January 2002.
[6] Uganda Demographic and Health Survey 2000-2001.
[7] Mattias Lundberg, “Client Satisfaction and the Perceived Quality of Primary Health Services in Uganda,” in Are You Being Served? New Tools to Measure Service Delivery, Eds. Samia Amin, Markus Goldstein, Jishnu Das (World Bank, Washington, DC, 2008), p. 330.
[8] Juliet Nabyonga-Orem, Humphrey Karamagi, Lynn Atuyambe, et al, Maintaining quality of health services after abolition of user fees: A Uganda case study, BMC Health Services Research, 2008. 8. 102.
[10] Richard Alia, April 2005 quarterly report, p. 11.
[11] Personal correspondence between Susan Scribner, PHRplus Uganda, and Richard Alia, 19 August 2005.
[12] Richard Alia, April 2005 quarterly report, p. 3.
[13] Mattias Lundberg, personal communications, 12 February 2010.