Send us your news!
We'd love to hear from you! If you have news of goings-on in the world of RBF Health, please drop us a line and let us know the news and we'll see about getting it online.
RBF Health RSS
Subscribe to RBF Health - the latest news from around the world of RBF Health, delivered directly into your RSS feedreader.
RBF Health News Feed
RBF Newsletter Subscription
We publish a bulletin and newsletter highlighting new information, programs, and announcements concerning RBF.
Add me to the email list
Translation
Translate this page by selecting below:
Send to friendBy Caryn Bredenkamp, Nicolas de Borman, Patrick Mullen, Danic Ostiguy, Eric Sompwe, Waly Wane, Jacques Wangata
[Download the file to read the full paper]
THE PBF PILOT IN HAUT-KATANGA DISTRICT
Pilot program area
The pilot is embedded in the World Bank-financed Health Sector Rehabilitation and Support project (HSRSP) which involves contracting NGOs to provide support to health services in ten districts across different provinces. Included in this support are salary top-ups for health workers, of which a small portion is directly linked to performance indicators. The HSRSP provides the framework of the pilot, defining, among other things, the budget envelope, the purchaser (an international NGO with a strong local presence, but limited PBF experience), the time frame for implementation, procurement methods, the fact that drugs will be provided directly on an input basis, and the user fee policy.
The pilot is being implemented in Haut-Katanga, a district of around 1.26 million people in the province of Katanga in the south-eastern corner of the Democratic Republic of Congo. Haut-Katanga was chosen by the government and the World Bank team for multiple reasons: (i) the implementing NGO was believed to have good implementation capacity and an interest in PBF mechanisms, (ii) the provincial government and provincial health authorities of Katanga expressed interest in PBF mechanisms, and (iii) the district capital is more accessible than many other parts of the country, easing provision of external technical assistance and data collection for the impact evaluation.
Haut-Katanga is characterized by a low population density and great distances between health centers. The district covers 79,547 square kilometers and includes Le district de Haut-Katanga compte 7 Zones de Santé reparties en 110 Aires de Saneight health zones divided into 107 health areas. Du point de vue infrastructure le district du Haut-Katanga compte 4 Hôpitaux de références (HGR), 8 centres de santé de référence, 104 centres de santé, et 33 postes de s Among the 160 health facilities that are recognized as part of the government health system, there are five rural hospitals and three referral health centers that provide the entire complementary package of health services and, so, are considered to function similarly to hospitals. In addition, there are a further 152 facilities consisting of four referral health centers (which provide only part of the complementary package), 110 health centers and 38 health posts. Altogether, there are 85 public facilities, 31 faith-based facilities and 44 private facilities. The pilot focuses on the lower levels of care and includes most of these accredited facilities, but not the five Le District de Santé du Haut Katanga compte 149 structures de santé agrées, reconnues comme faisant partie de la carte sanitaire.4 sont des hôpitaux ou assimilés. Les hôpitaux sont exclus du projet pilote mais continueront à bénéficier d'un appui PARSS classique (Kasenga, Kilwa, Pweto, Sakahospitals or three larger referral health centers.
The district was severely affected by the civil war during the first part of this decade and the health system has been severely degraded by the decades of low or absent government funding. Although the World Bank project has recently channeled significant investment and resources into the system, it is still characterized by severely deficient infrastructure and equipment, insufficient and poorly-trained human resources and intermittent supplies of drugs and consumables. Most of these facilities are reached by poor roads and are often inaccessible during the rainy season. The Kasongo Kamulumbi health area in Pweto, for example, is particularly difficult to access. One needs all-terrain vehicles to reach the Luapula River, which is then crossed by motorized canoe. From there, Kasongo Kamulumbi is approximately 260 kilometers, accessible by trails only during the dry season by motorcycle or bicycle. Many of the health facilities in the zones of Pweto, Kasenga and Kilwa are also only accessible by motorized canoe or motorbike. Whether or not PBF can have an impact in such difficult conditions – which prevail over much of DRC – is an underlying question for the pilot program.
Overall Design
This pilot involves the provision of performance-based payments to health centers and referral centers using a fee-for-service[1] mechanism in the “intervention” group and a predictable monthly transfer in the “comparison” group. There are 48 health areas (including 75 health facilities in these administrative areas) in the intervention group and 48 health areas (including 77 health facilities) in the comparison group. The impact evaluation compares the results of a lump-sum monthly transfer to a transfer that is tied to the quantity of services provided. Thus, it is only in the intervention group that health workers can influence their incomes through the level of effort exerted. L'hypothèse de départ est que l'on observera une variation de la performance (quantité et qualité des services de santé) des formations sanitaires statistiquement plus importante dans le groupe intervention recevant une subvention liée à l'activité que dans le groupe contrôle recevant une subvention fixe. The hypothesis is that the performance (i.e. quantity and quality of health services delivered) of the health facilities in the intervention group that receives the performance-related payment will differ in a statistically different way from the performance of health facilities in the comparison group that receives the fixed payment.
To the extent possible, both groups of facilities are treated similarly in all other respects, such as the training provided to health workers, the quantity of drugs provided, the level of supervision etc. All facilities have a large degree of autonomy in the allocation of the payment received, both across expenditure categories (staff bonuses and operating expenses) and between staff. Finally, the total budget allocated to performance bonuses across health facilities in the intervention group is the same as the total budget allocated across all health facilities in the comparison group.
While not part of the pilot, in the sense that they are not randomized into any experimental group, the health zone teams/authorities also receive performance incentives financed by the HSRSP. These are in the form of salary bonuses that contain a small performance-related component. Staff in the five hospitals and in the three large referral health centers that are not part of the pilot also receive salary top-ups (also defined by the HSRSP), but these are related to staff rank, not performance.
The purchaser of services is an international NGO that is also responsible for provision of comprehensive support to primary health care services in Haut-Katanga under the HSRSP. The NGO manages the implementation of the pilot, participates in the technical verification of the results reported in health information system records, leads the calculation and processing of payments, and facilitates the process of community verification. Oversight responsibilities are shared by the provincial ministry of public health and the provincial health inspectorate. An informal technical committee is responsible for addressing design and implementation issues that may arise, while a formal steering committee validates major decisions.
The pilot employs two types of data verification. Facility-level or “technical” verification is carried out by NGO supervisors, together with the health zone administration, as part of the regular facility supervision. This is to verify that information reported by the facility corresponds with the information contained in the facility registers and also to monitor the quality of the care being provided. Random cLa supervision par l'ECZ doit conserver l'esprit de la ommunity-level verification is carried out by local community associations with the primary purpose of verifying the accuracy of the information reported in the facility registers.
The effect of the pilot will be evaluated through the implementation of a rigorous impact evaluation with an experimental design by which health areas are randomized into treatment and comparison groups. Quantitative data collection from households, facilities and patients during the baseline and follow-up surveys is complemented by qualitative data collection at various points during project implementation.
DISCUSSION OF CORE DESIGN ELEMENTS
(download the full paper to see this section)
IMPACT EVALUATION
The effect of the intervention will be assessed through a rigorous impact evaluation. Health areas (with their catchment populations) have been randomly assigned to intervention and comparison groups. Several health areas include more than one facility, so that there are 59 health facilities in the intervention group and 58 facilities in the comparison group (rather than 48 in each). Health facilities in the intervention group of health areas receive the performance-related payments described above. Health centers and referral health centers in the comparison group of health areas receive a monthly lump-sum transfer, designed to be equivalent to the average transfer received by facilities in the fee-for-service treatment group. Except for the faith-based and private facilities not supported by HSRSP but included in the pilot, the intention is to treat all facilities in both intervention and comparison groups similarly in all other respects, such as the training provided to health workers, the quantity of drugs provided to facilities, the level of supervision, etc. In reality, however, it is likely to prove impossible to completely equalize the level and type of resources (i.e. from government, HSRSP, and patient revenues) made available to the different facilities. This potential important confounding factor, along with other differences between facilities, is intended to be addressed by the randomized allocation of the intervention. In addition, the study will measure the resources available to each facility, as well as a wide range of other facility and catchment population characteristics, in order to enable potential control of such factors in multivariate modeling.
We expect to observe a better level of performance (as measured by the quantity of health services delivered) among health facilities in the treatment group than among the health facilities in the comparison group. We also expect to observe changes in the quality of services of provided. This change may be for the better (since facilities have an in incentive to improve quality in order to attract more patients) or it may be for the worse (since this pilot does not create explicit incentive for quality).
It is also worth noting that technical supervision and community verification activities will also be undertaken in the “comparison” health facilities. This should account for the possibility that it is improved performance-monitoring, rather than the financial incentive, that is driving any observed changes in performance.
Baseline and follow-up data collection is being carried out using health facility and household surveys. Interviews of health personnel and patients will be done in facilities, while information on community characteristics will be collected as part of the household survey. Data collection for the baseline study, carried out in partnership with the University of Lubumbashi, was completed in October-November 2009. The follow-up study is planned for the same season in October-November 2011.
At baseline, given the lack of a reliable list of health facilities in the district, the first step needed was to survey all of the health facilities that had been identified. Because many facilities identified turned out to be non-existent, not operational or inaccessible, only 87% of the health facilities chosen to be included in the program could be surveyed. Data were also collected from 26 facilities not included in the pilot, mainly private or faith-based. In each surveyed facility, all of the medical staff present at the time of the visit were interviewed as well as a maximum of ten patients (or the total number available if fewer than ten were present). For the community and household surveys, a list of communities in each health area was compiled on the basis of information from the health zone administration. Two communities were randomly selected, along with three others who were randomly selected as back-ups in case either of the first two could not be reached for any reason. Within each sampled community, every third or fifth (depending on the size of the community) household was sampled. It is intended to re-survey the same households for the follow-up in order to create a panel. The sampling strategy resulted in the collection of data from 152 health facilities, 522 health workers, 868 patients, and 1,060 households in 106 communities. GPS coordinates were recorded for surveyed health facilities, communities and other locations important to the communities. The geographic inaccessibility of many health facilities and communities presented severe challenges to the data collection team, who were also racing against the start of the rainy season which would have cut off many locations. Data collection was completed in 44 days.
WHAT DOES IT COST TO IMPLEMENT AND EVALUATE THE PILOT?
Overall support to health services in the district by the HSRSP, including infrastructure and equipment investments, drugs and consumables (including anti-malarial bednets), technical supervision and support (and incentives for health administrators and hospitals not included in the experiment) is approximately US$ 3.50 per capita annually. Added to this are the incentives to health facilities included in the intervention and comparison groups, equivalent to US$0.75 per capita per year. Compared to results-based financing interventions elsewhere in DRC and in Central Africa region, this is a small incentive. The performance-based payment allocated to health centers in the results-based financing schemes in the provinces of Kasai Oriental, Kasai Occidental, South Kivu and North Kivu range from about US$ 1 to US$ 1.50 per capita per year, with still higher figures in Rwanda and Burundi. Yet, this small incentive would still count for a large proportion of the total income of the facilities, and is expected to be sufficient to modify staff behavior. Implementation cost, including establishing systems, technical supervision and community verification, totals an additional US$ 0.22 per capita annually.
Additional cost, unique to the experimental pilot nature of the program, is external technical assistance to the design and implementation of the pilot and the impact evaluation, as well as data collection and analysis, totaling approximately US$0.50 per capita annually.
CONCLUSION
The design of the performance-based strategy was kept simple so that it could be feasibly implemented in the difficult conditions of DRC within the available project timeframe of 12-18 months. Consequently, the strategy lacks some of the more complex refinements that might be seen in other PBF interventions, such as equity bonuses and quality measures, which while desirable, take a longer time and are more difficult to implement successfully.
The relatively short timeframe of the pilot program may attenuate the impact of the PBF strategy that will be measurable by the impact evaluation. Nevertheless, through its design and implementation, the pilot has already contributed to the policy dialogue in DRC, since it reflects a performance-based financing model that could feasibly become part of the government’s long-term health financing policy. In particular, planned decentralization of responsibility for payment of health workers to the provinces may allow some provinces to move ahead with such innovations while others are more cautious. It is also designed to complement what is being learned from PBF initiatives financed by different donors in other parts of DRC. As well, while it took about a year to build consensus around a PBF design that, in addition to being technically sound, also satisfies key stakeholders, there is now considerable commitment to the pilot and interest in the results of the impact evaluation.
The governance structure of the pilot, in the form of the steering committee, has strong representation from the central and provincial health ministries, so that experience during design and implementation is regularly transmitted to policy-makers. The pilot’s design is flexible, with mechanisms that allow for regular review and adjustment in response to implementation experience and conditions.
Given the range of interests and issues that are at play, it is unlikely that the PBF strategy will alone provide the solution to the challenges facing the health sector in DRC. Like in other contexts, this innovation will be most effective if part of a wider set of reforms, most importantly relating to governance and accountability. Decentralization of resource allocation, particularly management autonomy at the health zone and facility levels, are reforms (as demonstrated by the experience so far of this pilot) that can be closely tied to a PBF strategy. There is also potential for improving accountability at the local level by involving communities and patients more closely in the PBF payments, just as they are tightly in control of the user fees that the performance-based payment is intended to replace.
[1] The intervention group is a “fee-for-service” scheme meaning that each unit of service that is provided, of those services specified in the contract, is remunerated by a particular fee. The total payment is equal to the product of the number of services delivered and the fee for each service, i.e.