Send us your news!
We'd love to hear from you! If you have news of goings-on in the world of RBF Health, please drop us a line and let us know the news and we'll see about getting it online.
RBF Health RSS
Subscribe to RBF Health - the latest news from around the world of RBF Health, delivered directly into your RSS feedreader.
RBF Health News Feed
RBF Newsletter Subscription
We publish a bulletin and newsletter highlighting new information, programs, and announcements concerning RBF.
Add me to the email list
Send to friend{Below we present three sections of the case study paper. Please download the document to see the entire paper.}
by Petra Vergeer (World Bank), Deirdre Rogers (JSI), Richard Brennan (JSI), Shiril Sarcar(JSI), Photo by Curt Carnemark
Background
Ministries of Health and development agencies in a number of post-conflict countries have adopted Performance Based Contracting (PBC). This approach, whereby government contracts Non Government Organizations (NGO) to deliver health services, was initially tested in Cambodia[i],[ii] and was subsequently implemented in fragile settings, including Haiti[iii] and Afghanistan[iv],[v]. Liberia has recently adopted a PBC model, similar to that of Cambodia[1], which contracts NGOs to manage and support Ministry of Health and Social Welfare (MOHSW) health facilities but with the additional aim of building the MOHSW capacity in the process.
PBC rewards the contracted party upon achievement or progress towards pre-agreed targets with either financial or non-financial (e.g., attending training) incentives. When performance has not improved, the contracted party may be sanctioned; for example, the contract may not be extended or a portion of the fee (i.e., payment and/or bonus as incentive) may be withheld.[vi] Intended results, such as improvements in health worker performance and subsequently health outcomes, are to be attained in PBC through the use of incentives to motivate and/or change the behavior of key actors (i.e., the NGO and/or the service provider). To bring about change it is essential that incentives be directed to the service provider and/or beneficiary level.[vii]This trickling down of incentives to providers was found to be especially important in the case of management contracting in Cambodia.[viii]
As PBC focuses on results, the identification and selection of indicators is vital. Indicators can be defined as “a set of key measures that help you define and track progress towards your objectives”[ix]. Using indicators for monitoring projects or health systems development is not new; attaching a reward and/or sanction to their attainment, however, is new. Identifying performance indicators for PBC and setting targets, which will form the basis of pay, requires serious and sometimes protracted deliberations. This case study describes the rationale and the process followed in selecting indicators for the PBC of NGOs through the Rebuilding Basic Health Services (RBHS) project in Liberia. As such, the report endeavors to make a contribution to lessons learned about PBC design and implementation. The process of identifying indicators for PBC is not only a technical process but also a political one due to competing priorities of stakeholders, and time constraints.
Addressing multiple objectives through PBC in Liberia
Fourteen years of conflict in Liberia, which ended in 2003, resulted in about 75% of the health services being provided by relief organizations as the government health system was no longer fully functional.[x] Some important health outcomes also worsened, with a maternal mortality ratio that is now amongst the highest in the world at 994 deaths per 100,000 births.[xi] To help ameliorate these conditions, NGOs are needed to continue to support facilities and improve the availability and quality of the MOHSW’s Basic Package of Health Services (BPHS). In early 2007, the MOHSW requested donors to continue funding NGOs, as their imminent departure would have created major gaps because of insufficient capacity at the Ministry.[xii]
The MOHSW’s 2008 ‘Policy on Contracting’[xiii] aims to maintain and improve access to, and quality of, the MOHSW approved BPHS package. Moreover, it aims to “leverage partner capacity to prepare the County Health Teams to resume management of health facilities and the workforce”[xiv]. The MOHSW views the contracting of NGOs as a means to facilitate the transition from relief to development by improving the management of, and collaboration with, the County Health Teams (CHTs) responsible for managing health services in their respective Counties[2]. The capacity of CHTs is to be developed, in part, through the PBCs with the NGOs. The performance based contracts with NGOs, and the selected performance indicators contained therein, therefore needed to reflect this multiplicity of objectives.
Rebuilding Basic Health Services (RBHS) is a 5-year project funded by United States Agency for International Development (USAID) and implemented by JSI Research & Training Institute, Inc. (JSI) and its partners JHPIEGO, the John Hopkins University Center for Community Programs (CCP), and Management Sciences for Health (MSH). A major component of RBHS is to support the MOHSW in increasing access to quality basic health services and strengthening the decentralized management of the health system through PBC of NGOs in seven counties. In February-March 2009 an RBHS Request for Proposals (RFP)[xv] was developed to contract NGOs to provide management support to 105 health facilities. These contracts are performance based. The primary role of the contracted NGOs is to contribute to improving BPHS access through the following three objectives:
1. Ensuring delivery of evidence-based BPHS services
2. Expansion of selected BPHS services to communities
3. Strengthening the capacity of County Health Teams to manage a decentralized health system
RBHS introduction of performance-based contracting and the ongoing involvement of the MOHSW (particularly the MOHSW Performance Based Financing working group) in this process are intended to contribute to lessons learned for the MOHSW’s own performance based contracts. The MOHSW, funded through a Pool Fund (with donors such as the Department For International Development (DFID) and Irish Aid), intended to follow a similar PBC approach in October 2009 for NGOs to provide continued support to 46 additional health facilities, and released their own RFP in July 2009.
Lessons learnt on selecting performance indicators in Liberia
Many different performance indicators were considered for measuring progress toward achievement of the relevant objectives of the Scope of Work of the NGOs. The selection of appropriate performance indicators was particularly difficult in Liberia as the NGOs are not responsible for delivering the health services but are contracted to provide management support to MOHSW health facilities. Consequently, not everything is under the control of the NGOs, such as the management (e.g., hiring and firing) of MOHSW employed staff or the provision of certain supplies such as Insecticide Treated Nets (ITNs), and therefore the ability to influence certain changes at community and/or provider level may be more complicated. As a result, some indicators were not suitable to be linked to performance incentives as they could not be sufficiently influenced by the NGOs. For that reason, the Scope of Work was developed concurrently with identification of performance indicators and due consideration was given whether NGOs were in a position to influence the indicators. Additional measures, such as agreements between the NGOs and CHTs describing respective roles and responsibilities, will need to be implemented to achieve the improved health system performance. A key lesson learned is that utilizing performance based contracting in the case of management contracting is especially challenging. Selection of performance indicators requires substantial consideration to ensure they will be feasible to achieve.
In coming to the final determination of indicators, several were not considered appropriate from the start. Impact level indicators related to mortality (infant, under-five and maternal), fertility, HIV/AIDS prevalence and cure rate for TB were not considered appropriate due to problems of attribution and difficulties in measuring them at annual intervals. The availability of baseline data and the feasibility of accurately measuring the indicator plus capacity to measure them on an annual basis were important criteria for indicator selection. It was realized at the outset that capacity to monitor and verify the indicators was still limited. Hence, a phased approach in the types of indicators selected would be more feasible in Liberia. The data collection methods employed by the projectduring the first year will therefore focus on the facility level, whereas in future years it is anticipated that RBHS will also rely on household surveys to better assess use, coverage and perceived quality of care. It was found that MDG indicators were not necessarily appropriate as performance indicators for the project at this stage. Consideration of sustainability and ensuring sufficient budget allocation for such verification activities were also found to be important issues. It was vital to begin with a limited number of indicators feasible to collect and relevant to the objectives while building the capacity in data collection to allow for evolution of performance indicators over the life of the project.
The performance indicators selected for the PBC in Liberia RBHS were not based only on technical considerations. The indicators also had to reflect the projects objectives and the priorities of both the MOHSW and the donor. To align with these priorities and to harmonize monitoring systems, the suitability of many different indicators as performance indicators was considered. One of the key challenges was found to be juggling the different, and sometimes competing, interests of the different stakeholders when identifying suitable performance indicators.
The project was under significant time constraints to release the RFP. However, consideration of the appropriateness of indicators takes time and requires the involvement of multiple stakeholders to address the different priorities, especially since in the Liberia case this was the first time the PBC approach was being introduced. Moreover, the suitability of selected performance indicators can only be tested once operationalised. Another lesson learned is that sufficient time must be allocated for indicator selection and to ensure an inclusive approach whereby stakeholders at all levels are involved jointly. In addition, piloting of performance indicators is recommended to establish the feasibility and efficiency linked to data collection and aggregation, prior to implementation on a larger scale.
Finally, it was acknowledged that NGOs and other stakeholders (like CHT and health facility staff) needed to enhance their understanding of their role in PBC, the rationale of the pay for performance concept and the possible results. The ambitious performance targets proposed by the NGOs are a case in point. They highlighted the need for appropriate capacity building to be carried out to make PBC work so that it will not be viewed as a punitive system of rewards and punishments but an overall approach to improve performance on health outcomes.
[1] The two models tested were (i) whereby NGOs are responsible for management as well as the delivery of services- and (ii) whereby NGOs were responsible for the management of the service delivery while inputs (including staff and supplies) were provided by the MOH. In this case study, the latter form of contracting will be referred to as ‘management contracting’.
[2] There are 15 counties in Liberia, which are subdivided into districts.
[i] S Keller and JB Schwartz (2001) Final evaluation report: Contracting for Health Services Pilot Project (CHSPP), Asian Development Bank - Cambodia
[ii] B Schwartz and I Bhushan ‘Cambodia: Using Contracting to Reduce Inequity in Primary Health Care Delivery (p.137-161) in: Reaching the poor with health, nutrition and population services: what works, what doesn’t and why (2005), Edited by D R Gwatkin, A Wagstaff, A Yazbeck, The World Bank
[iii] R Eichler, P Auxila, U Antoine, B Desmangles (2007) ‘CGD Working Paper #121: Performance-Based Incentives for Health: Six Years of Results from Supply-Side Programs in Haiti’- Center for Global development
[iv] E Sondorp (2006) ‘Case-study: A time-series analysis of health service delivery in Afghanistan’ in the 2004 DFID report on Service Delivery in Difficult Environments, DFID Health Systems Resource Centre
[v] B Sabri, S Siddiqi, AM Ahmed, FK Kakar and J Perrot (2007) ‘Towards sustainable delivery of health services in Afghanistan: options for the future’ in Bulletin of the World Health organization, 85(9)
[vi] B Loevinsohn (2008) Performance-Based Contracting for Health Services in Developing Countries - A Toolkit, The World Bank, p.1
[vii] A Batson (2008) Presentation ‘Results Based Financing: Why? What? How?’, presented in workshop on Results Based Financing, June 2008 in Rwanda
[viii] Op Cit., Keller JB Schwartz (2001) Final evaluation report: CHSPP
[ix] J Naimoli, presentation during the ‘Asia Pay for Performance Workshop’, January 2009 in the Philippines
[x] P Vergeer and J Hughes (2008) ‘Liberia Case Study’ pp40-55 in Post-conflict health sectors: the myth and reality of transitional funding gaps, by A Canavan, P Vergeer, and O Bornemisza. Commissioned by Health and Fragile State Network and completed in collaboration with The Royal Tropical Institute
[xi] Liberia Institute of Statistics and Geo-Information Services (LISGIS) [Liberia], Ministry of Health and Social Welfare [Liberia], National AIDS Control Program [Liberia], and Macro International Inc. (2008), Liberia Demographic and Health Survey 2007, Monrovia, Liberia: Liberia Institute of Statistics and Geo-Information Services (LISGIS) and Macro International Inc.
[xii] Op Cit., P Vergeer and J Hughes (2008) ‘Liberia Case Study’ pp40-55 in Post-conflict health sectors: the myth and reality of transitional funding gaps
[xiii] Ministry of Health and Social Welfare (2008), National Health Policy on Contracting 2008-2011, Monrovia, Liberia
[xiv] Ministry of Health and Social Welfare (2008), Summary of Policy on Contracting 2008-2011, Monrovia, Liberia
[xv] JSI Research and Training Institute, Inc., Request for Proposals (RFP) Number RBHS-RFP-36514-01, “Rebuilding Basic Health Services in Liberia”, Issuance Date: March 12, 2009
