EVALUATION DESIGN

In 2012, Benin started a RBF pilot program in eight of its 34 health districts. In these health districts, all public, and not-for-profit health centers were contracted to participate in a Performance-Based Financing (PBF) project. The goal of this program was to provide financial incentives to health facilities and health workers based on performance. In addition, as part of the pilot program, an impact evaluation sought to determine whether linking financial incentives to improved performance, versus offering additional financial resources without conditions, improve health worker motivation, and thereby increase the quantity and the quality of health services delivered.

To evaluate the effect of the pilot program on health worker performance, including clinic productivity (e.g. absenteeism, average time per patient, and reduction of slack) and quality of care, an impact evaluation was embedded in the program. The evaluation used both quantitative and qualitative studies. The qualitative study sought to understand how the program was perceived by various stakeholders and to explain some of the findings of the quantitative study. The quantitative evaluation was a randomized control trial designed to distinguish between the effectiveness of managerial autonomy and financial incentives with or without conditions.

The randomized control trial was comprised of four treatment groups and a comparison group. In the first treatment arm (PBF intervention or T1), health facilities were offered financial incentives based on performance, and had managerial autonomy in using these funds. The second treatment arm (T2) acted as a control group for T1. The health facilities in the group T2 were offered additional financing without conditions, and were given managerial autonomy. In the third treatment arm (T3), health facilities obtained bonus payments based on performance but had no managerial autonomy over the use of these financial resources. The health facilities in the last treatment group (T4) received additional funds that were not linked to performance, and were given no autonomy in the management of the additional financial resources. Finally, the evaluation design also included a pure comparison group (C) of health facilities that did not participate in the pilot program and so continued doing business as usual. These health facilities were located outside of the eight pilot districts.

To date, a midline evaluation also using a mixed-method approach was conducted to explore the impact of the financial incentives on health worker performance, specifically with respect to the factors of quality of care, responsiveness towards patients, and the productivity or attendance levels of health workers. Since it was challenging to implement the “managerial autonomy” arms of the evaluation design, the midline survey focused only on the PBF treatment (T1: additional financing linked to performance) and control groups (T2: additional financial resources not linked to performance and C: pure comparison group).

FINDINGS

The results of the midline survey suggest that the PBF intervention in Benin improved some aspects of the performance of health workers. In general, the quantitative evaluation showed that there were some positive effects of the PBF bonus treatment on the quality of care provided by health workers. Specifically, when comparing health facilities in groups T1 and T2, there was an improvement in the thoroughness of physical examinations and history-taking for both antenatal care (ANC) and curative consultations. The same results also hold when comparing the health facilities in the group T1 to these in the pure control group (C), with an additional four minutes per patient, on average, spent on ANC services.  Moreover, in health facilities where bonuses were conditional on performance, patients reported a greater level of satisfaction with regard to staff attitude and competence.

The quantitative findings also indicated that there was no significant effect of the PBF intervention on the productivity or presence of health workers at their posts. This result was unexpected and contrary to the effect anticipated from the financial rewards potentially associated with each consultation. The absence of impact on clinical productivity can be explained by the strong focus of the PBF scheme in Benin on the quality of care of services and consultation. It is also possible that health workers focused more on the content of the care they provided, which may counteract the incentive to see as many patients as possible.

In general, the qualitative findings revealed that in both PBF intervention and control groups, staff were positive about its effects. The health personnel staff mentioned that the PBF bonuses were motivational and led to improved working environments. For instance, health workers declared that their facilities were cleaner than before the programs and that acquiring basic equipment and drugs had become easier.

Despite these encouraging results, the middle evaluation also noted that the PBF mechanism had some shortcomings. For example, health facility managers deplored the lack of autonomy to buy drugs or hire personnel because of national policies constraints, and the onerous perceived administrative rules under the PBF pilot program. Also, there was confusion between the PBF intervention and T2 group among health workers. The performance of health staff in the T2 group was monitored and scrutinized in the same manner as the performance of health workers in the PBF intervention group. Therefore, many health staff in the T2 group thought that their bonuses were linked to their performance. Hence, there were only small differences between the PBF intervention group and the T2 group.