Argentina - Stand-Alone Impact Evaluation

Project Information

Status: Active
Financing: HRITF $0 million and IDA $0 million

EVALUATION DESIGN

Since 2004, the Plan Nacer, known as the SUMAR program since 2012, finances “results-based” capitation payments to provinces for eligible beneficiaries. The payments are used to deliver a pre-defined package of maternal and child health services to eligible pregnant women and children aged under 6, who do not have a formal health insurance. The program allocates province-level funding on the basis on beneficiary enrollment and provides incentives following a pay-for-performance model based on indicators for the utilization and quality of maternal and child health (MCH) services, and health outcomes. The provinces use these resources to pay health facilities for MCH services provided to beneficiaries.

The World Bank, jointly with the Ministry of Health carried out two rounds of evaluation. The first evaluation aimed at identifying the impact of Plan Nacer program as a whole on selected health outcomes. It exploited the geographic phasing of clinics into the program over time within each province to create treatment and control groups and used a difference-in-difference strategy to determine the change in outcomes that can be attributed to the program. The evaluation relied on administrative data sources including birth and medical records at maternity hospitals. The second evaluation investigated the effects of temporary incentives paid to clinics for early initiation of prenatal care using a more restricted experimental approach in one province (Misiones). The field experiment provided temporary financial premium for early initiation of prenatal care to a randomly selected sub-set of health clinics. In particular, treatment clinics received a three-fold increase in payment for the first prenatal care visit if that visit took place in the first trimester of pregnancy.

FINDINGS

The first impact evaluation found that the program increased the number of perinatal care visits as well as the probability of receiving a tetanus vaccine. The beneficiaries had a 19% lower probability of low birth-weight compared to non-beneficiaries. In addition, the beneficiaries had a 74% lower chance of in-hospital neonatal mortality in larger facilities, approximately half of which is due to reduced incidence of low birth weight and half from better postnatal care. Results also showed that the financial autonomy provided to facilities by Plan Nacer allowed a better allocation of scare resources, which led to positive impact on health outcomes of beneficiaries. The cost-effectiveness analysis also found Plan Nacer highly cost-effective compared with Argentina’s gross domestic product per capita over this period. The study indicated small negative spillover effects on prenatal care utilization of non-beneficiary populations in clinics covered by Plan Nacer, but no spillover was found on birth outcomes.

The second impact evaluation aimed to estimate the impact of the temporary premium not only during the intervention but also after the premium was discontinued.  The evaluation found that the rate of early initiation of prenatal care was 34% higher in the treatment group than in the comparison group while the incentives were being paid, and that this effect persisted 15 months and likely 24 months after the incentives ended. The study also documented that health staff in clinics developed new strategies to identify women who were likely to be pregnant by changing their routines. For instance, community health workers were used extensively to find pregnant women and encourage them to start care early. These changes in routines persisted at least 15 months after the incentives ended. The results, however, also suggested that the quality of care might have remained a constraint to improving health outcomes as the increase in early initiation of prenatal care did not have any effect on birth outcomes. Nonetheless, the study found that large-but-temporary incentives might be more cost-effective at motivating provider performance and changing clinical practice than permanent incentives when providers face fixed costs to changing clinical practice routines.