Resources

Qualitative studies endeavor to answer the “how” and the “why” of Results-Based Financing (RBF) mechanisms in the health sector. For instance, how does implementation work toward a specific health outcome? Why does a program lead to positive health results in a specific setting and not in another? How do communities engage with the health system? Why are improvements in the quality of care not generating intended increases in service utilization? By investigating and answering such questions, qualitative research has the potential to help RBF practitioners and researchers appropriately harness the underlying behavioral, political and logistical factors that influence RBF mechanisms and which, ultimately, foster positive health outcomes.  

Recognizing this potential, RBF interventions increasingly use qualitative research methods. In The Gambia, for example, focus group discussions and interviews – carried out during the program planning process – enabled to identify existing socio-cultural barriers to service utilization. They also underlined the need to complement the supply-side Performance-Based Financing (PBF) intervention with a demand-side intervention. By highlighting the experiences and perspectives of communities, these methods helped inform program design and sharpened the program’s capacity to reach intended outcomes. Conversely, in Nigeria, qualitative research was conducted to understand large variations in performance as well as in the quality of care in pre-pilot health facilities. Focusing on the elements that impede community demand as well as on the determinants of performance, these qualitative studies informed project priorities, including transportation cost and availability, service cost and cost predictability, and management capacity. Based on these findings, a demand-side intervention, focusing especially on community incentives, community engagement and transport vouchers, was developed. In this context, qualitative research helped investigate why program activities did not yield expected results; it also helped devise “corrective measures”, inform program roll out and, in turn, strengthen performance and quality.  

Because the Health Results Innovation Trust Fund (HRITF) recognizes and values the utility of these qualitative endeavors in RBF in the health sector, it commissioned an external and independent review to help capitalize on and learn from existing RBF experiences. This review – conducted by Karina Kielmann and Fabian Cataldo in collaboration with the London School of Hygiene and Tropical Medicine – was presented during a workshop held at the World Bank in Washington DC. 

Seeking to understand how studies were conducted, the review examined research-related documentation of 17 RBF projects, including  research protocols, data collection tools, training guides, data analysis matrices, etc.  It particularly focused on 6 RBF projects to highlight best practices and identify opportunities to further strengthen qualitative studies in RBF and enhance their added value.

Based on this in-depth investigation, this external review importantly ascertained the added value, robustness and quality of qualitative studies conducted to date.  Indeed, among other things, it underlines that research design, especially the definition of concepts and their relationships, is methodologically consistent: the conceptual framework, which was drafted to delineate the behavioral, contextual and organizational changes instigated by RBF, is consistently used to design studies and identify research questions. Although the review stresses the usefulness of the PBF conceptual framework in this regard, it also suggests that it might confine studies, limiting their scope to already identified areas of inquiry. The review thus proposes to think “outside the box” to generate rich data. It suggests taking a closer look at more abstract concepts, such as organizational and managerial behaviors, and focusing on different levels of the health system such as the organizational culture and management structure of health facilities.

Similarly, the review indicates that data collection instruments and protocols are generally comprehensively designed. However, it also stipulates that, at times, questions are not formulated with enough emphasis on informants’ experiences. Indeed, because data collection methods are often based on structured interviews, they at times restrict informants’ capacity to organically share their experience – that is to say, they are unable to tell their story in a way that makes sense to them, using their own words.    To fully capture “what is really happening on the ground” and move away from away from abstract technical concepts, the review recommends designing smaller and more focused studies where more time can be spent with informants, thereby eliciting open exchanges and yielding richer and fit-for-purpose data. It thus advocates for less prescriptive data collection tools designed to allow individuals to share their thoughts without being influenced by the researcher (i.e. through structured questionnaires).  

To further elicit these open exchanges, the review stresses the importance of planning and logistics, underlining selection and sampling strategies as priorities.  Although most RBF qualitative studies designed effective strategies capable of capturing rich and meaningful data, the review suggests operating a shift from statistically representative to qualitatively representative samples. While the former provides large volumes of data – often requiring significant amounts of work – the later focuses on a reduced number of information rich informants, such as community gatekeepers or positive deviants[1].  This shift could enhance the focus of qualitative studies and contribute to reducing the volume of data, the time allocated to research as well as costs.

To be successful, nevertheless, researchers have to be trained and time should be allocated to accessing the field as well as to building rapport and trust with informants. Indeed, to fully tap into the potential of qualitative research, the review recommends interviewing the same individuals over time to provide a “real time” dimension to qualitative studies in RBF. This temporal dimension is instrumental to track changes as they occur and help maximize the positive effects of RBF mechanisms and/or mitigate their potential negative impact. Taking a closer look at the “now” is central to fully realize the potential of qualitative research and understand the causes change.

Illustrating the usefulness of qualitative research and providing other avenues for improvement, this review marks the start of a promising initiative, shining a light into the black box lying at the heart of RBF. It helps practitioners make changes that appropriately harness human behavior and limit potential perverse incentives or unintended consequences. The availability of qualitative data in RBF is therefore vital for tailored program design and implementation, for problem resolution and course corrections, for improved and sustained health outcomes as well as for effective policy making.

 

[1] Positive deviants are people whose uncommon but successful behaviors or strategies enable them to find better solutions to a problem than their peers, despite facing similar challenges and having no extra resources or knowledge than their peers. 

 

  • Qualitative Research to Enhance the Evaluation of Results-Based Financing Programmes : The Promise and the Reality
    Size: 631.27 KB

Resource Information

Document Type: (PDF) Download
Author/s: Joy Gebre Medhin
Countries: Gambia, The
Date of Publication: July 2016

Share This Resource