An essential element of Universal Health Coverage is access to adequate quality care. Despite significant increases in access, the quality of care remains low for significant proportions of the world’s poor. Importantly, research has shown(1–4) that one of the most effective ways to improve health outcomes is to increase adherence to medical protocol: following best practice according to the patient’s condition. Given that adherence to protocol is very low on average, it is important for policy makers to investigate the determinants of low adherence and to generate evidence on policies that serve to increase it. “The Three-Gap Model“advances a methodology that can be used to investigate potential determinants of low quality, to examine whether programs designed to improve quality actually work, and to test the theory of change motivating particular policies.
Designed as an evaluation strategy for an RBF project in Liberian hospitals, the Three-Gap Model is a way to dig deeper into the reasons for low performance. The model starts with the idea that there are appropriate and recommendedsteps that health workers should take for patients across a variety of settings, including outpatient and inpatient care across pediatrics, surgery, emergency and obstetric and maternal care. Quality is increased when performance on these steps (protocol adherence) increases. Measuring the performance of health workers on these steps exposes a gap between performance and the target. However, the size of this gap tells us little about how to improve performance. Some health workers are poorly trained, some don’t have the necessary equipment and others are insufficiently motivated.
To better understand inadequate protocol adherence, we need data for each health worker on knowledge to perform --- does the health worker know the steps required by protocol? --- and capacity to perform --- does the health worker know the steps and have access to the diagnostic tools equipment and medicines required to follow protocol --- as well as performance --- does the health worker follow protocol with actual patients. This allows us to describe each health worker by the know gap (the gap between the target and knowledge), the know-can gap (the gap between capacity and knowledge) and the can-do gap (the gap between performance and capacity). In the figure below, we show the average knowledge, capacity and performance and the target performance for a hypothetical sample of health workers. Performance and target performance are plotted on the y-axis and capacity and knowledge are plotted on the x-axis, all as a percentage of the items required. We can use the 45-degree line to translate capacity into the capacity to perform and knowledge into the knowledge to perform and then plot all four measures on the y-axis. This allows us to measure the average know gap, know-can gap and can-do gap.
We argue that the sizes of the gaps by themselves are not useful for designing policy, however. The Three-Gap model allows us to focus on the relationships between capacity, knowledge, and performance. In particular, we examine (1) the rate at which health workers can translate increased knowledge into capacity and increased capacity into performance, and (2) the role that policies, such as RBF, play in changing these rates and improving performance. In the same figure, we show the values of capacity and performance for the full hypothetical sample and the dashed line represents the approximate relationship between capacity and performance. Note that, although the can-do gap (G3) is relatively large, increases in capacity have a relatively small effect on performance because the slope of the relationship between capacity and performance is much lower than 1.
On the other hand, policies—such as RBF—that increase the motivation of health workers to more fully use their capacity could be more effective. Note that a policy to increase motivation could increase performance at every level of capacity (shifting the dashed line upwards) or increase the importance of capacity in performance (rotating the dashed line so that it had a greater slope). These are important distinctions because the first suggests that motivation is more important than capacity whereas the second suggests that motivation helps make capacity important.
Using 2015 baseline data from the Health Systems Strengthening Project in Liberia we examine the levels and patterns for pediatric (outpatient and emergency) care in 10 Liberian hospitals. Quality, as measured by adherence to protocol, is very low: about 35% of the target. The know gap is very large—health worker’s knowledge is only about 45% of the target. The know-can gap on the other hand is very small—health workers are not limited by access to equipment or consumables. Finally, the can-do gap represents almost 10% of the remaining short fall in quality. These levels suggest that knowledge is the most important policy element. However, by looking at the patterns in the data, a different story emerges. Although knowledge is low, there is significant variance. The data show that health workers with high knowledge also have high capacity, but they do not have markedly higher levels of performance: the relationship between capacity and performance is weak. In fact, the performance of a health worker with knowledge that is 20% of the target is only about 10% lower than the performance of a health worker with knowledge that is 80% of the target. Even if policy could significantly improve knowledge, it is likely to only make a small difference in performance.
In order to learn more about what could help improve performance, we examine the degree to which these patterns are affected by the motivation of health workers. Is it the case that health workers with high levels of motivation have higher levels of performance and/or are better able to translate skills into practice? As part of the baseline, we asked all health workers a series of questions about their perceptions of themselves and their attitudes towards their job, workplace and profession. For example, do they agree with the statements, “I regularly have access to relevant trainings to keep my skills up to date”, “Maintenance of broken equipment at this facility is prompt and reliable” and “In this organization, I am valuable.” Using confirmatory factor analysis, we combine these questions into a series of five composite indices that capture different facets of motivation. This allows us to show a potentially important pattern in the relationship between health workers who feel valued by their health facility (one of the five indices) and their capacity to provide high quality care. For health workers who have low levels of capacity (whose knowledge of protocol is inadequate), feeling valued by their facility does not improve performance; however, for those who have high levels of capacity, feeling valued does have an important positive effect.
From a diagnostic perspective, this result is potentially important. The baseline data shows that increases in equipment and consumables are not likely to improve performance of any health workers, but that increasing knowledge could have an important effect if it is paired with changes in motivation. Or, looking at the same facts from a different perspective, increasing the sense of being valued by the facility without also increasing knowledge and skills could have detrimental effects on quality.
The implications of the Three-Gap Model being applied to the baseline data were presented to a team from the Ministry of Health and Social Welfare in Liberia in March of 2019 for validation and as an input into policy discussions. The reaction of the team is captured by the statement of a member who had served as a Hospital Director before joining the Ministry: “If you had found anything else, I would not have believed you.” There was strong agreement that equipment is not the limiting factor in Liberian hospitals, but that increasing the motivation and skills of the staff are a first priority. The model helps to formalize the intuition held by policy experts, which is valuable in formulating policy.
The value of the Three-Gap Model as an evaluation tool is also important because it allows us to test the theory of change behind the design of any particular policy, not just the total impact. By introducing multiple measures for each health worker, the model is an important step towards understanding the relationship between many factors determining a health worker’s performance and a recognition that these patterns may be complex.
The full 3-gap model is now available online as a World Bank Policy Research Working Paper.
(1) Chopra, Mickey, Elizabeth Mason, John Borrazzo, Harry Campbell, Igor Rudan, Li Liu, Robert E. Black, and Zulfiqar A. Bhutta. “Ending of Preventable Deaths from Pneumonia and Diarrhoea: An Achievable Goal.” Lancet (London, England) 381, no. 9876 (April 2013): 1499–1506. https://doi.org/10.1016/S0140-6736(13)60319-0.
(2) Jaribu, Jennie, Suzanne Penfold, Fatuma Manzi, Joanna Schellenberg, and Constanze Pfeiffer. “Improving Institutional Childbirth Services in Rural Southern Tanzania: A Qualitative Study of Healthcare Workers’ Perspective.” BMJ Open 6, no. 9 (September 2016): e010317. https://doi.org/10.1136/bmjopen-2015-010317.
(3) Manzi, Anatole, Laetitia Nyirazinyoye, Joseph Ntaganira, Hema Magge, Evariste Bigirimana, Leoncie Mukanzabikeshimana, Lisa R. Hirschhorn, and Bethany Hedt-Gauthier. “Beyond Coverage: Improving the Quality of Antenatal Care Delivery through Integrated Mentorship and Quality Improvement at Health Centers in Rural Rwanda.” BMC Health Services Research 18 (February 2018): 136. https://doi.org/10.1186/s12913-018-2939-7.
(4) Peabody, John W., Riti Shimkhada, Stella Quimbo, Orville Solon, Xylee Javier, and Charles McCulloch. “The Impact of Performance Incentives on Child Health Outcomes: Results from a Cluster Randomized Controlled Trial in the Philippines.” Health Policy and Planning 29, no. 5 (August 2014): 615–621. https://doi.org/10.1093/heapol/czt047.