The Gavi Full Country Evaluations (FCE) were mixed-methods evaluations, covering the full results framework from inputs to impact over the period of 2013 to 2016. The FCE were three-year prospective studies that aimed to understand and quantify the barriers to and drivers of immunization program performance, with emphasis on the contribution of Gavi, the Vaccine Alliance in four countries: Bangladesh, Mozambique, Uganda, and Zambia. IHME conducted the evaluation to determine how Gavi’s process, products, and resources work at the country level to influence immunization-related outcomes.
With the help of universities and health institutes in each country, IHME gathered real-time data in order to provide feedback to Gavi and country stakeholders. The Gavi FCE mixed-methods evaluations collected data and conducted analysis at the national and subnational levels where possible. This permitted a more distinct examination of factors influencing successful program implementation and a more informed understanding of the factors that moderated the relationship between Gavi support and downstream indicators such as immunization coverage and child mortality.
Study methods
- The process evaluation examined the interface between the Vaccine Alliance and countries as Gavi inputs (financing, technical assistance, information, instruments/guidelines, and personal interactions) are applied for, received, and implemented. The process evaluation involved three different evaluation activities:
- Stakeholder network analysis: How are stakeholders at the local and international levels influencing the implementation of Gavi support?
- Process tracking: How does implementation of Gavi support compare to planned activities?
- Targeted studies including Milestone Event Studies and Contingent Studies: Flexible studies to focus on key topics of high priority to countries.
- The resource tracking component of this evaluation aimed to identify the flow and use of resources (financing, commodities, and technical assistance) for immunization programs. Adapting and applying existing RT tools like the National Health Accounts (NHA) framework and the Public Expenditure Tracking Survey (PETS) methodology, the RT component investigated research questions of interest to both Gavi and in-country stakeholders.
- Health facility surveys were employed to measure constraints to immunization delivery from both the supply side (such as vaccine cold-chain integrity) and demand side (such as health care seeking), as well as barriers to immunization delivery, to identify whether resources are being directed where they are most needed.
- Household surveys were employed to measure population-based indicators, including vaccine antibodies, and geographically complementing health facility and resources tracking data to allow for cross-method analysis.
- Vaccine effectiveness studies were used to measure true effectiveness of vaccines in the population using novel methods such as overlapping demographic and disease surveillance, nasopharyngeal carriage studies, and case-control.
- Novel outcome and impact analyses were used to analyze primary and secondary data using novel statistical methods to estimate trends over time at the subnational level and use difference-in-differences analyses at the subnational level to examine causal attribution. The outcome and impact evaluation analysis involved these elements:
- National- and subnational-level estimation of trends in key indicators by combining all available data
- Socioeconomic-related inequalities analysis
- District-level difference-in-differences analysis
- Cost-effectiveness analysis
In addition to improving future immunization activities, the prospective nature of the evaluation provided real-time feedback on vaccine program implementation to Gavi and country stakeholders, which allowed for timely action by country leaders and implementers, Gavi, and Vaccine Alliance partners. Last, the methods developed and utilized by the FCE were designed to be exportable and adaptable to future contexts and countries.
Survey instruments and protocols
Relevant survey data and protocols collected by IHME and local partner organizations for the Gavi FCE can be found on the Global Health Data Exchange (GHDx).
Funding and partners
The evaluations were carried out by a consortium of research partners led by IHME and were financed through Gavi, the Vaccine Alliance, a public-private partnership that funds and improves access to vaccination in the world’s poorest countries.
The Gavi FCE team harnessed the expertise of numerous distinguished organizations. The evaluation work in each of the four countries is led by the respective in-country institutions:
- University of Eduardo Mondlane (UEM), Mozambique
- Health Alliance International (HAI), Mozambique
- Manhiça Health Research Centre (CISM), Mozambique
- icddr,b, Bangladesh
- Infectious Diseases Research Collaboration (IDRC), Uganda
- University of Zambia (UNZA), Zambia
- PATH, US
