Author
Cheri Grace
Bio: Cheri Grace is an academic researcher. The author has contributed to research in topics: Essential medicines & Portfolio. The author has an hindex of 2, co-authored 2 publications receiving 116 citations.
Papers
More filters
01 Jan 2004
108 citations
01 Jan 2003
11 citations
14 Dec 2022
TL;DR: The National Institute for Health and Care Research's (NIHR) Global Health Research (GHR) portfolio as mentioned in this paper was evaluated by the World Health Organization (WHO) in 2018.
Abstract: Evaluation of the National Institute for Health and Care Research’s (NIHR) Global Health Research (GHR) Portfolio
Inception Report
October 2022
Contents
Executive Summary
1.0 NIHR GHR Portfolio and Introduction to the Evaluation
1.1 Overview of the GHR Portfolio
1.2 Evaluation Objectives and Scope
2.0 Evaluation Approach
2.1 Evaluation Conceptual Framework
2.2 Evaluation Framework
3.0 Evaluation Methodology
3.1 Overview of Research Timetable
3.2 Research Tasks
3.3 Overview of Sampling
3.4 Approach to Data Analysis
3.5 Framework to Assess the Strength of the Evaluation Findings by Strength of Evidence
4.0 Reporting and Dissemination
5.0 Annexes
5.1 Annex 1: Evaluation Framework
5.2 Annex 2: TOC Workshop Summary
Tables
Table 1 GHR Portfolio Summary
Table 2 Evaluation Criteria and Questions
Table 3 Number of awards to be evaluated by programme
Table 4 Robustness rating for emerging themes/main findings
Figures
Figure 1 Approach to Contribution Analysis
Abbreviation List
AAS: African Academy of Sciences
AMR: Anti-microbial resistance
CA: Contribution Analysis
CCF: Central Commissioning Facility
CEI: Community Engagement and Involvement
CEPI: Coalition for Epidemic Preparedness Innovations
DAC: Development Assistance Committee
DHSC: Department for Health and Social Care
EDCTP: European & Developing Countries Clinical Trials Partnership
EF: Evaluation Framework
EPSRC: Engineering and Physical Sciences Research Council
EQ: Evaluation Question
ESRC: Economic and Social Research Council
FAF: Financial Assurance Fund
FCDO: Foreign, Commonwealth and Development Office
FGD: Focus Group Discussion
FIND: Foundation for Innovative New Diagnostics
GACD: Global Alliance for Chronic Diseases
GARDP: Global Antibiotic Research and Development Partnership
GCC: Grand Challenges Canada
GECO: Global Effort on COVID-19
GFGP: Good Financial Grants Practice
GHR: Global Health Research
GPSC: Global Patient Safety Collaborative
GRSF: Global Road Safety Facility
HPSR: Health Policy and Systems Research
JGHTI: Joint Global Health Trials Initiative
KII: Key Informant Interview
LMIC: Low- and Middle-Income Countries
MMV: Medicines for Malaria Venture
MRC: Medical Research Council
NETSCC: NIHR Evaluation, Trials and Studies Coordinating Centre
NIHR: National Institute for Health and Care Research
ODA: Official Development Assistance
OECD: Organisation for Economic Co-operation and Development
REA: Rapid Evidence Assessment
RIGHT: Research and Innovation for Global Health Transformation
RSTMH: Royal Society of Tropical Medicine and Hygiene
RW: Research Wave
R2HC: Research for Health in Humanitarian Crises
SNA: Social Network Analysis
SORT-IT: Structured Operational Research and Training Initiative
SPARC: Short Placement Award for Research Collaboration
TB: Tuberculosis
ToC: Theory of Change
ToR: Terms of Reference
VFM: Value for Money
WHO: World Health Organisation
Acknowledgements and Disclaimer
Acknowledgments
The lead authors of this report are Paula Quigley, Seema Khan, Cheri Grace, Sarah Hanka and Korina Cox.
Disclaimer
This evaluation is funded by the National Institute for Health Research (NIHR) under its Policy Research Programme (PRP) (Grant Reference Number NIHR203816). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care (DHSC).
This report has been prepared by Ecorys for DHSC, for services specified in the Terms of Reference and contract of engagement. In line with the terms and conditions of the contract, all intellectual property rights in all material (including but not limited to reports, data, designs, whether or not electronically stored) produced by the Supplier or the Supplier's Personnel pursuant to the performance of the Services (‘the Material’) shall be the property of the Supplier. Under the terms of the contract, Ecorys, as the Supplier hereby grants to DHSC a perpetual, world-wide, non-exclusive, irrevocable, royalty-free licence to use all the Material. DHSC will be the final owner of the findings of the evaluation.
Ecorys will store all material related to the evaluation on a secure drive. Data will be managed under the terms and conditions of the contract with DHSC.
There are no conflicts of interest.
Executive Summary
The National Institute for Health and Care Research (NIHR)’s Global Health Research (GHR) portfolio was established by the Department for Health and Social Care (DHSC) in 2016 to support the objectives of the UK Aid Strategy and the United Nations’ Sustainable Development Goals. As part of the 2015 spending review, an initial budget of £430 million was allocated for the first phase (2016-2021) of the GHR portfolio to fund applied global health research in low- and middle-income countries (LMICs) eligible to receive Official Development Assistance (ODA). The goal of the initiative is to support high quality applied health research and training to address unmet needs in ODA-eligible countries by generating evidence for the direct benefit of people in these countries. It also aims to strengthen research capacity, develop more equitable partnerships between UK and LMIC research institutes, overcome barriers to health research uptake and ensure a high level of community engagement and involvement (CEI).
In December 2021, the DHSC commissioned Ecorys to undertake an evaluation of the first phase of the portfolio (2016/17 to 2020/21). The evaluation’s objectives are to assess the suitability of the design and implementation of the portfolio for achieving its intended results, and to identify key learning to inform development and delivery of the portfolio’s second phase (2021/22 onwards). In addition, the evaluation aims to provide accountability for the GHR portfolio performance to date, determining the Value for Money (VFM) of investments, and assessing whether the portfolio is on track to achieve desired outcomes and long-term impact.
The evaluation approach is based on the recognition that there has been notable learning during the first phase of the portfolio implementation and aims to capture this. The evaluation will employ a theory-based approach utilising both quantitative and qualitative methods to assess process, performance and learning based on the GHR portfolio’s Theory of Change (ToC). Specifically, the evaluation will use Contribution Analysis (CA), to build evidence of whether and how the GHR portfolio is contributing to its intended outcomes and impacts. The evaluation framework is structured around three pillars of evaluation criteria[1]: i) relevance and coherence ii) efficiency and effectiveness iii) sustainability and impact. As cross-cutting themes, the evaluation team will assess adaptability and learning across the portfolio, as well as CEI. The evaluation framework is intentionally designed as a comprehensive and overarching framework which will be tailored to the specific context of each stakeholder. Following the inception phase, there will be two research waves – the first from September/October 2022 to April 2023 and the second from May/June to December 2023, followed by a final dissemination phase up to March 2024.
The key methods comprise programme and award documentation and data review; interviews with key stakeholders at portfolio, programme and award levels; an online survey to all funded stakeholders; a Social Network Analysis (SNA) to assess interactions and networks; and country visits where feasible in the second year. A purposive sample of programmes and awards for in-depth review is proposed which the evaluation team judges to be reasonably reflective of the overall portfolio and provide the breadth to answer the questions in the evaluation framework. Data sources include primary data, for example, strategic interviews, case study interviews with award holders, observations at country level and survey data; and secondary data, such as portfolio level, delivery mechanism, and award level documentation.
All data will be coded according to the evaluation questions ensuring a consistent approach to aid synthesis and minimise bias. The evaluation team will assess and triangulate the evidence from different data sources to build up a picture of the GHR’s contribution to change. An interim report produced in March/April 2023 will collate findings from the portfolio, programme and award assessments from the first research wave and provide key learning points for NIHR and DHSC. This will include any necessary adaptations in the methodology for the second research wave and any proposed recommendations for changes in the ToC. A final evaluation report will be delivered by December 2023. Recommendations will be relevant and realistic, and all evaluation outputs accessible and useful. A range of dissemination products and channels are proposed for the final step.
The evaluation team is committed to ensuring that the entire process is responsive and helpful to the DHSC and NIHR, as well as other implementing partners, and will provide opportunities for feedback and interaction as needed. The team will maintain regular communications with key stakeholders throughout as appropriate and aim to be as inclusive as possible, while acknowledging the need to balance engagement with potential overburdening with demands for data and time. The team will ensure an ethical approach to this evaluation and adhere to relevant international guidelines. All data will be securely stored and managed.
This Inception Report presents the proposed approach to this evaluation – detailing the conceptual framework, evaluation questions, methodology for data collection and analysis, reporting and dissemination.
1.0 NIHR GHR Portfolio and Introduction to the Evaluation
1.1 Overview of the GHR Portfolio
The Global Health Research (GHR) portfolio was established by the Department of Health and Social Care (DHSC) with the National Institute for Health and Care Research (NIHR), following the publication of the 2015 UK Aid Strategy. It aims to directly address the diverse health needs of people in low- and middle-income countries (LMICs) by supporting high quality applied health research and training in areas where there is an unmet need; generating evidence for the direct benefit of people in LMICs; and strengthening LMIC and UK research capabilities and expertise in global health. As part of the 2015 spending review, an initial budget of £430 million was allocated for the first phase (2016-2021) of the GHR portfolio to contribute to improvement in global health outcomes through applied global health research, complementing other Official Development Assistance (ODA) research programmes such as the Global Challenges Research Fund and the Newton Fund. Since its launch in 2016, and as of September 2022, the portfolio has grown to include 17 thematic areas, with research activities being conducted in over 50 LMICs across Africa, Asia and Latin America, and through more than 20 delivery partners. In Phase 1, the GHR portfolio funded more than 650 awards which ranged from £5,000 to several million pounds[1]. In addition to being ODA-eligible, all the global health research funded by GHR must be underpinned by the following principles:
When the portfolio was launched in 2016, DHSC initially prioritised working with partners who were already well established in the global health research space, such as the Medical Research Council (MRC) and the Wellcome Trust. NIHR rapidly built up its internal capacity in managing global health research programmes, growing from a small core team of only three people to a much more extensive team today. Over time NIHR principles that govern all work, whether domestic or global, have also been embedded in the partnerships where feasible providing extra added value of working in partnership together, and extending the influence of NIHR on other funders and activities. More detail behind the story of the evolution of the GHR portfolio will be gathered in the next phase of the evaluation. Given the rapid evolution of the portfolio, this evaluation provides an opportunity for learning, and to inform the second phase and subsequent phases of the portfolio.
Table 1 GHR Portfolio Summary
NIHR-delivered
1.2 Evaluation Objectives and Scope
Ecorys has been contracted by DHSC to assess the suitability of the design and implementation of the first phase of the GHR portfolio (2016/17-2020/21) in achieving its intended outcomes and impacts as set out in GHR’s Theory of Change (ToC). The evaluation will contribute to the evidence base on health research impact, as well as building evidence and understanding of a) the extent to which researchers are engaged in effective knowledge mobilisation (and what types of support or interventions encourage the transfer of knowledge), and b) the extent to which community engagement and involvement (CEI) meaningfully and sustainably leads to higher quality research and intended impacts.
The ToC has a long-term perspective, acknowledging that it may require 10-25 years for changes in policy, practice and behaviour (outcomes) to contribute towards strengthened health systems and increased individual and community capacity for health promotion and disease prevention (impacts). The evaluation of the first phase of the portfolio takes into account this long-term nature of change and, therefore, will assess evidence that the portfolio is being delivered as expected and whether or not key assumptions are holding true. A core objective of the evaluation is to generate real-time learning to support implementation and inform development and delivery of subsequent phases of the portfolio. The evaluation will include a strategic assessment of VFM at the portfolio level in terms of allocative efficiency, technical efficiency and value/results. Equity, gender equality and social inclusion will be considered key cross cutting principles and integrated and addressed throughout the evaluation.
In scope for this evaluation are all GHR portfolio programmes and their awards that started during the Phase 1 timeframe and ideally completed by the second quarter of 2022. However, some programmes have no or very few awards that were completed by Q2 2022 (e.g. Professorships, RIGHT), therefore, the evaluation will also review some active awards to ensure coverage across the entire portfolio.
2.0 Evaluation Approach
2.1 Evaluation Conceptual Framework
2.1.1 GHR Theory of Change The GHR portfolio Theory of Change (ToC) produced collaboratively by the DHSC GHR Team, NIHR Coordinating Centres and other strategic partners and award holders provides the conceptual framework for this evaluation. The GHR ToC visually represents how the portfolio’s funded activities and outputs are intended to contribute to long-term positive changes in health and health systems.
A ToC workshop (see Annex 2) was organised with portfolio stakeholders during inception in order to develop an agreed and improved understanding of all elements of the NIHR GHR ToC, to further refine the evaluation questions and scope. During the workshop, it was agreed that this simplified diagram of expected results from inputs and activities to outputs, outcomes and impact remains broadly valid. However, since it was developed, the initial concept of three activity strands, namely, Programmes and Partnerships with an overlapping strand of People, has evolved over time. The Programmes strand which refers to NIHR-commissioned research calls and the Partnerships strand which refers to collaborations with other funders are the two delivery mechanisms. The People strand refers to training and development of researchers and research support staff. This does not quite capture the capacity strengthening approach that is embedded in all programmes and is a key principle across the portfolio. The evaluation will seek to determine if there is a more appropriate way to reflect the GHR programme delivery in the ToC.
It is anticipated that the GHR portfolio’s short-term contribution to producing high quality relevant research outputs, supporting capacity building, facilitating partnerships and networks, and supporting dissemination and knowledge exchange will influence policy making and practice in the medium term. The ToC identifies these anticipated causal links drawing on underpinning theory from literature on health research impact, including the uptake of research evidence to improve the responsiveness of health systems to population needs and improved use of evidence to support capacity for health promotion and disease prevention. Outcomes are dependent on a wide range of external factors, which are captured in the ToC narrative as a set of assumptions. These assumptions have been unpacked in the evaluation framework so that they can be examined and tested. Some of them have been affected by the COVID-19 pandemic and other stresses on the UK economy, for example, the assumption that ‘global health research funders continue funding at present rate’. The evaluation team will review all assumptions during the first year of assessment along with the flow diagram and suggestions for amendments will be made where appropriate.
The ToC has been used to inform the Evaluation Framework (EF) (Section 2.2) and will be used to structure a theory-based approach to assessing the portfolio’s outcomes and impact. This approach is appropriate for complex interventions in complex environments. Specifically, the evaluation will use Contribution Analysis (CA), a theory-based evaluation methodology, to build evidence of whether and how the GHR portfolio is contributing to its intended outcomes and impacts.
Contribution Analysis is well-suited to meeting the evaluation objectives and answering the evaluation questions, given the time and resources available. Specifically, the choice of this method is informed by the following considerations:
2.1.2 Approach to Contribution Analysis
The CA method and analytical tools provide a way to structure and build up evidence about GHR portfolio activities and their potential contribution to supporting long-term change in health and health systems. The evaluation team will use the ToC to gather evidence and draw conclusions about the relevance and coherence of GHR-supported activities and whether and how different programmes are achieving their intended results, whether these are being implemented in line with key principles that underpin a process of sustainable long-term change (e.g. supporting capacity strengthening, equitable research partnerships), and key factors influencing this.
Practically, the CA methodology facilitates the structuring of evidence during each of the evaluation phases outlined below. First, the evaluation will collate and collect evidence about observed changes related to the portfolio’s activities in the short term, exploring the processes and causal pathways through which these have occurred and identify emerging evidence of progress towards influence on policy making and practice (i.e. expected medium term outcomes). Second, the evaluation will assess the role played by the GHR funded activity and wider influence linked to partnership working, and the role of other factors in hindering or enabling positive change. This systematic approach will provide a foundation for the evaluation to report on evidence of the GHR portfolio’s contribution to outcomes and impact under the first phase and generate learning to inform future phases.
Figure 1 Approach to Contribution Analysis
Inception Phase
Inception Report
October 2022
Contents
Executive Summary
1.0 NIHR GHR Portfolio and Introduction to the Evaluation
1.1 Overview of the GHR Portfolio
1.2 Evaluation Objectives and Scope
2.0 Evaluation Approach
2.1 Evaluation Conceptual Framework
2.2 Evaluation Framework
3.0 Evaluation Methodology
3.1 Overview of Research Timetable
3.2 Research Tasks
3.3 Overview of Sampling
3.4 Approach to Data Analysis
3.5 Framework to Assess the Strength of the Evaluation Findings by Strength of Evidence
4.0 Reporting and Dissemination
5.0 Annexes
5.1 Annex 1: Evaluation Framework
5.2 Annex 2: TOC Workshop Summary
Tables
Table 1 GHR Portfolio Summary
Table 2 Evaluation Criteria and Questions
Table 3 Number of awards to be evaluated by programme
Table 4 Robustness rating for emerging themes/main findings
Figures
Figure 1 Approach to Contribution Analysis
Abbreviation List
AAS: African Academy of Sciences
AMR: Anti-microbial resistance
CA: Contribution Analysis
CCF: Central Commissioning Facility
CEI: Community Engagement and Involvement
CEPI: Coalition for Epidemic Preparedness Innovations
DAC: Development Assistance Committee
DHSC: Department for Health and Social Care
EDCTP: European & Developing Countries Clinical Trials Partnership
EF: Evaluation Framework
EPSRC: Engineering and Physical Sciences Research Council
EQ: Evaluation Question
ESRC: Economic and Social Research Council
FAF: Financial Assurance Fund
FCDO: Foreign, Commonwealth and Development Office
FGD: Focus Group Discussion
FIND: Foundation for Innovative New Diagnostics
GACD: Global Alliance for Chronic Diseases
GARDP: Global Antibiotic Research and Development Partnership
GCC: Grand Challenges Canada
GECO: Global Effort on COVID-19
GFGP: Good Financial Grants Practice
GHR: Global Health Research
GPSC: Global Patient Safety Collaborative
GRSF: Global Road Safety Facility
HPSR: Health Policy and Systems Research
JGHTI: Joint Global Health Trials Initiative
KII: Key Informant Interview
LMIC: Low- and Middle-Income Countries
MMV: Medicines for Malaria Venture
MRC: Medical Research Council
NETSCC: NIHR Evaluation, Trials and Studies Coordinating Centre
NIHR: National Institute for Health and Care Research
ODA: Official Development Assistance
OECD: Organisation for Economic Co-operation and Development
REA: Rapid Evidence Assessment
RIGHT: Research and Innovation for Global Health Transformation
RSTMH: Royal Society of Tropical Medicine and Hygiene
RW: Research Wave
R2HC: Research for Health in Humanitarian Crises
SNA: Social Network Analysis
SORT-IT: Structured Operational Research and Training Initiative
SPARC: Short Placement Award for Research Collaboration
TB: Tuberculosis
ToC: Theory of Change
ToR: Terms of Reference
VFM: Value for Money
WHO: World Health Organisation
Acknowledgements and Disclaimer
Acknowledgments
The lead authors of this report are Paula Quigley, Seema Khan, Cheri Grace, Sarah Hanka and Korina Cox.
Disclaimer
This evaluation is funded by the National Institute for Health Research (NIHR) under its Policy Research Programme (PRP) (Grant Reference Number NIHR203816). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care (DHSC).
This report has been prepared by Ecorys for DHSC, for services specified in the Terms of Reference and contract of engagement. In line with the terms and conditions of the contract, all intellectual property rights in all material (including but not limited to reports, data, designs, whether or not electronically stored) produced by the Supplier or the Supplier's Personnel pursuant to the performance of the Services (‘the Material’) shall be the property of the Supplier. Under the terms of the contract, Ecorys, as the Supplier hereby grants to DHSC a perpetual, world-wide, non-exclusive, irrevocable, royalty-free licence to use all the Material. DHSC will be the final owner of the findings of the evaluation.
Ecorys will store all material related to the evaluation on a secure drive. Data will be managed under the terms and conditions of the contract with DHSC.
There are no conflicts of interest.
Executive Summary
The National Institute for Health and Care Research (NIHR)’s Global Health Research (GHR) portfolio was established by the Department for Health and Social Care (DHSC) in 2016 to support the objectives of the UK Aid Strategy and the United Nations’ Sustainable Development Goals. As part of the 2015 spending review, an initial budget of £430 million was allocated for the first phase (2016-2021) of the GHR portfolio to fund applied global health research in low- and middle-income countries (LMICs) eligible to receive Official Development Assistance (ODA). The goal of the initiative is to support high quality applied health research and training to address unmet needs in ODA-eligible countries by generating evidence for the direct benefit of people in these countries. It also aims to strengthen research capacity, develop more equitable partnerships between UK and LMIC research institutes, overcome barriers to health research uptake and ensure a high level of community engagement and involvement (CEI).
In December 2021, the DHSC commissioned Ecorys to undertake an evaluation of the first phase of the portfolio (2016/17 to 2020/21). The evaluation’s objectives are to assess the suitability of the design and implementation of the portfolio for achieving its intended results, and to identify key learning to inform development and delivery of the portfolio’s second phase (2021/22 onwards). In addition, the evaluation aims to provide accountability for the GHR portfolio performance to date, determining the Value for Money (VFM) of investments, and assessing whether the portfolio is on track to achieve desired outcomes and long-term impact.
The evaluation approach is based on the recognition that there has been notable learning during the first phase of the portfolio implementation and aims to capture this. The evaluation will employ a theory-based approach utilising both quantitative and qualitative methods to assess process, performance and learning based on the GHR portfolio’s Theory of Change (ToC). Specifically, the evaluation will use Contribution Analysis (CA), to build evidence of whether and how the GHR portfolio is contributing to its intended outcomes and impacts. The evaluation framework is structured around three pillars of evaluation criteria[1]: i) relevance and coherence ii) efficiency and effectiveness iii) sustainability and impact. As cross-cutting themes, the evaluation team will assess adaptability and learning across the portfolio, as well as CEI. The evaluation framework is intentionally designed as a comprehensive and overarching framework which will be tailored to the specific context of each stakeholder. Following the inception phase, there will be two research waves – the first from September/October 2022 to April 2023 and the second from May/June to December 2023, followed by a final dissemination phase up to March 2024.
The key methods comprise programme and award documentation and data review; interviews with key stakeholders at portfolio, programme and award levels; an online survey to all funded stakeholders; a Social Network Analysis (SNA) to assess interactions and networks; and country visits where feasible in the second year. A purposive sample of programmes and awards for in-depth review is proposed which the evaluation team judges to be reasonably reflective of the overall portfolio and provide the breadth to answer the questions in the evaluation framework. Data sources include primary data, for example, strategic interviews, case study interviews with award holders, observations at country level and survey data; and secondary data, such as portfolio level, delivery mechanism, and award level documentation.
All data will be coded according to the evaluation questions ensuring a consistent approach to aid synthesis and minimise bias. The evaluation team will assess and triangulate the evidence from different data sources to build up a picture of the GHR’s contribution to change. An interim report produced in March/April 2023 will collate findings from the portfolio, programme and award assessments from the first research wave and provide key learning points for NIHR and DHSC. This will include any necessary adaptations in the methodology for the second research wave and any proposed recommendations for changes in the ToC. A final evaluation report will be delivered by December 2023. Recommendations will be relevant and realistic, and all evaluation outputs accessible and useful. A range of dissemination products and channels are proposed for the final step.
The evaluation team is committed to ensuring that the entire process is responsive and helpful to the DHSC and NIHR, as well as other implementing partners, and will provide opportunities for feedback and interaction as needed. The team will maintain regular communications with key stakeholders throughout as appropriate and aim to be as inclusive as possible, while acknowledging the need to balance engagement with potential overburdening with demands for data and time. The team will ensure an ethical approach to this evaluation and adhere to relevant international guidelines. All data will be securely stored and managed.
This Inception Report presents the proposed approach to this evaluation – detailing the conceptual framework, evaluation questions, methodology for data collection and analysis, reporting and dissemination.
1.0 NIHR GHR Portfolio and Introduction to the Evaluation
1.1 Overview of the GHR Portfolio
The Global Health Research (GHR) portfolio was established by the Department of Health and Social Care (DHSC) with the National Institute for Health and Care Research (NIHR), following the publication of the 2015 UK Aid Strategy. It aims to directly address the diverse health needs of people in low- and middle-income countries (LMICs) by supporting high quality applied health research and training in areas where there is an unmet need; generating evidence for the direct benefit of people in LMICs; and strengthening LMIC and UK research capabilities and expertise in global health. As part of the 2015 spending review, an initial budget of £430 million was allocated for the first phase (2016-2021) of the GHR portfolio to contribute to improvement in global health outcomes through applied global health research, complementing other Official Development Assistance (ODA) research programmes such as the Global Challenges Research Fund and the Newton Fund. Since its launch in 2016, and as of September 2022, the portfolio has grown to include 17 thematic areas, with research activities being conducted in over 50 LMICs across Africa, Asia and Latin America, and through more than 20 delivery partners. In Phase 1, the GHR portfolio funded more than 650 awards which ranged from £5,000 to several million pounds[1]. In addition to being ODA-eligible, all the global health research funded by GHR must be underpinned by the following principles:
- Builds on NIHR’s operating principles of impact, excellence, effectiveness, inclusion and collaboration; and,
- Strengthens research capability and training through equitable partnerships.
When the portfolio was launched in 2016, DHSC initially prioritised working with partners who were already well established in the global health research space, such as the Medical Research Council (MRC) and the Wellcome Trust. NIHR rapidly built up its internal capacity in managing global health research programmes, growing from a small core team of only three people to a much more extensive team today. Over time NIHR principles that govern all work, whether domestic or global, have also been embedded in the partnerships where feasible providing extra added value of working in partnership together, and extending the influence of NIHR on other funders and activities. More detail behind the story of the evolution of the GHR portfolio will be gathered in the next phase of the evaluation. Given the rapid evolution of the portfolio, this evaluation provides an opportunity for learning, and to inform the second phase and subsequent phases of the portfolio.
Table 1 GHR Portfolio Summary
NIHR-delivered
- GHR Groups, NETSCC, £90m
- GHR Units, NETSCC, £84m
- Research & Innovation for Global Health Transformation (RIGHT), CCF, £54m
- Global Health Policy and Systems research (HPSR) Development Awards, NETSCC, £17m
- Professorships, NIHR Academy, £9.5m
- Financial Management Guidance for Awards (FAF), NIHR Academy, N/A
- Global Health Research Short Placement Award for Research Collaboration (SPARC), NIHR Academy, N/A
- European and Developing Countries Clinical Trials Partnership (EDCTP)[2], EDCTP, £79m
- Joint Global Health Trials Initiative (JGHTI)[3], MRC, £32m
- Coalition for Epidemic Preparedness Innovations (CEPI), FCDO, £20m
- Research for Health in Humanitarian Crises (R2HC)[4], ELRHA/Save the Children UK, £8m
- Diagnostics, Prosthetics and Orthotics to Tackle Health Challenges in Developing Countries, EPSRC, £7m
- Medicines for Malaria Venture (MMV), MMV, £6m
- Anti-Microbial Resistance (AMR) in a Global Context, MRC, £6m
- Research to Improve Adolescent Health in Low- and Middle-Income Country (LMIC)[5], MRC, £6m
- Global Road Safety Facility (GRSF), World Bank, £5m
- Structured Operational Research and Training Initiative on building, sustainable operational research capacity on AMR in LMICs (AMR SORT-IT), WHO, £5m
- Global Alliance TB Drug Development, TB Alliance, £4.5m
- Global Antibiotic Research and Development Partnership’s Neonatal Sepsis Programme (GARDP), GARDP, £4m
- NIHR-Wellcome Global Health Research Partnership, Wellcome, £4m
- Antimicrobial Resistance (AMR) Cross-Council Initiative: Behaviour Within and Beyond the Healthcare Setting, ESRC, £3m
- Grand Challenges Canada (GCC) Global Mental Health, GCC, £2.5m
- Global Effort on COVID-19 (GECO) Health Research, MRC, £2m
- Global Alliance for Chronic Diseases (GACD)[6], MRC, £2m
- Global Patient Safety Collaborative (GPSC), WHO, £1.5m
- Foundation for Innovative New Diagnostics (FIND), FIND, £1m
- Biomedical Resources Grant, Wellcome, £0.9m
- Royal Society of Tropical Medicine and Hygiene (RSTMH) Small grants scheme[7], RSTMH, £0.75m
- Good Financial Grants Practice (GFGP), AAS, £0.5m
- Global Maternal and Neonatal Health, MRC, £0.45m
1.2 Evaluation Objectives and Scope
Ecorys has been contracted by DHSC to assess the suitability of the design and implementation of the first phase of the GHR portfolio (2016/17-2020/21) in achieving its intended outcomes and impacts as set out in GHR’s Theory of Change (ToC). The evaluation will contribute to the evidence base on health research impact, as well as building evidence and understanding of a) the extent to which researchers are engaged in effective knowledge mobilisation (and what types of support or interventions encourage the transfer of knowledge), and b) the extent to which community engagement and involvement (CEI) meaningfully and sustainably leads to higher quality research and intended impacts.
The ToC has a long-term perspective, acknowledging that it may require 10-25 years for changes in policy, practice and behaviour (outcomes) to contribute towards strengthened health systems and increased individual and community capacity for health promotion and disease prevention (impacts). The evaluation of the first phase of the portfolio takes into account this long-term nature of change and, therefore, will assess evidence that the portfolio is being delivered as expected and whether or not key assumptions are holding true. A core objective of the evaluation is to generate real-time learning to support implementation and inform development and delivery of subsequent phases of the portfolio. The evaluation will include a strategic assessment of VFM at the portfolio level in terms of allocative efficiency, technical efficiency and value/results. Equity, gender equality and social inclusion will be considered key cross cutting principles and integrated and addressed throughout the evaluation.
In scope for this evaluation are all GHR portfolio programmes and their awards that started during the Phase 1 timeframe and ideally completed by the second quarter of 2022. However, some programmes have no or very few awards that were completed by Q2 2022 (e.g. Professorships, RIGHT), therefore, the evaluation will also review some active awards to ensure coverage across the entire portfolio.
2.0 Evaluation Approach
2.1 Evaluation Conceptual Framework
2.1.1 GHR Theory of Change The GHR portfolio Theory of Change (ToC) produced collaboratively by the DHSC GHR Team, NIHR Coordinating Centres and other strategic partners and award holders provides the conceptual framework for this evaluation. The GHR ToC visually represents how the portfolio’s funded activities and outputs are intended to contribute to long-term positive changes in health and health systems.
A ToC workshop (see Annex 2) was organised with portfolio stakeholders during inception in order to develop an agreed and improved understanding of all elements of the NIHR GHR ToC, to further refine the evaluation questions and scope. During the workshop, it was agreed that this simplified diagram of expected results from inputs and activities to outputs, outcomes and impact remains broadly valid. However, since it was developed, the initial concept of three activity strands, namely, Programmes and Partnerships with an overlapping strand of People, has evolved over time. The Programmes strand which refers to NIHR-commissioned research calls and the Partnerships strand which refers to collaborations with other funders are the two delivery mechanisms. The People strand refers to training and development of researchers and research support staff. This does not quite capture the capacity strengthening approach that is embedded in all programmes and is a key principle across the portfolio. The evaluation will seek to determine if there is a more appropriate way to reflect the GHR programme delivery in the ToC.
It is anticipated that the GHR portfolio’s short-term contribution to producing high quality relevant research outputs, supporting capacity building, facilitating partnerships and networks, and supporting dissemination and knowledge exchange will influence policy making and practice in the medium term. The ToC identifies these anticipated causal links drawing on underpinning theory from literature on health research impact, including the uptake of research evidence to improve the responsiveness of health systems to population needs and improved use of evidence to support capacity for health promotion and disease prevention. Outcomes are dependent on a wide range of external factors, which are captured in the ToC narrative as a set of assumptions. These assumptions have been unpacked in the evaluation framework so that they can be examined and tested. Some of them have been affected by the COVID-19 pandemic and other stresses on the UK economy, for example, the assumption that ‘global health research funders continue funding at present rate’. The evaluation team will review all assumptions during the first year of assessment along with the flow diagram and suggestions for amendments will be made where appropriate.
The ToC has been used to inform the Evaluation Framework (EF) (Section 2.2) and will be used to structure a theory-based approach to assessing the portfolio’s outcomes and impact. This approach is appropriate for complex interventions in complex environments. Specifically, the evaluation will use Contribution Analysis (CA), a theory-based evaluation methodology, to build evidence of whether and how the GHR portfolio is contributing to its intended outcomes and impacts.
Contribution Analysis is well-suited to meeting the evaluation objectives and answering the evaluation questions, given the time and resources available. Specifically, the choice of this method is informed by the following considerations:
- CA assumes there are complex processes at play in achieving any outcome, which is well-suited to a programme with multiple funding and partnering models.
- CA is particularly useful when the evaluated programme already has a clearly articulated theory of change, including assumptions linking outputs, outcomes, and impact.
- CA provides a methodology for assessing causality in evaluations where a comparison or baseline would not be appropriate or has not been established prior to the start of programming.
- CA is useful when evaluations are intended to support internal learning as it supports the development of lessons learned rather than exclusively focusing on showcasing success and accountability
2.1.2 Approach to Contribution Analysis
The CA method and analytical tools provide a way to structure and build up evidence about GHR portfolio activities and their potential contribution to supporting long-term change in health and health systems. The evaluation team will use the ToC to gather evidence and draw conclusions about the relevance and coherence of GHR-supported activities and whether and how different programmes are achieving their intended results, whether these are being implemented in line with key principles that underpin a process of sustainable long-term change (e.g. supporting capacity strengthening, equitable research partnerships), and key factors influencing this.
Practically, the CA methodology facilitates the structuring of evidence during each of the evaluation phases outlined below. First, the evaluation will collate and collect evidence about observed changes related to the portfolio’s activities in the short term, exploring the processes and causal pathways through which these have occurred and identify emerging evidence of progress towards influence on policy making and practice (i.e. expected medium term outcomes). Second, the evaluation will assess the role played by the GHR funded activity and wider influence linked to partnership working, and the role of other factors in hindering or enabling positive change. This systematic approach will provide a foundation for the evaluation to report on evidence of the GHR portfolio’s contribution to outcomes and impact under the first phase and generate learning to inform future phases.
Figure 1 Approach to Contribution Analysis
Inception Phase
- Work closely with DHSC and NIHR to finalise Evaluation Questions (EQs) and articulate a testable framework for analysis against the GHR ToC.
- Interviews/focus groups will focus on gathering and mapping evidence to verify or challenge the evidence and causal pathways identified through the ToC and inception phase.
- Analysis of existing evidence will continue through ongoing review of (new) documentation and interview data, producing a contribution st
Cited by
More filters
Oswaldo Cruz Foundation1, Joint United Nations Programme on HIV/AIDS2, PATH3, Indian Council of Medical Research4, Rockefeller Foundation5, Council for Scientific and Industrial Research6, United Nations7, University of Cape Town8, George Washington University9, Bill & Melinda Gates Foundation10, Boston University11, Arizona State University12, Universidade Federal do Rio Grande do Sul13, University of Oxford14, Seoul National University15, Global Forum for Health Research16, Concept Foundation17, World Health Organization18, University of Toronto19, Korea Institute for International Economic Policy20
TL;DR: A complementary and increasingly important means to improve health equity: the growing ability of some developing countries to undertake health innovation is highlighted.
Abstract: Gross inequities in disease burden between developed and developing countries are now the subject of intense global attention. Public and private donors have marshaled resources and created organizational structures to accelerate the development of new health products and to procure and distribute drugs and vaccines for the poor. Despite these encouraging efforts directed primarily from and funded by industrialized countries, sufficiency and sustainability remain enormous challenges because of the sheer magnitude of the problem. Here we highlight a complementary and increasingly important means to improve health equity: the growing ability of some developing countries to undertake health innovation.
204 citations
TL;DR: To maximize the potential of this phenomenon for global health, countries and donors need to link two disparate schools of thought: a search for technological solutions exemplifying by global public-private product development partnerships, and a focus on systemic solutions exemplified by health policy and systems research.
Abstract: This paper highlights the growing capacity for innovation in some developing countries. To maximize the potential of this phenomenon for global health, countries and donors need to link two disparate schools of thought: (1) a search for technological solutions exemplified by global public-private product development partnerships, and (2) a focus on systemic solutions exemplified by health policy and systems research. A strong capacity for both technological and social innovation in developing countries represents the only truly sustainable means of improving the effectiveness of health systems. Local public-private research and development partnerships, implementation research, and individual leadership are needed to achieve this goal.
115 citations
01 Jan 2004
108 citations
Journal Article•
TL;DR: The objective of this study is to examine the extend to which the flexibilities contained in the WTO agreement on TRIPS have been incorporated into the legislation of developing countries and the extent of the actual use for public health purposes.
Abstract: This study has the objective to: 1) examine the extend to which the flexibilities contained in the WTO agreement on TRIPS have been incorporated into the legislation of developing countries and the extent of the actual use for public health purposes; 2) review the stated trade policies of major industrialized countries, particularly the US and the EU and3) examine the practical effect and implications of recently concluded bilateral and regional FTAs for public health protection in developing countries.
96 citations
TL;DR: In this article, the authors examine the relationship between intellectual property and public health, with a focus on the extension of AIDS treatment in the developing world, while most of the literature on IP and health examines the conditions affecting poor countries' capacities to acquire essential medicines.
Abstract: This article examines the relationship between intellectual property (IP) and public health, with a focus on the extension of AIDS treatment in the developing world While most of the literature on IP and health examines the conditions affecting poor countries’ capacities to acquire essential medicines, I show the distinct—and more complicated—political economy of production and supply IP regulations alter the structure of generic pharmaceutical sectors in the countries capable of supplying essential medicines, and changes in market structure affect actors’ economic and political interests and capacities These new constellations of interests and capacities have profound implications for the creation and maintenance of political coalitions in support of on-going drug supply The result is that the global AIDS treatment campaign becomes marked by mismatches of interests and capacities: those actors capable of taking the economic, legal, and political steps necessary to increase the supply and availability of essential drugs have diminished interest in doing so, and those actors with an interest in expanding treatment may lack the capacities to address the problem of undersupply By focusing centrally on actors’ interests in and capacities for economic and political action, the article restores political economy to analysis of an issue-area that has been dominated by attention to international law And by examining the fragility of the coalitions supporting the production and supply of generic drugs, the article points to the limits of transnational activist networks as enduring agents of change
78 citations