Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health (and why it matters)

Tom Drake; Y-Ling Chi; Alec Morton; Catherine Pitt

doi:10.12688/f1000research.131230.2

Home Browse Why cost-effectiveness thresholds for global health donors should...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Opinion Article

Revised

Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health (and why it matters)

[version 2; peer review: 2 approved]

Tom Drake ¹, Y-Ling Chi¹, Alec Morton², Catherine Pitt³

PUBLISHED 18 Jan 2024

Author details Author details

¹ Department of Global Health, Centre for Global Development, London, UK
² Strathclyde Business School, University of Strathclyde, Strathclyde, UK
³ Department of Global Health and Development, London School of Hygiene and Tropical Medicine, London, UK

Tom Drake
Roles: Conceptualization, Writing – Original Draft Preparation

Y-Ling Chi
Roles: Writing – Original Draft Preparation

Alec Morton
Roles: Writing – Original Draft Preparation

Catherine Pitt
Roles: Writing – Original Draft Preparation

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the Health Services gateway.

Abstract

Healthcare cost-effectiveness analysis is increasingly used to inform priority-setting in low- and middle-income countries and by global health donors. As part of such analyses, cost-effectiveness thresholds are commonly used to determine what is, or is not, cost-effective. Recent years have seen a shift in best practice from a rule-of-thumb 1x or 3x per capita GDP threshold towards using thresholds that, in theory, reflect the opportunity cost of new investments within a given country. In this paper, we observe that international donors face both different resource constraints and opportunity costs compared to national decision-makers. Hence, their perspective on cost-effectiveness thresholds must be different. We discuss the potential implications of distinguishing between national and donor thresholds and outline broad options for how to approach setting a donor-perspective threshold. Further work is needed to clarify healthcare cost-effectiveness threshold theory in the context of international aid and to develop practical policy frameworks for implementation.

Keywords

Global health, health financing, aid, priority-setting, cost-effectiveness, threshold

Corresponding author: Tom Drake

Competing interests: No competing interests were disclosed.

Grant information: TD and YLC and supported by a grant for the International Decision Support Initiative (IDSI) from the Bill and Melinda Gates Foundation (grant number: OPP1202541).

Copyright: © 2024 Drake T et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Drake T, Chi YL, Morton A and Pitt C. Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health (and why it matters) [version 2; peer review: 2 approved]. F1000Research 2024, 12:214 (https://doi.org/10.12688/f1000research.131230.2) First published: 27 Feb 2023, 12:214 (https://doi.org/10.12688/f1000research.131230.1) Latest published: 18 Jan 2024, 12:214 (https://doi.org/10.12688/f1000research.131230.2)

Revised Amendments from Version 1

With thanks to the reviewers, the revised version of the article has several minor changes that have improved the clarity of our proposal. There remains much to be done to fully develop practical methods for donor decision-making using cost-effectiveness analysis. We hope we have outlined some of the theoretical and practical reasons for doing so and highlighted starting points for further work.

See the authors' detailed response to the review by Jessica Ochalek and Karl Claxton
See the authors' detailed response to the review by Chris Sampson

Introduction

To maximise population health for the resources available and accelerate progress towards universal health coverage, health systems must make use of evidence to identify which interventions and services to prioritise for investment. The last two decades have seen increasing use of evidence-informed priority-setting to guide resource allocation decisions in many low- and middle-income countries (LMICs). In 2014, the World Health Assembly Resolution WHA67.23 urged countries to consider the use of Health Technology Assessment (HTA) to inform a range of resource allocation decisions from coverage of medicines in formularies to inclusions in benefits packages.¹ The use of formal priority-setting processes and methods has intensified in the past decade.² A cost-effectiveness threshold is a decision-rule that can be used alongside a cost-effectiveness analysis (CEA) to determine whether an intervention’s incremental cost-effectiveness ratio (ICER) – that is, the ratio of the additional costs and benefits of an intervention, compared to the next-best alternative – indicates that it would constitute an efficient (i.e., cost-effective) use of scarce resources in a given context. Cost-effectiveness thresholds have gained greater prominence in academic and policy circles in recent years; however, the use of such thresholds is still nascent in many LMICs, in some cases due to the lack of formal processes and institutions to guide coverage decisions.³

This push for the use of CEA and cost-effectiveness thresholds at the national level stands in contrast to methods applied by donors to inform the allocation of Development Assistance for Health (DAH). In recent years, annual DAH has stabilised at around $40bn ($54bn in 2020 including Covid) and represents about a quarter of health spending in low-income countries.⁴^,⁵ While some DAH supports research, advocacy, or other catalytic activities, the vast majority supports service delivery, including both financing for specific services (which we focus on in this article) and health system strengthening such as infrastructure investments.⁴ While many donors have adopted Value for Money (VfM) frameworks⁶^,⁷^,⁸ or other forms of assessments, such as cash benchmarking,⁹ to our knowledge, few rely substantially on CEA to prioritise their funding allocation to programmes or between countries. The use of explicit cost-effectiveness thresholds by donors is even rarer; with the exception of foundations from the effective altruism movement (i.e., Givewell¹⁰ and Open Philanthropy¹¹), we have found none. When such methods are applied, they typically do not consider investments from national decision-makers, which can create issues of fragmentation, lack of alignment with national priorities, displacement of national funds, and duplication of investments.¹²

There is much debate over what cost-effectiveness thresholds are meant to represent.¹³^–¹⁵ For country thresholds, an emerging consensus is that the threshold should: i) reflect national resource availability, and ii) in application, be equal to the opportunity cost of alternative marginal healthcare spending.¹⁴^–¹⁶ Two papers from Woods et al.,¹⁷ and Ochalek et al.,¹⁸ provide initial estimates of national thresholds for 182 countries based on this “supply-side” perspective. Some health economists have suggested that donors should align with national cost-effectiveness thresholds.¹⁹

In this piece, we outline the case for distinguishing between donor and national cost-effectiveness thresholds, both in terms of the theoretical basis for the threshold and the potential benefits of clearer separation. We seek to build on and complement existing conversations on the use of cost-effectiveness thresholds in LMICs.¹⁶^–²³ Our aim is to encourage donors towards actions that promote health maximisation across LMICs, while recognising the importance of other public health goals such as equity and aid effectiveness principles such as country ownership and donor alignment.²⁴

Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health

Donors may distribute DAH to pursue various objectives, which may include reducing preventable mortality and morbidity, protecting their own country’s health (e.g., by reducing global burden of infectious disease) or fostering wider political and economic national interests.²⁵^–²⁷ Where health maximisation is at least one of the purposes of a donor’s DAH, and some form of cost-effectiveness thresholds is therefore potentially relevant, then we argue that the adoption of a separate threshold to national decision-makers becomes useful for the following reasons:

i) Different resource constraints. National decision-makers face budget constraints which reflect their country’s overall resources; the extent to which their government is able to mobilise those resources; and the prioritisation of health within national budgets. Donors, by contrast, face wholly separate resource constraints. Country donors’ budgets are constrained by their country’s resources and prioritisation of (health) aid to other countries, while private and multilateral donors’ budgets are constrained by their ability to raise revenue from government and private contributors globally. If cost-effectiveness thresholds reflect the scarcity of resources, then the differing resource constraints imply that differing thresholds should be used.
ii) Different opportunity costs. Unlike national institutions, which must prioritise their investments within a single country, global health donors may choose between support for health services across many countries. Therefore, a donor’s opportunity cost of investing in intervention A in country X is not only intervention B in country X, but also intervention C in country Y. For example, while investing in Covid vaccines for the over 60’s in Kenya may represent good value compared with alternative investments in Kenya, it may be more cost-effective to support the roll-out of bed nets in Malawi.

Why does this matter?

The lack of clarity around differences in donor and national cost-effectiveness thresholds is indicative of the lack of clarity in the decision perspectives and the roles that different actors have in funding healthcare in LMICs. Despite global health financing being a multi-billion-dollar sector where rhetoric on evidence-informed priority-setting is commonplace, many donors lack a clear framework for prioritisation. Collaboration between donors and national institutions in countries which receive DAH is often complex, political, and constantly negotiated for both donors and countries. The result is a fragmented system of financial support that impedes national health leaders in their work to develop an efficient and effective health system.²⁸

At the country level, DAH can alleviate local resource constraints and increase the fiscal space for health and thereby support the provision of health services that otherwise would not be possible. While this often takes the form of “plugging the gaps”, as donors see them, an efficiently allocated package of health services run by countries would imply a certain (hypothetical) cost-effectiveness threshold and additional donor funding would imply an (equally hypothetical) threshold that is higher than the country threshold.

The application of separate cost-effectiveness thresholds that reflect the perspective of decision-makers and donors can help to clarify the roles and responsibilities of national vs international funders of health services in LMICs; in other words, it would create a structure for who-should-fund-what. National institutions could design and fund a cohesive core package of the most-cost-effective services up to their national thresholds and “invite” donors to support a top-up package of the next-most-cost-effective services (see Figure 1). Each service – or book in the bookshelf metaphor – could be defined either at the level of the national population or for specific subpopulations for whom the cost-effectiveness differs because of high need, geographical inaccessibility, or other factors. The role of DAH would therefore be auxiliary: donors would fund interventions above the national threshold, up to their own threshold (we will discuss what this would look like below). In other words, the national cost-effectiveness threshold would represent a ceiling for a national payers and a floor for donors, below which they would not seek to fund activities in that country.

Figure 1. National and donor cost-effectiveness thresholds using the bookshelf metaphor.

This approach could address some of the greatest challenges in global health financing.¹² First, it could focus national resources towards funding a core package of the most essential services, which could ensure that funding for the provision of key services is not affected by aid volatility. Second, fragmentation of health financing²⁹^,³⁰ (and resulting duplication) could be reduced by a clearer separation of funding responsibilities. Further, the application of separate cost-effectiveness thresholds can avoid displacement of domestic resources by aid. Greater prioritisation of health interventions from a national perspective could be achieved, especially in designing a core package of most essential services – which can maximise the impact of overall health funding, rather than of funding streams operating in silos.

The approach would also empower national institutions to set their own priorities, rather than needing to work within the complex and fragmented financing space created by ad hoc donor support. At present, a significant share of health prioritisation is de facto done in donor headquarters and does not necessarily reflect national priorities, particularly in countries where DAH constitutes a large share of total health expenditure. In this framework, as the domestic health budget and cost-effectiveness thresholds increase, health aid is naturally crowded out. Transition from aid or the ending of specific aid programmes does not disrupt the provision of the most cost-effective services. Such transitions to domestic financing will nonetheless continue to require careful design and implementation – possibly including technical assistance and infrastructure investment – to ensure that they are smooth and do not disrupt the delivery of the next-most-cost-effective services. Further work is needed to consider the public financial management implications of on- or off-budget support using this framework.

Beyond reforms to within-country resource allocation, clearer frameworks for evidence-informed prioritisation could help donors equitably and effectively prioritise investments between countries. A clear donor cost-effectiveness threshold would promote the concentration of funding from global health donors in the programmes and contexts in which the greatest health gains can be made with the resources available. This approach is consistent with the ethical position that all health gains should be valued equally, regardless of where occur and how they are produced.

Options for setting thresholds

How might cost-effectiveness thresholds be set to reflect those two decision perspectives?

For national thresholds, methodological approaches to setting a threshold have been discussed extensively elsewhere.¹⁶^,³¹ In 1993, the World Bank suggested income-group-specific thresholds of US$50-200 per disability-adjusted life-year (DALY) averted.³² These were superseded by the World Health Organisation’s Choosing Interventions that are Cost-Effective (WHO-CHOICE) programme, which suggested that interventions with an ICER below 3x or 1x gross domestic product (GDP) per capita could be considered “cost-effective” or “highly cost-effective”, respectively.²³ The use of WHO-CHOICE thresholds is now being discouraged because they appear to be too high and do not adequately reflect the resource limitations of LMICs. Indeed, WHO health economists note they were never intended to be used in the way they often were.¹⁴^,³³ Recently, health economists have sought to clarify the theory underpinning thresholds, linking it to resource availability and local opportunity costs; with a set of estimates produced for LMICs in two papers.¹⁷^,¹⁸ However, only a few countries have defined an explicit cost-effectiveness threshold – for example, the UK and Thailand have, while Austria and Kenya have not.³⁴

In contrast, there is almost no literature discussing what a cost-effectiveness threshold could look like from a donor perspective. Drake (2014) outlines a case for a minimum DALY value to guide donor prioritisation.²⁰ Morton et al., (2017) describe an approach towards subsidising and crowding-in services which are just cost-ineffective from a national perspective, but does not address donor-perspective thresholds.²¹ GiveWell uses a benchmark that charity programmes should be expected to provide value at least 10 times greater than cash transfers³⁵ and Open Philanthropy requires a 1000-fold expected return on investment for their (generally higher risk and upstream) investments.¹¹

In attempting to set a threshold from a donor perspective, a first question is whether global health donors would all follow a single cost-effectiveness threshold or develop their own, agency-specific threshold. It is tempting to view each donor as having its own decision perspective and institutional mandate and therefore its own threshold. Use of numerous donor-specific thresholds could retain the advantage of improving the efficiency of each donor’s allocation between contexts, but the key challenges to coordination with national institutions and other donors would remain. To realise the benefits of improved donor harmonisation, a shared donor threshold is necessary. This threshold could be jointly agreed between donors at a country-level forum, such as a Sector Wide Approach (SWAp), which would facilitate the benefits of country-level harmonisation, but would lose the benefits of between-country resource allocation. A generalised global threshold would be required to achieve both within- and between-country benefits.

In broad terms, how could a global health donor threshold be set?

Option 1: Notional. Many countries and organisations use CEA to guide healthcare prioritisation without formally defining a cost-effectiveness threshold. One option is for donors to use the theoretical possibility of a separate cost-effectiveness threshold to shape policy and to clarify roles with regards to national decision-makers, without quantifying the threshold itself.

Option 2: Supply-side. Supply-side estimation means linking the threshold to the resources available and what they currently achieve in health production, at the margin. That is, a new investment opportunity should be more cost-effective than the next-best alternative that additional funding could support instead. If a cost-effectiveness threshold should reflect the payer’s opportunity costs, then a donor’s threshold should reflect opportunity costs at the global level, and should therefore be the same across all countries in which the donor may consider investing. It may also be possible to use statistical analysis analogous to the techniques used for estimating healthcare opportunity cost at the country level for domestic finances³⁶ to estimate the opportunity cost of marginal health aid globally.

Option 3: Demand-side. In contrast to the resource-linked supply-side approach, a demand-side route to setting a donor-perspective threshold could mean defining an aspirational benchmark that relevant stakeholders agree on. For example, participants in a World Health Assembly could support an aspirational declaration that all countries should be able to provide services that produce health for up to $X per DALY averted. That is, a minimum DALY value above which services should be considered worthy of investment, regardless of affordability to the national healthcare provider. Such an approach bridges the philosophical position of right-to-health advocates and technical optimisation approaches of health economists by effectively placing a minimum value on health and therefore a right to services that can produce health for this minimum standard. The drawback of the aspirational target is that it may facilitate sub-optimal allocation decisions if the demand-side aspiration is radically different from the supply-side reality. However, an important advantage of such a threshold is that it would function not only as an optimisation tool, but an advocacy goal.

Conclusions

Economic analysis has huge potential to help donors maximise the achievement of their objectives in the face of the vast gap between need and the resources available. However, the current CEA paradigm was designed for within-country decision-making and has limited relevance to the very different environment faced by donors. In this article, we have argued that cost-effectiveness threshold(s) for global health donors should differ from thresholds for national institutions because they have different decision perspectives, budgets, and opportunity costs. We then explored some of the potential benefits of distinguishing explicitly between donor and national thresholds and briefly outlined the options for setting those thresholds. We acknowledge that the approach we propose will entail a major shift in the way donors operate by explicitly moving from maximising the direct impact and cost-effectiveness of their own investments, towards playing a supporting role to national decision-makers. There are also practical challenges in the application of this framework, including the absence of explicit national thresholds (or ‘threshold thinking’), lack of country processes and institutions to prioritise interventions and develop a core package of essential services, and the lack of cost-effectiveness evidence. Despite these challenges, developing an improved framework for priority-setting in countries where aid constitutes a substantial share of health financing could substantially strengthen health systems in those countries. For this reason, we call for further work to: i) advance methodological theory for national and donor collaboration on resource allocation, and ii) explore the political economy of such reforms.

Data availability

No data are associated with this article.

Acknowledgements

The authors are grateful to Tony Culyer, Peter Baker and David Bath for comments on earlier drafts of the manuscript.

References

1. World Health Assembly 67: Sixty-seventh World Health Assembly, Geneva, 19-24 May 2014: resolutions and decisions: annexes. World Health Organization; 2014 [cited 2022 Dec 21]. Report No.: WHA67/2014/REC/1. Reference Source
2. Bump JB: Global health aid allocation in the 21st century. Health Policy Plan. 2018 Feb 1; 33(suppl_1): i1–i3. PubMed Abstract | Publisher Full Text | Free Full Text
3. Teerawattananon Y, Painter C, Dabak S, et al.: Avoiding health technology assessment: a global survey of reasons for not using health technology assessment in decision making. Cost Eff. Resour. Alloc. 2021 Sep 22; 19(1): 62. PubMed Abstract | Publisher Full Text | Free Full Text
4. Micah AE, Cogswell IE, Cunningham B, et al.: Tracking development assistance for health and for COVID-19: a review of development assistance, government, out-of-pocket, and other private spending on health for 204 countries and territories, 1990–2050. Lancet. 2021 Oct 9; 398(10308): 1317–1343. PubMed Abstract | Publisher Full Text | Free Full Text
5. OECD Statistics: [cited 2023 Jan 26]. Reference Source
6. DFID’s approach to value for money (VfM): GOV.UK.[cited 2022 Dec 21]. Reference Source
7. Value for Money Technical Brief. 2019. The Global Fund. https://www.theglobalfund.org/media/8596/core_valueformoney_technicalbrief_en.pdf
8. Ensuring Value for Money. UNICEF. 2020. https://www.unicef.org/documents/ensuring-value-money
9. Committing to Cost-Effectiveness: USAID’s New Effort to Benchmark for Greater Impact. Center for Global Development|Ideas to Action.[cited 2022 Dec 21]. Reference Source
10. GiveWell’s Cost-Effectiveness Analyses|GiveWell.[cited 2023 Feb 5]. Reference Source
11. Technical Updates to Our Global Health and Wellbeing Cause Prioritization Framework - Open Philanthropy. Open Philanthropy. 2021 [cited 2022 Dec 21]. Reference Source
12. Drake T, Regan L, Baker P: Reimagining Global Health Financing: How Refocusing Health Aid at the Margin Could Strengthen Health Systems and Futureproof Aid Financial Flows. Washington DC: Centre for Global Development; 2023 Feb. Report No.: 285. Reference Source
13. Vallejo-Torres L, García-Lorenzo B, Castilla I, et al.: On the Estimation of the Cost-Effectiveness Threshold: Why, What, How? Value Health. 2016 Jul 1; 19(5): 558–566. PubMed Abstract | Publisher Full Text
14. Bertram MY, Lauer JA, De Joncheere K, et al.: Cost-effectiveness thresholds: pros and cons. Bull. World Health Organ. 2016 Dec 1; 94(12): 925–930. PubMed Abstract | Publisher Full Text | Free Full Text
15. Culyer AJ: Cost-effectiveness thresholds in health care: a bookshelf guide to their meaning and use. Health Econ. Policy Law. 2016 Oct; 11(4): 415–432. PubMed Abstract | Publisher Full Text
16. Chi YL, Blecher M, Chalkidou K, et al.: What next after GDP-based cost-effectiveness thresholds? Gates Open Res. 2020 Nov 30; 4: 176. PubMed Abstract | Publisher Full Text | Free Full Text
17. Woods B, Revill P, Sculpher M, et al.: Country-Level Cost-Effectiveness Thresholds: Initial Estimates and the Need for Further Research. Value Health. 2016 Dec 1; 19(8): 929–935. PubMed Abstract | Publisher Full Text | Free Full Text
18. Ochalek J, Lomas J, Claxton K: Estimating health opportunity costs in low-income and middle-income countries: a novel approach and evidence from cross-country data. BMJ Glob. Health. 2018 Nov 1; 3(6): e000964. PubMed Abstract | Publisher Full Text | Free Full Text
19. Revill P, Ochalek J, Lomas J, et al.: Cost-Effectiveness Thresholds: Guiding Health Care Spending for Population Health Improvement. World Scientific Publishing Co. Pte. Ltd.: 2020 [cited 2022 Dec 21]; 75–97. Reference Source
20. Drake T: Priority setting in global health: towards a minimum DALY value. Health Econ. 2014 Feb; 23(2): 248–252. PubMed Abstract | Publisher Full Text
21. Morton A, Arulselvan A, Thomas R: Allocation rules for global donors. J. Health Econ. 2018 Mar; 58: 67–75. PubMed Abstract | Publisher Full Text
22. Leech AA, Kim DD, Cohen JT, et al.: Use and Misuse of Cost-Effectiveness Analysis Thresholds in Low- and Middle-Income Countries: Trends in Cost-per-DALY Studies. Value Health. 2018 Jul; 21(7): 759–761. PubMed Abstract | Publisher Full Text | Free Full Text
23. Kazibwe J, Gheorghe A, Wilson D, et al.: The Use of Cost-Effectiveness Thresholds for Evaluating Health Interventions in Low- and Middle-Income Countries From 2015 to 2020: A Review. Value Health. 2022 Mar 1; 25(3): 385–389. PubMed Abstract | Publisher Full Text | Free Full Text
24. Ogbuoji O, Yamey G: Aid Effectiveness in the Sustainable Development Goals Era. Int. J. Health Policy Manag. 2018; 8(3): 184–186. PubMed Abstract | Publisher Full Text | Free Full Text
25. Bendavid E, Ottersen T, Peilong L, et al.: Development Assistance for Health. In Disease Control Priorities: Improving Health and Reducing Poverty. Jamison DT, Gelband H, Horton S, et al. editors. 3rd ed.Washington (DC): The International Bank for Reconstruction and Development/The World Bank; 2017. Publisher Full Text http://www.ncbi.nlm.nih.gov/books/NBK525305/
26. Berthélemy J-C: Bilateral donors' interest vs. Recipients' development motives in aid allocation: do all donors behave the same?Cahiers de la Maison des Sciences Economiques bla05001, Université Panthéon-Sorbonne (Paris 1). 2005.
27. Gulrajani N, Calleja R: Understanding Donor Motivations: Developing the Principled Aid Index. ODI Working Paper; 2019.
28. Mladovsky P: Fragmentation by design: Universal health coverage policies as governmentality in Senegal. Soc. Sci. Med. 2020 Sep 1; 260: 113153. PubMed Abstract | Publisher Full Text
29. Spicer N, Agyepong I, Ottersen T, et al.: ‘It’s far too complicated’: why fragmentation persists in global health. Glob. Health. 2020 Jul 9; 16(1): 60. PubMed Abstract | Publisher Full Text | Free Full Text
30. Ihekweazu C: Lessons from Nigeria’s Adaptation of Global Health Initiatives during the COVID-19 Pandemic - Volume 28, Supplement—November 2022. Emerging Infectious Diseases journal - CDC [cited 2022 Dec 21]; S299–S301. PubMed Abstract | Publisher Full Text | , Free Full Text Reference Source
31. Santos AS, Guerra-Junior AA, Godman B, et al.: Cost-effectiveness thresholds: methods for setting and examples from around the world. Expert Rev. Pharmacoecon. Outcomes Res. 2018 Feb 27 [cited 2022 May 30]; 18: 277–288. PubMed Abstract | Publisher Full Text
32. The World Bank: World Development Report 1993: Investing in Health. Oxford University Press; 1993 [cited 2015 Nov 27]; 346. (World Development Report). Publisher Full Text
33. Bertram MY, Lauer JA, Stenberg K, et al.: Methods for the Economic Evaluation of Health Care Interventions for Priority Setting in the Health System: An Update From WHO CHOICE. Int. J. Health Policy Manag. 2021 Nov 1; 10(Special Issue on WHO-CHOICE Update): 673–677. Publisher Full Text
34. Schwarzer R, Rochau U, Saverno K, et al.: Systematic overview of cost-effectiveness thresholds in ten countries across four continents. J. Comp. Eff. Res. 2015 Sep; 4(5): 485–504. PubMed Abstract | Publisher Full Text
35. Elie: An update on GiveWell’s funding projections. The GiveWell Blog.2022 [cited 2023 Jan 23]. Reference Source
36. Claxton K, Martin S, Soares M, et al.: Methods for the Estimation of the NICE Cost Effectiveness Threshold. Health Technol. Assess. 2015 Feb 18; 19(14): 1–504. PubMed Abstract | Publisher Full Text | Free Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 27 Feb 2023

Author details Author details

Tom Drake
Roles: Conceptualization, Writing – Original Draft Preparation

Y-Ling Chi
Roles: Writing – Original Draft Preparation

Alec Morton
Roles: Writing – Original Draft Preparation

Catherine Pitt
Roles: Writing – Original Draft Preparation

Competing interests

No competing interests were disclosed.

Grant information

TD and YLC and supported by a grant for the International Decision Support Initiative (IDSI) from the Bill and Melinda Gates Foundation (grant number: OPP1202541).

Article Versions (2)

version 2

Revised

Published: 18 Jan 2024, 12:214

https://doi.org/10.12688/f1000research.131230.2

version 1

Published: 27 Feb 2023, 12:214

https://doi.org/10.12688/f1000research.131230.1

© 2024 Drake T et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Drake T, Chi YL, Morton A and Pitt C. Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health (and why it matters) [version 2; peer review: 2 approved]. F1000Research 2024, 12:214 (https://doi.org/10.12688/f1000research.131230.2)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 2

VERSION 2

PUBLISHED 18 Jan 2024

Revised

Views

Reviewer Report 27 Feb 2024

Jessica Ochalek, Centre for Health Economics, University of York, York, England, UK

Approved

https://doi.org/10.5256/f1000research.161475.r238873

I have ... Continue reading

CITE

Report a concern

Respond or Comment

Version 1

VERSION 1

PUBLISHED 27 Feb 2023

Views

Reviewer Report 01 Aug 2023

Chris Sampson, Office of Health Economics, London, England, UK

Approved

https://doi.org/10.5256/f1000research.144045.r183976

I see no reason why this opinion piece should not be indexed. There are no factual errors or essential omissions. However, there are several respects in which the article may be improved, and I would encourage the authors to consider the following suggestions for a revised version. My suggestions are - roughly - in order of importance.

1. The article proceeds on the assumption that health maximisation is the objective of (or at least an important consideration for) global health donors. It isn't clear to me that this is the case. This assumption requires i) clearer articulation as part of the authors' argumentation and ii) further discussion of its validity.

2. Even if health maximisation is the objective of global health donors, it isn't clear to me that the proposal summarised in Figure 1 provides a means of achieving this. It seems far more likely to me that global health donors might prefer to spend on infrastructural investments and not marginal technologies, in part because the former is more conducive to health maximisation. The authors should discuss the scenarios in which a donor's use of a threshold would and would not support health maximisation.

3. The authors should consider (and describe) the suitability of the bookshelf analogy. In the context of low-income countries - perhaps more so than others - the bookshelf analogy is problematic. In particular, the bookshelf analogy starts to fall apart when we allow for divisibility, which is certainly a relevant consideration in countries in which providers cannot reach all patients.

4. Some further consideration should be given (and discussion provided) to the claim that a "donor threshold should be higher". There are numerous assumptions inherent in this, and the authors have not described them. For instance, it seems to assume that resources are currently being allocated efficiently by national decision-makers. It behoves the authors to outline all of the assumptions in their claim.

5. Throughout the article, 'thresholds' are characterised as a matter of fact, an observable quantity. This is not correct; thresholds are a policymaking tool with a primarily normative basis. For instance, the title should read that cost-effectiveness thresholds *could* or *should* differ, rather than stating that they *do* differ.

6. The authors state that few donors rely on CEA. This raises the question of why we should care about cost-effectiveness thresholds at all! The authors should acknowledge the limited applicability and importance of their assertions in this context. Indeed, there needs to be more justification for the idea that donor cost-effectiveness thresholds should be specified at all.

7. The authors refer to the use of decision thresholds by Givewell and Open Philanthropy. It would be helpful to state these thresholds in the text.

8. The claim is made that "many donors" have adopted value for money frameworks, but then only one is cited. Either more citations should be added or the claim should be altered.

Is the topic of the opinion article discussed accurately in the context of the current literature?

Yes
Are all factual statements correct and adequately supported by citations?

Yes
Are arguments sufficiently supported by evidence from the published literature?

Partly
Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Health economics

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response 18 Jan 2024

Tom Drake, Department of Global Health, Centre for Global Development, London, UK

18 Jan 2024

Author Response

We thank the reviewer for this helpful review of the article. Below we offer point-by-point responses to the feedback using [R] to denote reviewer text and [A] for author text.
... Continue reading We thank the reviewer for this helpful review of the article. Below we offer point-by-point responses to the feedback using [R] to denote reviewer text and [A] for author text.

[A] We would like to thank the reviewer for his time in providing the feedback, which has helped us to improve and clarify the manuscript. We have taken on many of the comments and adjusted the article accordingly. Point-by-point responses are offered below.

[R] I see no reason why this opinion piece should not be indexed. There are no factual errors or essential omissions. However, there are several respects in which the article may be improved, and I would encourage the authors to consider the following suggestions for a revised version. My suggestions are - roughly - in order of importance.

1. The article proceeds on the assumption that health maximisation is the objective of (or at least an important consideration for) global health donors. It isn't clear to me that this is the case. This assumption requires i) clearer articulation as part of the authors' argumentation and ii) further discussion of its validity.
[A] Thank you for this helpful comment. To address this point, we have added a sentence that explicitly acknowledges the range of objectives that donors may pursue at the top of the section titled, “Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health”:
The section now reads: “Donors may distribute DAH to pursue various objectives, which may include reducing preventable mortality and morbidity, protecting their own country’s health (e.g., by reducing global infectious burden of disease) or fostering wider political and economic national interests. Where health maximisation is at least one of the purposes of a donor’s DAH, and some form of cost-effectiveness thresholds are therefore potentially relevant, then we argue that the adoption of a separate threshold to national decision-makers becomes useful for the following reasons:”
2. Even if health maximisation is the objective of global health donors, it isn't clear to me that the proposal summarised in Figure 1 provides a means of achieving this. It seems far more likely to me that global health donors might prefer to spend on infrastructural investments and not marginal technologies, in part because the former is more conducive to health maximisation. The authors should discuss the scenarios in which a donor's use of a threshold would and would not support health maximisation.
[A] We agree that investments like infrastructure are outside the scope of this commentary. To the extent that aid is directed towards health services (rather than health systems investments, global public goods and so on), we argue that conceptualising separate country and donor cost-effectiveness thresholds is theoretically clearer and could present opportunities for practical benefits. We have added a clarification in the second paragraph of the introduction to this effect.

The sentence reads “While some DAH supports research, advocacy, or other catalytic activities, the vast majority supports service delivery, including both financing for specific services (which we focus on in this article) and health system strengthening such as infrastructure investments”.

3. The authors should consider (and describe) the suitability of the bookshelf analogy. In the context of low-income countries - perhaps more so than others - the bookshelf analogy is problematic. In particular, the bookshelf analogy starts to fall apart when we allow for divisibility, which is certainly a relevant consideration in countries in which providers cannot reach all patients.

[A] We think that the bookshelf analogy remains relevant. It depends on how the “books” are specified. Additional sentence added: “Each service – or book in the bookshelf metaphor – could be defined either at the level of the national population or for specific subpopulations for whom the cost-effectiveness differs because of high need, geographical inaccessibility, or other factors.”

4. Some further consideration should be given (and discussion provided) to the claim that a "donor threshold should be higher". There are numerous assumptions inherent in this, and the authors have not described them. For instance, it seems to assume that resources are currently being allocated efficiently by national decision-makers. It behoves the authors to outline all of the assumptions in their claim.

[A] We have now revised the start of the section, “Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health” to articulate more clearly the different objectives donors may pursue and our assumptions about the relevance of health maximisation (albeit not as sole criterion) to decision-making. We certainly do not intend to claim that resources are currently efficiently allocated and our arguments do not depend on such an assumption. Instead, we are arguing that increased use of thresholds could support a transition to more efficient resource allocation. We have also modified our claim in this section to argue only that the differing resource constraints mean that donor and country thresholds should be “different”; we present the wider rationale for why donor thresholds should be higher in the following section, where we are able to outline the considerations more fully.

A revised section on why resource constraints differ now reads:
“Different resource constraints. National decision-makers face budget constraints which reflect their country’s overall resources; the extent to which their government is able to mobilise those resources; and the prioritisation of health within national budgets. Donors, by contrast, face wholly separate resource constraints. Country donors’ budgets are constrained by their country’s resources and prioritisation of (health) aid to other countries, while private and multilateral donors’ budgets are constrained by their ability to raise revenue from government and private contributors globally. If cost-effectiveness thresholds reflect the scarcity of resources, then the differing resource constraints imply that differing thresholds should be used.”

5. Throughout the article, 'thresholds' are characterised as a matter of fact, an observable quantity. This is not correct; thresholds are a policymaking tool with a primarily normative basis. For instance, the title should read that cost-effectiveness thresholds *could* or *should* differ, rather than stating that they *do* differ.

[A] Thank you for raising this important point. We agree and have adjusted the language throughout, including in the title of the article.

6. The authors state that few donors rely on CEA. This raises the question of why we should care about cost-effectiveness thresholds at all! The authors should acknowledge the limited applicability and importance of their assertions in this context. Indeed, there needs to be more justification for the idea that donor cost-effectiveness thresholds should be specified at all.

[A] CEA is used quite widely in several countries because the analytic community has done a good job of crafting analytic tools which are rooted in economic theory but which speak directly to the decisions faced by in-country decision makers and are contextually appropriate. We haven’t got there for donor aid yet, we would argue because the approach has been to carry over the in-country CEA paradigm rather than figuring out how to implement those economic principles in the donor environment. On prima facie basis, economic analysis, as the science of resource scarcity should have a greater role in the allocation of donor aid than it currently does. We hope that our commentary will encourage wider use of CEA and appropriate cost-effectiveness thresholds.
New text: “Economic analysis has huge potential to help donors maximise the achievement of their objectives in the face of the vast gap between need and the resources available. However, the current CEA paradigm was designed for within-country decision making and has limited read-across to the very different environment faced by donors.”

7. The authors refer to the use of decision thresholds by Givewell and Open Philanthropy. It would be helpful to state these thresholds in the text.

[A] We include the following in the text: “GiveWell uses a benchmark that charity programmes should be expected to provide value at least 10 time greater than cash transfers ²⁹and Open Philanthropy require a 1000-fold expected return on investment for their (generally higher risk and upstream) investments.” These specifics come somewhat later in the text than where these donors were first mentioned, as we felt it fit better with the main flow of the text.

8. The claim is made that "many donors" have adopted value for money frameworks, but then only one is cited. Either more citations should be added or the claim should be altered.

[A] Thank you for this suggestion. We have now added references to UNICEF and the Global Fund’s value-for-money frameworks, alongside the existing reference to the UK’s framework.
We thank the reviewer for this helpful review of the article. Below we offer point-by-point responses to the feedback using [R] to denote reviewer text and [A] for author text.

[A] We would like to thank the reviewer for his time in providing the feedback, which has helped us to improve and clarify the manuscript. We have taken on many of the comments and adjusted the article accordingly. Point-by-point responses are offered below.

[R] I see no reason why this opinion piece should not be indexed. There are no factual errors or essential omissions. However, there are several respects in which the article may be improved, and I would encourage the authors to consider the following suggestions for a revised version. My suggestions are - roughly - in order of importance.

1. The article proceeds on the assumption that health maximisation is the objective of (or at least an important consideration for) global health donors. It isn't clear to me that this is the case. This assumption requires i) clearer articulation as part of the authors' argumentation and ii) further discussion of its validity.
[A] Thank you for this helpful comment. To address this point, we have added a sentence that explicitly acknowledges the range of objectives that donors may pursue at the top of the section titled, “Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health”:
The section now reads: “Donors may distribute DAH to pursue various objectives, which may include reducing preventable mortality and morbidity, protecting their own country’s health (e.g., by reducing global infectious burden of disease) or fostering wider political and economic national interests. Where health maximisation is at least one of the purposes of a donor’s DAH, and some form of cost-effectiveness thresholds are therefore potentially relevant, then we argue that the adoption of a separate threshold to national decision-makers becomes useful for the following reasons:”
2. Even if health maximisation is the objective of global health donors, it isn't clear to me that the proposal summarised in Figure 1 provides a means of achieving this. It seems far more likely to me that global health donors might prefer to spend on infrastructural investments and not marginal technologies, in part because the former is more conducive to health maximisation. The authors should discuss the scenarios in which a donor's use of a threshold would and would not support health maximisation.
[A] We agree that investments like infrastructure are outside the scope of this commentary. To the extent that aid is directed towards health services (rather than health systems investments, global public goods and so on), we argue that conceptualising separate country and donor cost-effectiveness thresholds is theoretically clearer and could present opportunities for practical benefits. We have added a clarification in the second paragraph of the introduction to this effect.

The sentence reads “While some DAH supports research, advocacy, or other catalytic activities, the vast majority supports service delivery, including both financing for specific services (which we focus on in this article) and health system strengthening such as infrastructure investments”.

3. The authors should consider (and describe) the suitability of the bookshelf analogy. In the context of low-income countries - perhaps more so than others - the bookshelf analogy is problematic. In particular, the bookshelf analogy starts to fall apart when we allow for divisibility, which is certainly a relevant consideration in countries in which providers cannot reach all patients.

[A] We think that the bookshelf analogy remains relevant. It depends on how the “books” are specified. Additional sentence added: “Each service – or book in the bookshelf metaphor – could be defined either at the level of the national population or for specific subpopulations for whom the cost-effectiveness differs because of high need, geographical inaccessibility, or other factors.”

4. Some further consideration should be given (and discussion provided) to the claim that a "donor threshold should be higher". There are numerous assumptions inherent in this, and the authors have not described them. For instance, it seems to assume that resources are currently being allocated efficiently by national decision-makers. It behoves the authors to outline all of the assumptions in their claim.

[A] We have now revised the start of the section, “Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health” to articulate more clearly the different objectives donors may pursue and our assumptions about the relevance of health maximisation (albeit not as sole criterion) to decision-making. We certainly do not intend to claim that resources are currently efficiently allocated and our arguments do not depend on such an assumption. Instead, we are arguing that increased use of thresholds could support a transition to more efficient resource allocation. We have also modified our claim in this section to argue only that the differing resource constraints mean that donor and country thresholds should be “different”; we present the wider rationale for why donor thresholds should be higher in the following section, where we are able to outline the considerations more fully.

A revised section on why resource constraints differ now reads:
“Different resource constraints. National decision-makers face budget constraints which reflect their country’s overall resources; the extent to which their government is able to mobilise those resources; and the prioritisation of health within national budgets. Donors, by contrast, face wholly separate resource constraints. Country donors’ budgets are constrained by their country’s resources and prioritisation of (health) aid to other countries, while private and multilateral donors’ budgets are constrained by their ability to raise revenue from government and private contributors globally. If cost-effectiveness thresholds reflect the scarcity of resources, then the differing resource constraints imply that differing thresholds should be used.”

5. Throughout the article, 'thresholds' are characterised as a matter of fact, an observable quantity. This is not correct; thresholds are a policymaking tool with a primarily normative basis. For instance, the title should read that cost-effectiveness thresholds *could* or *should* differ, rather than stating that they *do* differ.

[A] Thank you for raising this important point. We agree and have adjusted the language throughout, including in the title of the article.

6. The authors state that few donors rely on CEA. This raises the question of why we should care about cost-effectiveness thresholds at all! The authors should acknowledge the limited applicability and importance of their assertions in this context. Indeed, there needs to be more justification for the idea that donor cost-effectiveness thresholds should be specified at all.

[A] CEA is used quite widely in several countries because the analytic community has done a good job of crafting analytic tools which are rooted in economic theory but which speak directly to the decisions faced by in-country decision makers and are contextually appropriate. We haven’t got there for donor aid yet, we would argue because the approach has been to carry over the in-country CEA paradigm rather than figuring out how to implement those economic principles in the donor environment. On prima facie basis, economic analysis, as the science of resource scarcity should have a greater role in the allocation of donor aid than it currently does. We hope that our commentary will encourage wider use of CEA and appropriate cost-effectiveness thresholds.
New text: “Economic analysis has huge potential to help donors maximise the achievement of their objectives in the face of the vast gap between need and the resources available. However, the current CEA paradigm was designed for within-country decision making and has limited read-across to the very different environment faced by donors.”

7. The authors refer to the use of decision thresholds by Givewell and Open Philanthropy. It would be helpful to state these thresholds in the text.

[A] We include the following in the text: “GiveWell uses a benchmark that charity programmes should be expected to provide value at least 10 time greater than cash transfers ²⁹and Open Philanthropy require a 1000-fold expected return on investment for their (generally higher risk and upstream) investments.” These specifics come somewhat later in the text than where these donors were first mentioned, as we felt it fit better with the main flow of the text.

8. The claim is made that "many donors" have adopted value for money frameworks, but then only one is cited. Either more citations should be added or the claim should be altered.

[A] Thank you for this suggestion. We have now added references to UNICEF and the Global Fund’s value-for-money frameworks, alongside the existing reference to the UK’s framework.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 18 Jan 2024

Tom Drake, Department of Global Health, Centre for Global Development, London, UK

18 Jan 2024

Author Response

We thank the reviewer for this helpful review of the article. Below we offer point-by-point responses to the feedback using [R] to denote reviewer text and [A] for author text.
... Continue reading We thank the reviewer for this helpful review of the article. Below we offer point-by-point responses to the feedback using [R] to denote reviewer text and [A] for author text.

[A] We would like to thank the reviewer for his time in providing the feedback, which has helped us to improve and clarify the manuscript. We have taken on many of the comments and adjusted the article accordingly. Point-by-point responses are offered below.

[R] I see no reason why this opinion piece should not be indexed. There are no factual errors or essential omissions. However, there are several respects in which the article may be improved, and I would encourage the authors to consider the following suggestions for a revised version. My suggestions are - roughly - in order of importance.

1. The article proceeds on the assumption that health maximisation is the objective of (or at least an important consideration for) global health donors. It isn't clear to me that this is the case. This assumption requires i) clearer articulation as part of the authors' argumentation and ii) further discussion of its validity.
[A] Thank you for this helpful comment. To address this point, we have added a sentence that explicitly acknowledges the range of objectives that donors may pursue at the top of the section titled, “Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health”:
The section now reads: “Donors may distribute DAH to pursue various objectives, which may include reducing preventable mortality and morbidity, protecting their own country’s health (e.g., by reducing global infectious burden of disease) or fostering wider political and economic national interests. Where health maximisation is at least one of the purposes of a donor’s DAH, and some form of cost-effectiveness thresholds are therefore potentially relevant, then we argue that the adoption of a separate threshold to national decision-makers becomes useful for the following reasons:”
2. Even if health maximisation is the objective of global health donors, it isn't clear to me that the proposal summarised in Figure 1 provides a means of achieving this. It seems far more likely to me that global health donors might prefer to spend on infrastructural investments and not marginal technologies, in part because the former is more conducive to health maximisation. The authors should discuss the scenarios in which a donor's use of a threshold would and would not support health maximisation.
[A] We agree that investments like infrastructure are outside the scope of this commentary. To the extent that aid is directed towards health services (rather than health systems investments, global public goods and so on), we argue that conceptualising separate country and donor cost-effectiveness thresholds is theoretically clearer and could present opportunities for practical benefits. We have added a clarification in the second paragraph of the introduction to this effect.

The sentence reads “While some DAH supports research, advocacy, or other catalytic activities, the vast majority supports service delivery, including both financing for specific services (which we focus on in this article) and health system strengthening such as infrastructure investments”.

3. The authors should consider (and describe) the suitability of the bookshelf analogy. In the context of low-income countries - perhaps more so than others - the bookshelf analogy is problematic. In particular, the bookshelf analogy starts to fall apart when we allow for divisibility, which is certainly a relevant consideration in countries in which providers cannot reach all patients.

[A] We think that the bookshelf analogy remains relevant. It depends on how the “books” are specified. Additional sentence added: “Each service – or book in the bookshelf metaphor – could be defined either at the level of the national population or for specific subpopulations for whom the cost-effectiveness differs because of high need, geographical inaccessibility, or other factors.”

4. Some further consideration should be given (and discussion provided) to the claim that a "donor threshold should be higher". There are numerous assumptions inherent in this, and the authors have not described them. For instance, it seems to assume that resources are currently being allocated efficiently by national decision-makers. It behoves the authors to outline all of the assumptions in their claim.

[A] We have now revised the start of the section, “Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health” to articulate more clearly the different objectives donors may pursue and our assumptions about the relevance of health maximisation (albeit not as sole criterion) to decision-making. We certainly do not intend to claim that resources are currently efficiently allocated and our arguments do not depend on such an assumption. Instead, we are arguing that increased use of thresholds could support a transition to more efficient resource allocation. We have also modified our claim in this section to argue only that the differing resource constraints mean that donor and country thresholds should be “different”; we present the wider rationale for why donor thresholds should be higher in the following section, where we are able to outline the considerations more fully.

A revised section on why resource constraints differ now reads:
“Different resource constraints. National decision-makers face budget constraints which reflect their country’s overall resources; the extent to which their government is able to mobilise those resources; and the prioritisation of health within national budgets. Donors, by contrast, face wholly separate resource constraints. Country donors’ budgets are constrained by their country’s resources and prioritisation of (health) aid to other countries, while private and multilateral donors’ budgets are constrained by their ability to raise revenue from government and private contributors globally. If cost-effectiveness thresholds reflect the scarcity of resources, then the differing resource constraints imply that differing thresholds should be used.”

5. Throughout the article, 'thresholds' are characterised as a matter of fact, an observable quantity. This is not correct; thresholds are a policymaking tool with a primarily normative basis. For instance, the title should read that cost-effectiveness thresholds *could* or *should* differ, rather than stating that they *do* differ.

[A] Thank you for raising this important point. We agree and have adjusted the language throughout, including in the title of the article.

6. The authors state that few donors rely on CEA. This raises the question of why we should care about cost-effectiveness thresholds at all! The authors should acknowledge the limited applicability and importance of their assertions in this context. Indeed, there needs to be more justification for the idea that donor cost-effectiveness thresholds should be specified at all.

[A] CEA is used quite widely in several countries because the analytic community has done a good job of crafting analytic tools which are rooted in economic theory but which speak directly to the decisions faced by in-country decision makers and are contextually appropriate. We haven’t got there for donor aid yet, we would argue because the approach has been to carry over the in-country CEA paradigm rather than figuring out how to implement those economic principles in the donor environment. On prima facie basis, economic analysis, as the science of resource scarcity should have a greater role in the allocation of donor aid than it currently does. We hope that our commentary will encourage wider use of CEA and appropriate cost-effectiveness thresholds.
New text: “Economic analysis has huge potential to help donors maximise the achievement of their objectives in the face of the vast gap between need and the resources available. However, the current CEA paradigm was designed for within-country decision making and has limited read-across to the very different environment faced by donors.”

7. The authors refer to the use of decision thresholds by Givewell and Open Philanthropy. It would be helpful to state these thresholds in the text.

[A] We include the following in the text: “GiveWell uses a benchmark that charity programmes should be expected to provide value at least 10 time greater than cash transfers ²⁹and Open Philanthropy require a 1000-fold expected return on investment for their (generally higher risk and upstream) investments.” These specifics come somewhat later in the text than where these donors were first mentioned, as we felt it fit better with the main flow of the text.

8. The claim is made that "many donors" have adopted value for money frameworks, but then only one is cited. Either more citations should be added or the claim should be altered.

[A] Thank you for this suggestion. We have now added references to UNICEF and the Global Fund’s value-for-money frameworks, alongside the existing reference to the UK’s framework.
We thank the reviewer for this helpful review of the article. Below we offer point-by-point responses to the feedback using [R] to denote reviewer text and [A] for author text.

[A] We would like to thank the reviewer for his time in providing the feedback, which has helped us to improve and clarify the manuscript. We have taken on many of the comments and adjusted the article accordingly. Point-by-point responses are offered below.

[R] I see no reason why this opinion piece should not be indexed. There are no factual errors or essential omissions. However, there are several respects in which the article may be improved, and I would encourage the authors to consider the following suggestions for a revised version. My suggestions are - roughly - in order of importance.

1. The article proceeds on the assumption that health maximisation is the objective of (or at least an important consideration for) global health donors. It isn't clear to me that this is the case. This assumption requires i) clearer articulation as part of the authors' argumentation and ii) further discussion of its validity.
[A] Thank you for this helpful comment. To address this point, we have added a sentence that explicitly acknowledges the range of objectives that donors may pursue at the top of the section titled, “Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health”:
The section now reads: “Donors may distribute DAH to pursue various objectives, which may include reducing preventable mortality and morbidity, protecting their own country’s health (e.g., by reducing global infectious burden of disease) or fostering wider political and economic national interests. Where health maximisation is at least one of the purposes of a donor’s DAH, and some form of cost-effectiveness thresholds are therefore potentially relevant, then we argue that the adoption of a separate threshold to national decision-makers becomes useful for the following reasons:”
2. Even if health maximisation is the objective of global health donors, it isn't clear to me that the proposal summarised in Figure 1 provides a means of achieving this. It seems far more likely to me that global health donors might prefer to spend on infrastructural investments and not marginal technologies, in part because the former is more conducive to health maximisation. The authors should discuss the scenarios in which a donor's use of a threshold would and would not support health maximisation.
[A] We agree that investments like infrastructure are outside the scope of this commentary. To the extent that aid is directed towards health services (rather than health systems investments, global public goods and so on), we argue that conceptualising separate country and donor cost-effectiveness thresholds is theoretically clearer and could present opportunities for practical benefits. We have added a clarification in the second paragraph of the introduction to this effect.

The sentence reads “While some DAH supports research, advocacy, or other catalytic activities, the vast majority supports service delivery, including both financing for specific services (which we focus on in this article) and health system strengthening such as infrastructure investments”.

3. The authors should consider (and describe) the suitability of the bookshelf analogy. In the context of low-income countries - perhaps more so than others - the bookshelf analogy is problematic. In particular, the bookshelf analogy starts to fall apart when we allow for divisibility, which is certainly a relevant consideration in countries in which providers cannot reach all patients.

[A] We think that the bookshelf analogy remains relevant. It depends on how the “books” are specified. Additional sentence added: “Each service – or book in the bookshelf metaphor – could be defined either at the level of the national population or for specific subpopulations for whom the cost-effectiveness differs because of high need, geographical inaccessibility, or other factors.”

4. Some further consideration should be given (and discussion provided) to the claim that a "donor threshold should be higher". There are numerous assumptions inherent in this, and the authors have not described them. For instance, it seems to assume that resources are currently being allocated efficiently by national decision-makers. It behoves the authors to outline all of the assumptions in their claim.

[A] We have now revised the start of the section, “Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health” to articulate more clearly the different objectives donors may pursue and our assumptions about the relevance of health maximisation (albeit not as sole criterion) to decision-making. We certainly do not intend to claim that resources are currently efficiently allocated and our arguments do not depend on such an assumption. Instead, we are arguing that increased use of thresholds could support a transition to more efficient resource allocation. We have also modified our claim in this section to argue only that the differing resource constraints mean that donor and country thresholds should be “different”; we present the wider rationale for why donor thresholds should be higher in the following section, where we are able to outline the considerations more fully.

A revised section on why resource constraints differ now reads:
“Different resource constraints. National decision-makers face budget constraints which reflect their country’s overall resources; the extent to which their government is able to mobilise those resources; and the prioritisation of health within national budgets. Donors, by contrast, face wholly separate resource constraints. Country donors’ budgets are constrained by their country’s resources and prioritisation of (health) aid to other countries, while private and multilateral donors’ budgets are constrained by their ability to raise revenue from government and private contributors globally. If cost-effectiveness thresholds reflect the scarcity of resources, then the differing resource constraints imply that differing thresholds should be used.”

5. Throughout the article, 'thresholds' are characterised as a matter of fact, an observable quantity. This is not correct; thresholds are a policymaking tool with a primarily normative basis. For instance, the title should read that cost-effectiveness thresholds *could* or *should* differ, rather than stating that they *do* differ.

[A] Thank you for raising this important point. We agree and have adjusted the language throughout, including in the title of the article.

6. The authors state that few donors rely on CEA. This raises the question of why we should care about cost-effectiveness thresholds at all! The authors should acknowledge the limited applicability and importance of their assertions in this context. Indeed, there needs to be more justification for the idea that donor cost-effectiveness thresholds should be specified at all.

[A] CEA is used quite widely in several countries because the analytic community has done a good job of crafting analytic tools which are rooted in economic theory but which speak directly to the decisions faced by in-country decision makers and are contextually appropriate. We haven’t got there for donor aid yet, we would argue because the approach has been to carry over the in-country CEA paradigm rather than figuring out how to implement those economic principles in the donor environment. On prima facie basis, economic analysis, as the science of resource scarcity should have a greater role in the allocation of donor aid than it currently does. We hope that our commentary will encourage wider use of CEA and appropriate cost-effectiveness thresholds.
New text: “Economic analysis has huge potential to help donors maximise the achievement of their objectives in the face of the vast gap between need and the resources available. However, the current CEA paradigm was designed for within-country decision making and has limited read-across to the very different environment faced by donors.”

7. The authors refer to the use of decision thresholds by Givewell and Open Philanthropy. It would be helpful to state these thresholds in the text.

[A] We include the following in the text: “GiveWell uses a benchmark that charity programmes should be expected to provide value at least 10 time greater than cash transfers ²⁹and Open Philanthropy require a 1000-fold expected return on investment for their (generally higher risk and upstream) investments.” These specifics come somewhat later in the text than where these donors were first mentioned, as we felt it fit better with the main flow of the text.

8. The claim is made that "many donors" have adopted value for money frameworks, but then only one is cited. Either more citations should be added or the claim should be altered.

[A] Thank you for this suggestion. We have now added references to UNICEF and the Global Fund’s value-for-money frameworks, alongside the existing reference to the UK’s framework.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 03 Apr 2023

Jessica Ochalek, Centre for Health Economics, University of York, York, England, UK

Karl Claxton, Centre for Health Economics, University of York, York, England, UK

Approved with Reservations

https://doi.org/10.5256/f1000research.144045.r166801

This opinion article aims to provide a neat and tidy solution to the problem of how donors should prioritise their funding for health both within and across countries.

The authors start with the assumption that donors should apply a different threshold than that which should be applied by national decision makers allocating government resources for health. The latter, we agree, should reflect the marginal productivity of the healthcare system. Where there is less agreement is around whether donors should use a separate threshold at all and, if they must, what that threshold should be based upon.

The authors rightly point out that the opportunity cost of investing in intervention A in country X is not only intervention B in country X, but also intervention C in country Y. One way to consider this is using the net health effect of the intervention within each country and across countries. (See Claxton, K. P., Ochalek, J. M., Revill, P., Rollinger, A. & Walker, D., Informing Decisions in Global Health: Cost Per DALY Thresholds and Health Opportunity Costs, Nov 2016, 4 p. Centre for Health Economics, University of York). Net health effects for each country are a function of the cost to the country (or the cost savings), health impacts and an estimate of health opportunity cost to the country of government expenditure on health. Summing these across countries gives the global net health impact. Once this is known, the donor can select what they regard as the best (based not just the net health effects but also on their distribution across countries).

It may be helpful to distinguish between general budget support and off-budget support. Where funding is provided in the form of general budget support, it effectively becomes part of the government pool. Therefore, the opportunity cost of such general budget support is most closely the opportunity cost of government expenditure on health. There is no reason for a separate threshold, and donors should consider the marginal productivity of expenditure on health within the country when considering general budget support. Ceteris paribus, an expansion of the budget would result in an increase in the marginal productivity of the healthcare system.

Where donors are providing off-budget support, given that one possibility is to give the money for the intervention to countries (on budget), donors need to consider if their off-budget project performs better or, if not, whether there are good reasons to fund it nonetheless.

Where donors are providing off-budget support and have an exogenously fixed budget, then a supply side approach might be reasonable. This requires two things: an estimate of a “donor ICER” and an estimate of what is displaced from among existing donor-funded healthcare when the donor funds a new intervention. Where the latter is unknown, a reasonable approach would be to rank by donor ICER. Calculating a “donor ICER” requires first calculating the global net health effects. Then calculating a “donor ICER” from the cost to the donor and the global net health.

Where donors are providing off-budget support and have an endogenous budget, they may choose to expand their budget to continue to fund interventions until the donor’s willingness to pay for health is exhausted. Ideally, funding decisions would be taken in order of most to least cost-effective. This better aligns with the objective of health maximisation than the demand-side approach described by the authors in Option 3. Donors taking a demand side approach as described in Option 3 may fund healthcare that would be highly cost-ineffective for the country to fund. The problem with this is that donor funding can negatively impact on the efficiency of systems and sustainability of interventions through “the duplication of services, dilution and distortion of limited human and financial resources, and weak coordination between levels of care” (Barr, A., Garrett, L., Marten, R. et al. Health sector fragmentation: three examples from Sierra Leone. Global Health 15, 8 (2019). https://doi.org/10.1186/s12992-018-0447-5). A better advocacy tool might be to illustrate the extent of possible health gains by funding healthcare closer to the margin of what is generated by the healthcare system as illustrated in Figure 1, where spending the money required to fund aid-supported services would generate more health than spending the same money to fund excluded services.

While the authors are right that focusing national resources towards funding a core package of the most essential services would help to ensure that those services are not subject to aid volatility, a caveat to the proposal for aid supported services to be those which are just cost-ineffective for the country rather than those which are cost-effective for the country, is that many countries are not able to deliver the healthcare interventions they do provide to the full population in need of them. Adding additional services may only further detract from this ability by further diluting human resource capacity etc. Donors might instead consider spending their resources to ensure that the essential services included in the core package are accessible to everyone who requires them.

Is the topic of the opinion article discussed accurately in the context of the current literature?

Yes
Are all factual statements correct and adequately supported by citations?

Yes
Are arguments sufficiently supported by evidence from the published literature?

Partly
Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Partly

References

1. Claxton K, Ochalek J, Revill P, Rollinger A, et al.: Informing Decisions in Global /Health: Cost Per DALY Thresholds and Health Opportunity Costs [version 1; not peer reviewed] (document). F1000Research. 2017; 6 (467). Publisher Full Text
2. Barr A, Garrett L, Marten R, Kadandale S: Health sector fragmentation: three examples from Sierra Leone.Global Health. 2019; 15 (1): 8 PubMed Abstract | Publisher Full Text

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Application of economic evaluation principles and methods to inform priority setting in healthcare, particularly in low- and middle-income countries.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.

CITE

Report a concern

Author Response 18 Jan 2024

Tom Drake, Department of Global Health, Centre for Global Development, London, UK

18 Jan 2024

Author Response

We thank for the reviewers for their commentary in response to our article and appreciate their research in this space. Below we offer point-by-point responses to the feedback using [R] ... Continue reading We thank for the reviewers for their commentary in response to our article and appreciate their research in this space. Below we offer point-by-point responses to the feedback using [R] to denote reviewer text and [A] for author text.

[R] This opinion article aims to provide a neat and tidy solution to the problem of how donors should prioritise their funding for health both within and across countries.

The authors start with the assumption that donors should apply a different threshold than that which should be applied by national decision makers allocating government resources for health. The latter, we agree, should reflect the marginal productivity of the healthcare system. Where there is less agreement is around whether donors should use a separate threshold at all and, if they must, what that threshold should be based upon.

[A] We build a case throughout the article (and especially in the first section titled, “Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health“) that donors should apply a different threshold from national decision-makers. We give two key reasons for the need for a different approach, namely, the differing resource constraints and the differing opportunity costs, which we explain. To make this clearer we have updated the beginning of this first section to read: “Donors may distribute DAH to pursue various objectives, which may include reducing preventable mortality and morbidity, protecting their own country’s health (e.g., by reducing global infectious burden of disease) or fostering wider political and economic national interests. Where health maximisation is at least one of the purposes of a donor’s DAH, and some form of cost-effectiveness thresholds are therefore potentially relevant, then we argue that the adoption of a separate threshold to national decision-makers becomes useful for the following reasons:”

[R] The authors rightly point out that the opportunity cost of investing in intervention A in country X is not only intervention B in country X, but also intervention C in country Y. One way to consider this is using the net health effect of the intervention within each country and across countries. (See Claxton, K. P., Ochalek, J. M., Revill, P., Rollinger, A. & Walker, D., Informing Decisions in Global Health: Cost Per DALY Thresholds and Health Opportunity Costs, Nov 2016, 4 p. Centre for Health Economics, University of York). Net health effects for each country are a function of the cost to the country (or the cost savings), health impacts and an estimate of health opportunity cost to the country of government expenditure on health. Summing these across countries gives the global net health impact. Once this is known, the donor can select what they regard as the best (based not just the net health effects but also on their distribution across countries).

[A] Thank you for this comment. We agree that net health (or monetary) benefit is one way to make cross-country comparisons. However, a key limitation to this approach is that the conversion factor between money and health in net health calculations, the health production function, is typically the cost-effectiveness threshold. If the CET should reflect the total resource envelope, then it will also be a product of the donor allocation decision as well as - in this use case - a determinant of it.

Secondly, a key element of our proposed approach in the section "Why does this matter" is the process of negotiation and partnership that can result from explicit negotiation of who will fund which services and why. A joint threshold, concealed in a net health benefit calculation does not support such an approach.

[R] It may be helpful to distinguish between general budget support and off-budget support. Where funding is provided in the form of general budget support, it effectively becomes part of the government pool. Therefore, the opportunity cost of such general budget support is most closely the opportunity cost of government expenditure on health. There is no reason for a separate threshold, and donors should consider the marginal productivity of expenditure on health within the country when considering general budget support. Ceteris paribus, an expansion of the budget would result in an increase in the marginal productivity of the healthcare system.

Where donors are providing off-budget support, given that one possibility is to give the money for the intervention to countries (on budget), donors need to consider if their off-budget project performs better or, if not, whether there are good reasons to fund it nonetheless.

Where donors are providing off-budget support and have an exogenously fixed budget, then a supply side approach might be reasonable. This requires two things: an estimate of a “donor ICER” and an estimate of what is displaced from among existing donor-funded healthcare when the donor funds a new intervention. Where the latter is unknown, a reasonable approach would be to rank by donor ICER. Calculating a “donor ICER” requires first calculating the global net health effects. Then calculating a “donor ICER” from the cost to the donor and the global net health.

[A] Thank you for raising the important practical consideration of how the proposed framework might be affected by the practical approach to financing and delivery. We agree that these are important considerations and in fact are the subject of ongoing follow up work. We disagree that our proposed approach is incompatible with on-budget support. First, tracking and monitoring of on-budget support is technically feasible (if burdensome), enabling donor funding to be earmarked for specific services while being channelled through public systems. Second, thresholds could be used during budget formulation to negotiate and agree levels of on budget support to provide, even if contributions are not then traced through public financial systems. Thirdly, when a donor provides on-budget support to a country, the opportunity cost of that support is the value that could have been gained had that DAH been invested in either a project in that country OR in a project or on-budget support in another country. So the fact that for donors the opportunity cost of on-budget support is not restricted to the recipient country implies a different decision threshold.
Public financial management considerations will be the subject of future publications and are beyond the scope of this commentary. We have added a note to the article that “Further work is needed to consider the public financial management implications of on- or off-budget support using this framework.”

[R] Where donors are providing off-budget support and have an endogenous budget, they may choose to expand their budget to continue to fund interventions until the donor’s willingness to pay for health is exhausted. Ideally, funding decisions would be taken in order of most to least cost-effective. This better aligns with the objective of health maximisation than the demand-side approach described by the authors in Option 3. Donors taking a demand side approach as described in Option 3 may fund healthcare that would be highly cost-ineffective for the country to fund. The problem with this is that donor funding can negatively impact on the efficiency of systems and sustainability of interventions through “the duplication of services, dilution and distortion of limited human and financial resources, and weak coordination between levels of care” (Barr, A., Garrett, L., Marten, R. et al. Health sector fragmentation: three examples from Sierra Leone. Global Health 15, 8 (2019). https://doi.org/10.1186/s12992-018-0447-5). A better advocacy tool might be to illustrate the extent of possible health gains by funding healthcare closer to the margin of what is generated by the healthcare system as illustrated in Figure 1, where spending the money required to fund aid-supported services would generate more health than spending the same money to fund excluded services.

[A] Thank you for this comment. We do not promote Option 3 over and above Options 1 and 2, but rather, attempt to describe the strengths and weaknesses of each. We agree with the potential risks of a demand-derived threshold and acknowledge them in the article, where we write: "The drawback of the aspirational target is that it may allow sub-optimal allocation decisions if the demand-side aspiration is radically different from the supply-side reality." We also allude to the potential for endogenous budgets and the potential for aid budgets to be expanded in line with aspirational targets, that is, indications of what donors’ collective willingness-to-pay should be. At the beginning of the section on options for setting thresholds, we recognise the historic and ongoing issues with using the 1x and 3x GDP thresholds for optimising resource allocation from a country decision-maker perspective. For completeness in describing the options available for setting a donor perspective threshold, it makes sense to include both supply and demand options and the significant differences between the country and donor perspective thresholds, both in terms of the cohesiveness of the budget/payer and how the threshold could inform resource allocation and mobilisation.

[R] While the authors are right that focusing national resources towards funding a core package of the most essential services would help to ensure that those services are not subject to aid volatility, a caveat to the proposal for aid supported services to be those which are just cost-ineffective for the country rather than those which are cost-effective for the country, is that many countries are not able to deliver the healthcare interventions they do provide to the full population in need of them. Adding additional services may only further detract from this ability by further diluting human resource capacity etc. Donors might instead consider spending their resources to ensure that the essential services included in the core package are accessible to everyone who requires them.

[A] We agree that ensuring effective quality delivery of the core package is a priority and that development partners may have an important role to play in achieving this goal, including by financing (or even directly delivering) these services for a period. However, as part of the development process, countries must be supported to take on the responsibility for financing and delivering the highest priority services and this could easily happen while total budget envelope is still low enough to require financial support to achieve an acceptable essential package of services. In short, we agree that the approach should be a new deal or compact to strengthen country processes for prioritisation, financing and delivery, place countries in the driving seat of the prioritisation process and (implicitly or explicitly) link additional financing for services to those at the margin as defined by countries, not to funder priorities.
We thank for the reviewers for their commentary in response to our article and appreciate their research in this space. Below we offer point-by-point responses to the feedback using [R] to denote reviewer text and [A] for author text.

[R] This opinion article aims to provide a neat and tidy solution to the problem of how donors should prioritise their funding for health both within and across countries.

The authors start with the assumption that donors should apply a different threshold than that which should be applied by national decision makers allocating government resources for health. The latter, we agree, should reflect the marginal productivity of the healthcare system. Where there is less agreement is around whether donors should use a separate threshold at all and, if they must, what that threshold should be based upon.

[A] We build a case throughout the article (and especially in the first section titled, “Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health“) that donors should apply a different threshold from national decision-makers. We give two key reasons for the need for a different approach, namely, the differing resource constraints and the differing opportunity costs, which we explain. To make this clearer we have updated the beginning of this first section to read: “Donors may distribute DAH to pursue various objectives, which may include reducing preventable mortality and morbidity, protecting their own country’s health (e.g., by reducing global infectious burden of disease) or fostering wider political and economic national interests. Where health maximisation is at least one of the purposes of a donor’s DAH, and some form of cost-effectiveness thresholds are therefore potentially relevant, then we argue that the adoption of a separate threshold to national decision-makers becomes useful for the following reasons:”

[R] The authors rightly point out that the opportunity cost of investing in intervention A in country X is not only intervention B in country X, but also intervention C in country Y. One way to consider this is using the net health effect of the intervention within each country and across countries. (See Claxton, K. P., Ochalek, J. M., Revill, P., Rollinger, A. & Walker, D., Informing Decisions in Global Health: Cost Per DALY Thresholds and Health Opportunity Costs, Nov 2016, 4 p. Centre for Health Economics, University of York). Net health effects for each country are a function of the cost to the country (or the cost savings), health impacts and an estimate of health opportunity cost to the country of government expenditure on health. Summing these across countries gives the global net health impact. Once this is known, the donor can select what they regard as the best (based not just the net health effects but also on their distribution across countries).

[A] Thank you for this comment. We agree that net health (or monetary) benefit is one way to make cross-country comparisons. However, a key limitation to this approach is that the conversion factor between money and health in net health calculations, the health production function, is typically the cost-effectiveness threshold. If the CET should reflect the total resource envelope, then it will also be a product of the donor allocation decision as well as - in this use case - a determinant of it.

Secondly, a key element of our proposed approach in the section "Why does this matter" is the process of negotiation and partnership that can result from explicit negotiation of who will fund which services and why. A joint threshold, concealed in a net health benefit calculation does not support such an approach.

[R] It may be helpful to distinguish between general budget support and off-budget support. Where funding is provided in the form of general budget support, it effectively becomes part of the government pool. Therefore, the opportunity cost of such general budget support is most closely the opportunity cost of government expenditure on health. There is no reason for a separate threshold, and donors should consider the marginal productivity of expenditure on health within the country when considering general budget support. Ceteris paribus, an expansion of the budget would result in an increase in the marginal productivity of the healthcare system.

Where donors are providing off-budget support, given that one possibility is to give the money for the intervention to countries (on budget), donors need to consider if their off-budget project performs better or, if not, whether there are good reasons to fund it nonetheless.

Where donors are providing off-budget support and have an exogenously fixed budget, then a supply side approach might be reasonable. This requires two things: an estimate of a “donor ICER” and an estimate of what is displaced from among existing donor-funded healthcare when the donor funds a new intervention. Where the latter is unknown, a reasonable approach would be to rank by donor ICER. Calculating a “donor ICER” requires first calculating the global net health effects. Then calculating a “donor ICER” from the cost to the donor and the global net health.

[A] Thank you for raising the important practical consideration of how the proposed framework might be affected by the practical approach to financing and delivery. We agree that these are important considerations and in fact are the subject of ongoing follow up work. We disagree that our proposed approach is incompatible with on-budget support. First, tracking and monitoring of on-budget support is technically feasible (if burdensome), enabling donor funding to be earmarked for specific services while being channelled through public systems. Second, thresholds could be used during budget formulation to negotiate and agree levels of on budget support to provide, even if contributions are not then traced through public financial systems. Thirdly, when a donor provides on-budget support to a country, the opportunity cost of that support is the value that could have been gained had that DAH been invested in either a project in that country OR in a project or on-budget support in another country. So the fact that for donors the opportunity cost of on-budget support is not restricted to the recipient country implies a different decision threshold.
Public financial management considerations will be the subject of future publications and are beyond the scope of this commentary. We have added a note to the article that “Further work is needed to consider the public financial management implications of on- or off-budget support using this framework.”

[R] Where donors are providing off-budget support and have an endogenous budget, they may choose to expand their budget to continue to fund interventions until the donor’s willingness to pay for health is exhausted. Ideally, funding decisions would be taken in order of most to least cost-effective. This better aligns with the objective of health maximisation than the demand-side approach described by the authors in Option 3. Donors taking a demand side approach as described in Option 3 may fund healthcare that would be highly cost-ineffective for the country to fund. The problem with this is that donor funding can negatively impact on the efficiency of systems and sustainability of interventions through “the duplication of services, dilution and distortion of limited human and financial resources, and weak coordination between levels of care” (Barr, A., Garrett, L., Marten, R. et al. Health sector fragmentation: three examples from Sierra Leone. Global Health 15, 8 (2019). https://doi.org/10.1186/s12992-018-0447-5). A better advocacy tool might be to illustrate the extent of possible health gains by funding healthcare closer to the margin of what is generated by the healthcare system as illustrated in Figure 1, where spending the money required to fund aid-supported services would generate more health than spending the same money to fund excluded services.

[A] Thank you for this comment. We do not promote Option 3 over and above Options 1 and 2, but rather, attempt to describe the strengths and weaknesses of each. We agree with the potential risks of a demand-derived threshold and acknowledge them in the article, where we write: "The drawback of the aspirational target is that it may allow sub-optimal allocation decisions if the demand-side aspiration is radically different from the supply-side reality." We also allude to the potential for endogenous budgets and the potential for aid budgets to be expanded in line with aspirational targets, that is, indications of what donors’ collective willingness-to-pay should be. At the beginning of the section on options for setting thresholds, we recognise the historic and ongoing issues with using the 1x and 3x GDP thresholds for optimising resource allocation from a country decision-maker perspective. For completeness in describing the options available for setting a donor perspective threshold, it makes sense to include both supply and demand options and the significant differences between the country and donor perspective thresholds, both in terms of the cohesiveness of the budget/payer and how the threshold could inform resource allocation and mobilisation.

[R] While the authors are right that focusing national resources towards funding a core package of the most essential services would help to ensure that those services are not subject to aid volatility, a caveat to the proposal for aid supported services to be those which are just cost-ineffective for the country rather than those which are cost-effective for the country, is that many countries are not able to deliver the healthcare interventions they do provide to the full population in need of them. Adding additional services may only further detract from this ability by further diluting human resource capacity etc. Donors might instead consider spending their resources to ensure that the essential services included in the core package are accessible to everyone who requires them.

[A] We agree that ensuring effective quality delivery of the core package is a priority and that development partners may have an important role to play in achieving this goal, including by financing (or even directly delivering) these services for a period. However, as part of the development process, countries must be supported to take on the responsibility for financing and delivering the highest priority services and this could easily happen while total budget envelope is still low enough to require financial support to achieve an acceptable essential package of services. In short, we agree that the approach should be a new deal or compact to strengthen country processes for prioritisation, financing and delivery, place countries in the driving seat of the prioritisation process and (implicitly or explicitly) link additional financing for services to those at the margin as defined by countries, not to funder priorities.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 18 Jan 2024

Tom Drake, Department of Global Health, Centre for Global Development, London, UK

18 Jan 2024

Author Response

We thank for the reviewers for their commentary in response to our article and appreciate their research in this space. Below we offer point-by-point responses to the feedback using [R] ... Continue reading We thank for the reviewers for their commentary in response to our article and appreciate their research in this space. Below we offer point-by-point responses to the feedback using [R] to denote reviewer text and [A] for author text.

[R] This opinion article aims to provide a neat and tidy solution to the problem of how donors should prioritise their funding for health both within and across countries.

The authors start with the assumption that donors should apply a different threshold than that which should be applied by national decision makers allocating government resources for health. The latter, we agree, should reflect the marginal productivity of the healthcare system. Where there is less agreement is around whether donors should use a separate threshold at all and, if they must, what that threshold should be based upon.

[A] We build a case throughout the article (and especially in the first section titled, “Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health“) that donors should apply a different threshold from national decision-makers. We give two key reasons for the need for a different approach, namely, the differing resource constraints and the differing opportunity costs, which we explain. To make this clearer we have updated the beginning of this first section to read: “Donors may distribute DAH to pursue various objectives, which may include reducing preventable mortality and morbidity, protecting their own country’s health (e.g., by reducing global infectious burden of disease) or fostering wider political and economic national interests. Where health maximisation is at least one of the purposes of a donor’s DAH, and some form of cost-effectiveness thresholds are therefore potentially relevant, then we argue that the adoption of a separate threshold to national decision-makers becomes useful for the following reasons:”

[R] The authors rightly point out that the opportunity cost of investing in intervention A in country X is not only intervention B in country X, but also intervention C in country Y. One way to consider this is using the net health effect of the intervention within each country and across countries. (See Claxton, K. P., Ochalek, J. M., Revill, P., Rollinger, A. & Walker, D., Informing Decisions in Global Health: Cost Per DALY Thresholds and Health Opportunity Costs, Nov 2016, 4 p. Centre for Health Economics, University of York). Net health effects for each country are a function of the cost to the country (or the cost savings), health impacts and an estimate of health opportunity cost to the country of government expenditure on health. Summing these across countries gives the global net health impact. Once this is known, the donor can select what they regard as the best (based not just the net health effects but also on their distribution across countries).

[A] Thank you for this comment. We agree that net health (or monetary) benefit is one way to make cross-country comparisons. However, a key limitation to this approach is that the conversion factor between money and health in net health calculations, the health production function, is typically the cost-effectiveness threshold. If the CET should reflect the total resource envelope, then it will also be a product of the donor allocation decision as well as - in this use case - a determinant of it.

Secondly, a key element of our proposed approach in the section "Why does this matter" is the process of negotiation and partnership that can result from explicit negotiation of who will fund which services and why. A joint threshold, concealed in a net health benefit calculation does not support such an approach.

[R] It may be helpful to distinguish between general budget support and off-budget support. Where funding is provided in the form of general budget support, it effectively becomes part of the government pool. Therefore, the opportunity cost of such general budget support is most closely the opportunity cost of government expenditure on health. There is no reason for a separate threshold, and donors should consider the marginal productivity of expenditure on health within the country when considering general budget support. Ceteris paribus, an expansion of the budget would result in an increase in the marginal productivity of the healthcare system.

Where donors are providing off-budget support, given that one possibility is to give the money for the intervention to countries (on budget), donors need to consider if their off-budget project performs better or, if not, whether there are good reasons to fund it nonetheless.

Where donors are providing off-budget support and have an exogenously fixed budget, then a supply side approach might be reasonable. This requires two things: an estimate of a “donor ICER” and an estimate of what is displaced from among existing donor-funded healthcare when the donor funds a new intervention. Where the latter is unknown, a reasonable approach would be to rank by donor ICER. Calculating a “donor ICER” requires first calculating the global net health effects. Then calculating a “donor ICER” from the cost to the donor and the global net health.

[A] Thank you for raising the important practical consideration of how the proposed framework might be affected by the practical approach to financing and delivery. We agree that these are important considerations and in fact are the subject of ongoing follow up work. We disagree that our proposed approach is incompatible with on-budget support. First, tracking and monitoring of on-budget support is technically feasible (if burdensome), enabling donor funding to be earmarked for specific services while being channelled through public systems. Second, thresholds could be used during budget formulation to negotiate and agree levels of on budget support to provide, even if contributions are not then traced through public financial systems. Thirdly, when a donor provides on-budget support to a country, the opportunity cost of that support is the value that could have been gained had that DAH been invested in either a project in that country OR in a project or on-budget support in another country. So the fact that for donors the opportunity cost of on-budget support is not restricted to the recipient country implies a different decision threshold.
Public financial management considerations will be the subject of future publications and are beyond the scope of this commentary. We have added a note to the article that “Further work is needed to consider the public financial management implications of on- or off-budget support using this framework.”

[R] Where donors are providing off-budget support and have an endogenous budget, they may choose to expand their budget to continue to fund interventions until the donor’s willingness to pay for health is exhausted. Ideally, funding decisions would be taken in order of most to least cost-effective. This better aligns with the objective of health maximisation than the demand-side approach described by the authors in Option 3. Donors taking a demand side approach as described in Option 3 may fund healthcare that would be highly cost-ineffective for the country to fund. The problem with this is that donor funding can negatively impact on the efficiency of systems and sustainability of interventions through “the duplication of services, dilution and distortion of limited human and financial resources, and weak coordination between levels of care” (Barr, A., Garrett, L., Marten, R. et al. Health sector fragmentation: three examples from Sierra Leone. Global Health 15, 8 (2019). https://doi.org/10.1186/s12992-018-0447-5). A better advocacy tool might be to illustrate the extent of possible health gains by funding healthcare closer to the margin of what is generated by the healthcare system as illustrated in Figure 1, where spending the money required to fund aid-supported services would generate more health than spending the same money to fund excluded services.

[A] Thank you for this comment. We do not promote Option 3 over and above Options 1 and 2, but rather, attempt to describe the strengths and weaknesses of each. We agree with the potential risks of a demand-derived threshold and acknowledge them in the article, where we write: "The drawback of the aspirational target is that it may allow sub-optimal allocation decisions if the demand-side aspiration is radically different from the supply-side reality." We also allude to the potential for endogenous budgets and the potential for aid budgets to be expanded in line with aspirational targets, that is, indications of what donors’ collective willingness-to-pay should be. At the beginning of the section on options for setting thresholds, we recognise the historic and ongoing issues with using the 1x and 3x GDP thresholds for optimising resource allocation from a country decision-maker perspective. For completeness in describing the options available for setting a donor perspective threshold, it makes sense to include both supply and demand options and the significant differences between the country and donor perspective thresholds, both in terms of the cohesiveness of the budget/payer and how the threshold could inform resource allocation and mobilisation.

[R] While the authors are right that focusing national resources towards funding a core package of the most essential services would help to ensure that those services are not subject to aid volatility, a caveat to the proposal for aid supported services to be those which are just cost-ineffective for the country rather than those which are cost-effective for the country, is that many countries are not able to deliver the healthcare interventions they do provide to the full population in need of them. Adding additional services may only further detract from this ability by further diluting human resource capacity etc. Donors might instead consider spending their resources to ensure that the essential services included in the core package are accessible to everyone who requires them.

[A] We agree that ensuring effective quality delivery of the core package is a priority and that development partners may have an important role to play in achieving this goal, including by financing (or even directly delivering) these services for a period. However, as part of the development process, countries must be supported to take on the responsibility for financing and delivering the highest priority services and this could easily happen while total budget envelope is still low enough to require financial support to achieve an acceptable essential package of services. In short, we agree that the approach should be a new deal or compact to strengthen country processes for prioritisation, financing and delivery, place countries in the driving seat of the prioritisation process and (implicitly or explicitly) link additional financing for services to those at the margin as defined by countries, not to funder priorities.
We thank for the reviewers for their commentary in response to our article and appreciate their research in this space. Below we offer point-by-point responses to the feedback using [R] to denote reviewer text and [A] for author text.

[R] This opinion article aims to provide a neat and tidy solution to the problem of how donors should prioritise their funding for health both within and across countries.

The authors start with the assumption that donors should apply a different threshold than that which should be applied by national decision makers allocating government resources for health. The latter, we agree, should reflect the marginal productivity of the healthcare system. Where there is less agreement is around whether donors should use a separate threshold at all and, if they must, what that threshold should be based upon.

[A] We build a case throughout the article (and especially in the first section titled, “Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health“) that donors should apply a different threshold from national decision-makers. We give two key reasons for the need for a different approach, namely, the differing resource constraints and the differing opportunity costs, which we explain. To make this clearer we have updated the beginning of this first section to read: “Donors may distribute DAH to pursue various objectives, which may include reducing preventable mortality and morbidity, protecting their own country’s health (e.g., by reducing global infectious burden of disease) or fostering wider political and economic national interests. Where health maximisation is at least one of the purposes of a donor’s DAH, and some form of cost-effectiveness thresholds are therefore potentially relevant, then we argue that the adoption of a separate threshold to national decision-makers becomes useful for the following reasons:”

[R] The authors rightly point out that the opportunity cost of investing in intervention A in country X is not only intervention B in country X, but also intervention C in country Y. One way to consider this is using the net health effect of the intervention within each country and across countries. (See Claxton, K. P., Ochalek, J. M., Revill, P., Rollinger, A. & Walker, D., Informing Decisions in Global Health: Cost Per DALY Thresholds and Health Opportunity Costs, Nov 2016, 4 p. Centre for Health Economics, University of York). Net health effects for each country are a function of the cost to the country (or the cost savings), health impacts and an estimate of health opportunity cost to the country of government expenditure on health. Summing these across countries gives the global net health impact. Once this is known, the donor can select what they regard as the best (based not just the net health effects but also on their distribution across countries).

[A] Thank you for this comment. We agree that net health (or monetary) benefit is one way to make cross-country comparisons. However, a key limitation to this approach is that the conversion factor between money and health in net health calculations, the health production function, is typically the cost-effectiveness threshold. If the CET should reflect the total resource envelope, then it will also be a product of the donor allocation decision as well as - in this use case - a determinant of it.

Secondly, a key element of our proposed approach in the section "Why does this matter" is the process of negotiation and partnership that can result from explicit negotiation of who will fund which services and why. A joint threshold, concealed in a net health benefit calculation does not support such an approach.

[R] It may be helpful to distinguish between general budget support and off-budget support. Where funding is provided in the form of general budget support, it effectively becomes part of the government pool. Therefore, the opportunity cost of such general budget support is most closely the opportunity cost of government expenditure on health. There is no reason for a separate threshold, and donors should consider the marginal productivity of expenditure on health within the country when considering general budget support. Ceteris paribus, an expansion of the budget would result in an increase in the marginal productivity of the healthcare system.

Where donors are providing off-budget support, given that one possibility is to give the money for the intervention to countries (on budget), donors need to consider if their off-budget project performs better or, if not, whether there are good reasons to fund it nonetheless.

Where donors are providing off-budget support and have an exogenously fixed budget, then a supply side approach might be reasonable. This requires two things: an estimate of a “donor ICER” and an estimate of what is displaced from among existing donor-funded healthcare when the donor funds a new intervention. Where the latter is unknown, a reasonable approach would be to rank by donor ICER. Calculating a “donor ICER” requires first calculating the global net health effects. Then calculating a “donor ICER” from the cost to the donor and the global net health.

[A] Thank you for raising the important practical consideration of how the proposed framework might be affected by the practical approach to financing and delivery. We agree that these are important considerations and in fact are the subject of ongoing follow up work. We disagree that our proposed approach is incompatible with on-budget support. First, tracking and monitoring of on-budget support is technically feasible (if burdensome), enabling donor funding to be earmarked for specific services while being channelled through public systems. Second, thresholds could be used during budget formulation to negotiate and agree levels of on budget support to provide, even if contributions are not then traced through public financial systems. Thirdly, when a donor provides on-budget support to a country, the opportunity cost of that support is the value that could have been gained had that DAH been invested in either a project in that country OR in a project or on-budget support in another country. So the fact that for donors the opportunity cost of on-budget support is not restricted to the recipient country implies a different decision threshold.
Public financial management considerations will be the subject of future publications and are beyond the scope of this commentary. We have added a note to the article that “Further work is needed to consider the public financial management implications of on- or off-budget support using this framework.”

[R] Where donors are providing off-budget support and have an endogenous budget, they may choose to expand their budget to continue to fund interventions until the donor’s willingness to pay for health is exhausted. Ideally, funding decisions would be taken in order of most to least cost-effective. This better aligns with the objective of health maximisation than the demand-side approach described by the authors in Option 3. Donors taking a demand side approach as described in Option 3 may fund healthcare that would be highly cost-ineffective for the country to fund. The problem with this is that donor funding can negatively impact on the efficiency of systems and sustainability of interventions through “the duplication of services, dilution and distortion of limited human and financial resources, and weak coordination between levels of care” (Barr, A., Garrett, L., Marten, R. et al. Health sector fragmentation: three examples from Sierra Leone. Global Health 15, 8 (2019). https://doi.org/10.1186/s12992-018-0447-5). A better advocacy tool might be to illustrate the extent of possible health gains by funding healthcare closer to the margin of what is generated by the healthcare system as illustrated in Figure 1, where spending the money required to fund aid-supported services would generate more health than spending the same money to fund excluded services.

[A] Thank you for this comment. We do not promote Option 3 over and above Options 1 and 2, but rather, attempt to describe the strengths and weaknesses of each. We agree with the potential risks of a demand-derived threshold and acknowledge them in the article, where we write: "The drawback of the aspirational target is that it may allow sub-optimal allocation decisions if the demand-side aspiration is radically different from the supply-side reality." We also allude to the potential for endogenous budgets and the potential for aid budgets to be expanded in line with aspirational targets, that is, indications of what donors’ collective willingness-to-pay should be. At the beginning of the section on options for setting thresholds, we recognise the historic and ongoing issues with using the 1x and 3x GDP thresholds for optimising resource allocation from a country decision-maker perspective. For completeness in describing the options available for setting a donor perspective threshold, it makes sense to include both supply and demand options and the significant differences between the country and donor perspective thresholds, both in terms of the cohesiveness of the budget/payer and how the threshold could inform resource allocation and mobilisation.

[R] While the authors are right that focusing national resources towards funding a core package of the most essential services would help to ensure that those services are not subject to aid volatility, a caveat to the proposal for aid supported services to be those which are just cost-ineffective for the country rather than those which are cost-effective for the country, is that many countries are not able to deliver the healthcare interventions they do provide to the full population in need of them. Adding additional services may only further detract from this ability by further diluting human resource capacity etc. Donors might instead consider spending their resources to ensure that the essential services included in the core package are accessible to everyone who requires them.

[A] We agree that ensuring effective quality delivery of the core package is a priority and that development partners may have an important role to play in achieving this goal, including by financing (or even directly delivering) these services for a period. However, as part of the development process, countries must be supported to take on the responsibility for financing and delivering the highest priority services and this could easily happen while total budget envelope is still low enough to require financial support to achieve an acceptable essential package of services. In short, we agree that the approach should be a new deal or compact to strengthen country processes for prioritisation, financing and delivery, place countries in the driving seat of the prioritisation process and (implicitly or explicitly) link additional financing for services to those at the margin as defined by countries, not to funder priorities.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 27 Feb 2023

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 18 Jan 24	read
Version 1 27 Feb 23	read	read

Jessica Ochalek, University of York, York, UK

Karl Claxton, University of York, York, UK
Chris Sampson, Office of Health Economics, London, UK

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

13 Views

27 Feb 2024 | for Version 2

Jessica Ochalek, Centre for Health Economics, University of York, York, England, UK

13 Views Cite this report Responses(0)

Approved

I have no further comments.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Application of economic evaluation principles and methods to inform priority setting in healthcare, particularly in low- and middle-income countries.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

23 Views

01 Aug 2023 | for Version 1

Chris Sampson, Office of Health Economics, London, England, UK

23 Views Cite this report Responses(1)

Approved

Is the topic of the opinion article discussed accurately in the context of the current literature?

Yes
Are all factual statements correct and adequately supported by citations?

Yes
Are arguments sufficiently supported by evidence from the published literature?

Partly
Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Health economics

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

18 Jan 2024

Tom Drake, Department of Global Health, Centre for Global Development, London, UK

We thank the reviewer for this helpful review of the article. Below we offer point-by-point responses to the feedback using [R] to denote reviewer text and [A] for author text.

[A] We would like to thank the reviewer for his time in providing the feedback, which has helped us to improve and clarify the manuscript. We have taken on many of the comments and adjusted the article accordingly. Point-by-point responses are offered below.

[R] I see no reason why this opinion piece should not be indexed. There are no factual errors or essential omissions. However, there are several respects in which the article may be improved, and I would encourage the authors to consider the following suggestions for a revised version. My suggestions are - roughly - in order of importance.

1. The article proceeds on the assumption that health maximisation is the objective of (or at least an important consideration for) global health donors. It isn't clear to me that this is the case. This assumption requires i) clearer articulation as part of the authors' argumentation and ii) further discussion of its validity.
[A] Thank you for this helpful comment. To address this point, we have added a sentence that explicitly acknowledges the range of objectives that donors may pursue at the top of the section titled, “Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health”:
The section now reads: “Donors may distribute DAH to pursue various objectives, which may include reducing preventable mortality and morbidity, protecting their own country’s health (e.g., by reducing global infectious burden of disease) or fostering wider political and economic national interests. Where health maximisation is at least one of the purposes of a donor’s DAH, and some form of cost-effectiveness thresholds are therefore potentially relevant, then we argue that the adoption of a separate threshold to national decision-makers becomes useful for the following reasons:”
2. Even if health maximisation is the objective of global health donors, it isn't clear to me that the proposal summarised in Figure 1 provides a means of achieving this. It seems far more likely to me that global health donors might prefer to spend on infrastructural investments and not marginal technologies, in part because the former is more conducive to health maximisation. The authors should discuss the scenarios in which a donor's use of a threshold would and would not support health maximisation.
[A] We agree that investments like infrastructure are outside the scope of this commentary. To the extent that aid is directed towards health services (rather than health systems investments, global public goods and so on), we argue that conceptualising separate country and donor cost-effectiveness thresholds is theoretically clearer and could present opportunities for practical benefits. We have added a clarification in the second paragraph of the introduction to this effect.

The sentence reads “While some DAH supports research, advocacy, or other catalytic activities, the vast majority supports service delivery, including both financing for specific services (which we focus on in this article) and health system strengthening such as infrastructure investments”.

3. The authors should consider (and describe) the suitability of the bookshelf analogy. In the context of low-income countries - perhaps more so than others - the bookshelf analogy is problematic. In particular, the bookshelf analogy starts to fall apart when we allow for divisibility, which is certainly a relevant consideration in countries in which providers cannot reach all patients.

[A] We think that the bookshelf analogy remains relevant. It depends on how the “books” are specified. Additional sentence added: “Each service – or book in the bookshelf metaphor – could be defined either at the level of the national population or for specific subpopulations for whom the cost-effectiveness differs because of high need, geographical inaccessibility, or other factors.”

4. Some further consideration should be given (and discussion provided) to the claim that a "donor threshold should be higher". There are numerous assumptions inherent in this, and the authors have not described them. For instance, it seems to assume that resources are currently being allocated efficiently by national decision-makers. It behoves the authors to outline all of the assumptions in their claim.

[A] We have now revised the start of the section, “Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health” to articulate more clearly the different objectives donors may pursue and our assumptions about the relevance of health maximisation (albeit not as sole criterion) to decision-making. We certainly do not intend to claim that resources are currently efficiently allocated and our arguments do not depend on such an assumption. Instead, we are arguing that increased use of thresholds could support a transition to more efficient resource allocation. We have also modified our claim in this section to argue only that the differing resource constraints mean that donor and country thresholds should be “different”; we present the wider rationale for why donor thresholds should be higher in the following section, where we are able to outline the considerations more fully.

A revised section on why resource constraints differ now reads:
“Different resource constraints. National decision-makers face budget constraints which reflect their country’s overall resources; the extent to which their government is able to mobilise those resources; and the prioritisation of health within national budgets. Donors, by contrast, face wholly separate resource constraints. Country donors’ budgets are constrained by their country’s resources and prioritisation of (health) aid to other countries, while private and multilateral donors’ budgets are constrained by their ability to raise revenue from government and private contributors globally. If cost-effectiveness thresholds reflect the scarcity of resources, then the differing resource constraints imply that differing thresholds should be used.”

5. Throughout the article, 'thresholds' are characterised as a matter of fact, an observable quantity. This is not correct; thresholds are a policymaking tool with a primarily normative basis. For instance, the title should read that cost-effectiveness thresholds *could* or *should* differ, rather than stating that they *do* differ.

[A] Thank you for raising this important point. We agree and have adjusted the language throughout, including in the title of the article.

6. The authors state that few donors rely on CEA. This raises the question of why we should care about cost-effectiveness thresholds at all! The authors should acknowledge the limited applicability and importance of their assertions in this context. Indeed, there needs to be more justification for the idea that donor cost-effectiveness thresholds should be specified at all.

[A] CEA is used quite widely in several countries because the analytic community has done a good job of crafting analytic tools which are rooted in economic theory but which speak directly to the decisions faced by in-country decision makers and are contextually appropriate. We haven’t got there for donor aid yet, we would argue because the approach has been to carry over the in-country CEA paradigm rather than figuring out how to implement those economic principles in the donor environment. On prima facie basis, economic analysis, as the science of resource scarcity should have a greater role in the allocation of donor aid than it currently does. We hope that our commentary will encourage wider use of CEA and appropriate cost-effectiveness thresholds.
New text: “Economic analysis has huge potential to help donors maximise the achievement of their objectives in the face of the vast gap between need and the resources available. However, the current CEA paradigm was designed for within-country decision making and has limited read-across to the very different environment faced by donors.”

7. The authors refer to the use of decision thresholds by Givewell and Open Philanthropy. It would be helpful to state these thresholds in the text.

[A] We include the following in the text: “GiveWell uses a benchmark that charity programmes should be expected to provide value at least 10 time greater than cash transfers ²⁹and Open Philanthropy require a 1000-fold expected return on investment for their (generally higher risk and upstream) investments.” These specifics come somewhat later in the text than where these donors were first mentioned, as we felt it fit better with the main flow of the text.

8. The claim is made that "many donors" have adopted value for money frameworks, but then only one is cited. Either more citations should be added or the claim should be altered.

[A] Thank you for this suggestion. We have now added references to UNICEF and the Global Fund’s value-for-money frameworks, alongside the existing reference to the UK’s framework.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

52 Views

03 Apr 2023 | for Version 1

Jessica Ochalek, Centre for Health Economics, University of York, York, England, UK

Karl Claxton, Centre for Health Economics, University of York, York, England, UK

52 Views Cite this report Responses(1)

Approved With Reservations

Is the topic of the opinion article discussed accurately in the context of the current literature?

Yes
Are all factual statements correct and adequately supported by citations?

Yes
Are arguments sufficiently supported by evidence from the published literature?

Partly
Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Partly

References

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Application of economic evaluation principles and methods to inform priority setting in healthcare, particularly in low- and middle-income countries.

Respond to this report

Responses (1)

Author Response

18 Jan 2024

Tom Drake, Department of Global Health, Centre for Global Development, London, UK

We thank for the reviewers for their commentary in response to our article and appreciate their research in this space. Below we offer point-by-point responses to the feedback using [R] to denote reviewer text and [A] for author text.

[R] This opinion article aims to provide a neat and tidy solution to the problem of how donors should prioritise their funding for health both within and across countries.

The authors start with the assumption that donors should apply a different threshold than that which should be applied by national decision makers allocating government resources for health. The latter, we agree, should reflect the marginal productivity of the healthcare system. Where there is less agreement is around whether donors should use a separate threshold at all and, if they must, what that threshold should be based upon.

[A] We build a case throughout the article (and especially in the first section titled, “Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health“) that donors should apply a different threshold from national decision-makers. We give two key reasons for the need for a different approach, namely, the differing resource constraints and the differing opportunity costs, which we explain. To make this clearer we have updated the beginning of this first section to read: “Donors may distribute DAH to pursue various objectives, which may include reducing preventable mortality and morbidity, protecting their own country’s health (e.g., by reducing global infectious burden of disease) or fostering wider political and economic national interests. Where health maximisation is at least one of the purposes of a donor’s DAH, and some form of cost-effectiveness thresholds are therefore potentially relevant, then we argue that the adoption of a separate threshold to national decision-makers becomes useful for the following reasons:”

[R] The authors rightly point out that the opportunity cost of investing in intervention A in country X is not only intervention B in country X, but also intervention C in country Y. One way to consider this is using the net health effect of the intervention within each country and across countries. (See Claxton, K. P., Ochalek, J. M., Revill, P., Rollinger, A. & Walker, D., Informing Decisions in Global Health: Cost Per DALY Thresholds and Health Opportunity Costs, Nov 2016, 4 p. Centre for Health Economics, University of York). Net health effects for each country are a function of the cost to the country (or the cost savings), health impacts and an estimate of health opportunity cost to the country of government expenditure on health. Summing these across countries gives the global net health impact. Once this is known, the donor can select what they regard as the best (based not just the net health effects but also on their distribution across countries).

[A] Thank you for this comment. We agree that net health (or monetary) benefit is one way to make cross-country comparisons. However, a key limitation to this approach is that the conversion factor between money and health in net health calculations, the health production function, is typically the cost-effectiveness threshold. If the CET should reflect the total resource envelope, then it will also be a product of the donor allocation decision as well as - in this use case - a determinant of it.

Secondly, a key element of our proposed approach in the section "Why does this matter" is the process of negotiation and partnership that can result from explicit negotiation of who will fund which services and why. A joint threshold, concealed in a net health benefit calculation does not support such an approach.

[R] It may be helpful to distinguish between general budget support and off-budget support. Where funding is provided in the form of general budget support, it effectively becomes part of the government pool. Therefore, the opportunity cost of such general budget support is most closely the opportunity cost of government expenditure on health. There is no reason for a separate threshold, and donors should consider the marginal productivity of expenditure on health within the country when considering general budget support. Ceteris paribus, an expansion of the budget would result in an increase in the marginal productivity of the healthcare system.

Where donors are providing off-budget support, given that one possibility is to give the money for the intervention to countries (on budget), donors need to consider if their off-budget project performs better or, if not, whether there are good reasons to fund it nonetheless.

Where donors are providing off-budget support and have an exogenously fixed budget, then a supply side approach might be reasonable. This requires two things: an estimate of a “donor ICER” and an estimate of what is displaced from among existing donor-funded healthcare when the donor funds a new intervention. Where the latter is unknown, a reasonable approach would be to rank by donor ICER. Calculating a “donor ICER” requires first calculating the global net health effects. Then calculating a “donor ICER” from the cost to the donor and the global net health.

[A] Thank you for raising the important practical consideration of how the proposed framework might be affected by the practical approach to financing and delivery. We agree that these are important considerations and in fact are the subject of ongoing follow up work. We disagree that our proposed approach is incompatible with on-budget support. First, tracking and monitoring of on-budget support is technically feasible (if burdensome), enabling donor funding to be earmarked for specific services while being channelled through public systems. Second, thresholds could be used during budget formulation to negotiate and agree levels of on budget support to provide, even if contributions are not then traced through public financial systems. Thirdly, when a donor provides on-budget support to a country, the opportunity cost of that support is the value that could have been gained had that DAH been invested in either a project in that country OR in a project or on-budget support in another country. So the fact that for donors the opportunity cost of on-budget support is not restricted to the recipient country implies a different decision threshold.
Public financial management considerations will be the subject of future publications and are beyond the scope of this commentary. We have added a note to the article that “Further work is needed to consider the public financial management implications of on- or off-budget support using this framework.”

[R] Where donors are providing off-budget support and have an endogenous budget, they may choose to expand their budget to continue to fund interventions until the donor’s willingness to pay for health is exhausted. Ideally, funding decisions would be taken in order of most to least cost-effective. This better aligns with the objective of health maximisation than the demand-side approach described by the authors in Option 3. Donors taking a demand side approach as described in Option 3 may fund healthcare that would be highly cost-ineffective for the country to fund. The problem with this is that donor funding can negatively impact on the efficiency of systems and sustainability of interventions through “the duplication of services, dilution and distortion of limited human and financial resources, and weak coordination between levels of care” (Barr, A., Garrett, L., Marten, R. et al. Health sector fragmentation: three examples from Sierra Leone. Global Health 15, 8 (2019). https://doi.org/10.1186/s12992-018-0447-5). A better advocacy tool might be to illustrate the extent of possible health gains by funding healthcare closer to the margin of what is generated by the healthcare system as illustrated in Figure 1, where spending the money required to fund aid-supported services would generate more health than spending the same money to fund excluded services.

[A] Thank you for this comment. We do not promote Option 3 over and above Options 1 and 2, but rather, attempt to describe the strengths and weaknesses of each. We agree with the potential risks of a demand-derived threshold and acknowledge them in the article, where we write: "The drawback of the aspirational target is that it may allow sub-optimal allocation decisions if the demand-side aspiration is radically different from the supply-side reality." We also allude to the potential for endogenous budgets and the potential for aid budgets to be expanded in line with aspirational targets, that is, indications of what donors’ collective willingness-to-pay should be. At the beginning of the section on options for setting thresholds, we recognise the historic and ongoing issues with using the 1x and 3x GDP thresholds for optimising resource allocation from a country decision-maker perspective. For completeness in describing the options available for setting a donor perspective threshold, it makes sense to include both supply and demand options and the significant differences between the country and donor perspective thresholds, both in terms of the cohesiveness of the budget/payer and how the threshold could inform resource allocation and mobilisation.

[R] While the authors are right that focusing national resources towards funding a core package of the most essential services would help to ensure that those services are not subject to aid volatility, a caveat to the proposal for aid supported services to be those which are just cost-ineffective for the country rather than those which are cost-effective for the country, is that many countries are not able to deliver the healthcare interventions they do provide to the full population in need of them. Adding additional services may only further detract from this ability by further diluting human resource capacity etc. Donors might instead consider spending their resources to ensure that the essential services included in the core package are accessible to everyone who requires them.

[A] We agree that ensuring effective quality delivery of the core package is a priority and that development partners may have an important role to play in achieving this goal, including by financing (or even directly delivering) these services for a period. However, as part of the development process, countries must be supported to take on the responsibility for financing and delivering the highest priority services and this could easily happen while total budget envelope is still low enough to require financial support to achieve an acceptable essential package of services. In short, we agree that the approach should be a new deal or compact to strengthen country processes for prioritisation, financing and delivery, place countries in the driving seat of the prioritisation process and (implicitly or explicitly) link additional financing for services to those at the margin as defined by countries, not to funder priorities.

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. World Health Assembly 67: Sixty-seventh World Health Assembly, Geneva, 19-24 May 2014: resolutions and decisions: annexes. World Health Organization; 2014 [cited 2022 Dec 21]. Report No.: WHA67/2014/REC/1. Reference Source

[2] 2. Bump JB: Global health aid allocation in the 21st century. Health Policy Plan. 2018 Feb 1; 33(suppl_1): i1–i3. PubMed Abstract | Publisher Full Text | Free Full Text

[3] 3. Teerawattananon Y, Painter C, Dabak S, et al.: Avoiding health technology assessment: a global survey of reasons for not using health technology assessment in decision making. Cost Eff. Resour. Alloc. 2021 Sep 22; 19(1): 62. PubMed Abstract | Publisher Full Text | Free Full Text

[4] 4. Micah AE, Cogswell IE, Cunningham B, et al.: Tracking development assistance for health and for COVID-19: a review of development assistance, government, out-of-pocket, and other private spending on health for 204 countries and territories, 1990–2050. Lancet. 2021 Oct 9; 398(10308): 1317–1343. PubMed Abstract | Publisher Full Text | Free Full Text

[5] 5. OECD Statistics: [cited 2023 Jan 26]. Reference Source

[6] 6. DFID’s approach to value for money (VfM): GOV.UK.[cited 2022 Dec 21]. Reference Source

[7] 7. Value for Money Technical Brief. 2019. The Global Fund. https://www.theglobalfund.org/media/8596/core_valueformoney_technicalbrief_en.pdf

[8] 8. Ensuring Value for Money. UNICEF. 2020. https://www.unicef.org/documents/ensuring-value-money

[9] 9. Committing to Cost-Effectiveness: USAID’s New Effort to Benchmark for Greater Impact. Center for Global Development|Ideas to Action.[cited 2022 Dec 21]. Reference Source

[10] 10. GiveWell’s Cost-Effectiveness Analyses|GiveWell.[cited 2023 Feb 5]. Reference Source

[11] 11. Technical Updates to Our Global Health and Wellbeing Cause Prioritization Framework - Open Philanthropy. Open Philanthropy. 2021 [cited 2022 Dec 21]. Reference Source

[12] 12. Drake T, Regan L, Baker P: Reimagining Global Health Financing: How Refocusing Health Aid at the Margin Could Strengthen Health Systems and Futureproof Aid Financial Flows. Washington DC: Centre for Global Development; 2023 Feb. Report No.: 285. Reference Source

[13] 13. Vallejo-Torres L, García-Lorenzo B, Castilla I, et al.: On the Estimation of the Cost-Effectiveness Threshold: Why, What, How? Value Health. 2016 Jul 1; 19(5): 558–566. PubMed Abstract | Publisher Full Text

[14] 14. Bertram MY, Lauer JA, De Joncheere K, et al.: Cost-effectiveness thresholds: pros and cons. Bull. World Health Organ. 2016 Dec 1; 94(12): 925–930. PubMed Abstract | Publisher Full Text | Free Full Text

[15] 15. Culyer AJ: Cost-effectiveness thresholds in health care: a bookshelf guide to their meaning and use. Health Econ. Policy Law. 2016 Oct; 11(4): 415–432. PubMed Abstract | Publisher Full Text

[16] 16. Chi YL, Blecher M, Chalkidou K, et al.: What next after GDP-based cost-effectiveness thresholds? Gates Open Res. 2020 Nov 30; 4: 176. PubMed Abstract | Publisher Full Text | Free Full Text

[17] 17. Woods B, Revill P, Sculpher M, et al.: Country-Level Cost-Effectiveness Thresholds: Initial Estimates and the Need for Further Research. Value Health. 2016 Dec 1; 19(8): 929–935. PubMed Abstract | Publisher Full Text | Free Full Text

[18] 18. Ochalek J, Lomas J, Claxton K: Estimating health opportunity costs in low-income and middle-income countries: a novel approach and evidence from cross-country data. BMJ Glob. Health. 2018 Nov 1; 3(6): e000964. PubMed Abstract | Publisher Full Text | Free Full Text

[19] 19. Revill P, Ochalek J, Lomas J, et al.: Cost-Effectiveness Thresholds: Guiding Health Care Spending for Population Health Improvement. World Scientific Publishing Co. Pte. Ltd.: 2020 [cited 2022 Dec 21]; 75–97. Reference Source

[20] 20. Drake T: Priority setting in global health: towards a minimum DALY value. Health Econ. 2014 Feb; 23(2): 248–252. PubMed Abstract | Publisher Full Text

[21] 21. Morton A, Arulselvan A, Thomas R: Allocation rules for global donors. J. Health Econ. 2018 Mar; 58: 67–75. PubMed Abstract | Publisher Full Text

[22] 22. Leech AA, Kim DD, Cohen JT, et al.: Use and Misuse of Cost-Effectiveness Analysis Thresholds in Low- and Middle-Income Countries: Trends in Cost-per-DALY Studies. Value Health. 2018 Jul; 21(7): 759–761. PubMed Abstract | Publisher Full Text | Free Full Text

[23] 23. Kazibwe J, Gheorghe A, Wilson D, et al.: The Use of Cost-Effectiveness Thresholds for Evaluating Health Interventions in Low- and Middle-Income Countries From 2015 to 2020: A Review. Value Health. 2022 Mar 1; 25(3): 385–389. PubMed Abstract | Publisher Full Text | Free Full Text

[24] 24. Ogbuoji O, Yamey G: Aid Effectiveness in the Sustainable Development Goals Era. Int. J. Health Policy Manag. 2018; 8(3): 184–186. PubMed Abstract | Publisher Full Text | Free Full Text

[25] 25. Bendavid E, Ottersen T, Peilong L, et al.: Development Assistance for Health. In Disease Control Priorities: Improving Health and Reducing Poverty. Jamison DT, Gelband H, Horton S, et al. editors. 3rd ed.Washington (DC): The International Bank for Reconstruction and Development/The World Bank; 2017. Publisher Full Text http://www.ncbi.nlm.nih.gov/books/NBK525305/

[26] 26. Berthélemy J-C: Bilateral donors' interest vs. Recipients' development motives in aid allocation: do all donors behave the same?Cahiers de la Maison des Sciences Economiques bla05001, Université Panthéon-Sorbonne (Paris 1). 2005.

[27] 27. Gulrajani N, Calleja R: Understanding Donor Motivations: Developing the Principled Aid Index. ODI Working Paper; 2019.

[28] 28. Mladovsky P: Fragmentation by design: Universal health coverage policies as governmentality in Senegal. Soc. Sci. Med. 2020 Sep 1; 260: 113153. PubMed Abstract | Publisher Full Text

[29] 29. Spicer N, Agyepong I, Ottersen T, et al.: ‘It’s far too complicated’: why fragmentation persists in global health. Glob. Health. 2020 Jul 9; 16(1): 60. PubMed Abstract | Publisher Full Text | Free Full Text

[30] 30. Ihekweazu C: Lessons from Nigeria’s Adaptation of Global Health Initiatives during the COVID-19 Pandemic - Volume 28, Supplement—November 2022. Emerging Infectious Diseases journal - CDC [cited 2022 Dec 21]; S299–S301. PubMed Abstract | Publisher Full Text | , Free Full Text Reference Source

[31] 31. Santos AS, Guerra-Junior AA, Godman B, et al.: Cost-effectiveness thresholds: methods for setting and examples from around the world. Expert Rev. Pharmacoecon. Outcomes Res. 2018 Feb 27 [cited 2022 May 30]; 18: 277–288. PubMed Abstract | Publisher Full Text

[32] 32. The World Bank: World Development Report 1993: Investing in Health. Oxford University Press; 1993 [cited 2015 Nov 27]; 346. (World Development Report). Publisher Full Text

[33] 33. Bertram MY, Lauer JA, Stenberg K, et al.: Methods for the Economic Evaluation of Health Care Interventions for Priority Setting in the Health System: An Update From WHO CHOICE. Int. J. Health Policy Manag. 2021 Nov 1; 10(Special Issue on WHO-CHOICE Update): 673–677. Publisher Full Text

[34] 34. Schwarzer R, Rochau U, Saverno K, et al.: Systematic overview of cost-effectiveness thresholds in ten countries across four continents. J. Comp. Eff. Res. 2015 Sep; 4(5): 485–504. PubMed Abstract | Publisher Full Text

[35] 35. Elie: An update on GiveWell’s funding projections. The GiveWell Blog.2022 [cited 2023 Jan 23]. Reference Source

[36] 36. Claxton K, Martin S, Soares M, et al.: Methods for the Estimation of the NICE Cost Effectiveness Threshold. Health Technol. Assess. 2015 Feb 18; 19(14): 1–504. PubMed Abstract | Publisher Full Text | Free Full Text

Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health (and why it matters)

Abstract

Keywords

Revised Amendments from Version 1

Introduction

Why cost-effectiveness thresholds for global health donors should differ from thresholds for Ministries of Health

Why does this matter?

Figure 1. National and donor cost-effectiveness thresholds using the bookshelf metaphor.

Options for setting thresholds

Conclusions

Data availability

Acknowledgements

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated