Scepticaemia: The impact on the health system and patients of delaying new treatments with uncertain evidence; a case study of the sepsis bundle

Robin Blythe; David Cook; Nicholas Graves

doi:10.12688/f1000research.14619.2

Home Browse Scepticaemia: The impact on the health system and patients of delaying...

ALL Metrics

-

Views

-

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

Revised

Scepticaemia: The impact on the health system and patients of delaying new treatments with uncertain evidence; a case study of the sepsis bundle

[version 2; peer review: 2 approved]

Robin Blythe ¹, David Cook², Nicholas Graves¹

PUBLISHED 05 Jul 2018

Author details Author details

¹ Australian Centre for Health Services Innovation, School of Public Health and Social Work, Institute for Health and Biomedical Innovation, Queensland University of Technology, Brisbane, Queensland, Australia
² Intensive Care Unit, Princess Alexandra Hospital, Brisbane, Queensland, Australia

Robin Blythe
Roles: Conceptualization, Formal Analysis, Investigation, Methodology, Project Administration, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

David Cook
Roles: Conceptualization, Methodology, Writing – Review & Editing

Nicholas Graves
Roles: Conceptualization, Formal Analysis, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background: A sepsis care bundle of intravenous vitamin C, thiamine, and hydrocortisone was reported to improve treatment outcomes. The data to support it are uncertain and decision makers are likely to be cautious about adopting it. The objective of this study was to model the opportunity costs in dollars and lives of waiting for better information before adopting the bundle.
Methods: A decision tree was built using information from the literature. We modelled the impact of bundle adoption under three scenarios using a simulation in which the bundle was effective as reported in the primary trial, less effective based on other information, and ineffective.
The measurements were health services costs, quality-adjusted life years, and transition probabilities.
Results: If the bundle proves to be effective under either scenario, it will save billions of dollars and millions of life-years in the United States. This is the opportunity cost of delaying an adoption decision and waiting for better quality evidence. We suggest that hospital decision-makers consider implementing the bundle on a trial basis while monitoring costs and outcomes data even while the evidence base is uncertain.
Conclusions: If the decision maker is unwilling to use the best available evidence now, but rather wishes to wait for definitive evidence they are risking incurring large costs for health care systems and for the patients they serve. An explicit analysis of uncertain clinical outcomes is a useful adjunct to guide decision making where there is clinical ambiguity. This approach offers a valid alternative to the default of clinical inactivity when faced with uncertainty.

Keywords

Sepsis, Costs and Cost Analysis, Evidence-based practice

Corresponding author: Robin Blythe

Competing interests: No competing interests were disclosed.

Grant information: The author(s) declared that no grants were involved in supporting this work.

Copyright: © 2018 Blythe R et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

How to cite: Blythe R, Cook D and Graves N. Scepticaemia: The impact on the health system and patients of delaying new treatments with uncertain evidence; a case study of the sepsis bundle [version 2; peer review: 2 approved]. F1000Research 2018, 7:500 (https://doi.org/10.12688/f1000research.14619.2) First published: 26 Apr 2018, 7:500 (https://doi.org/10.12688/f1000research.14619.1) Latest published: 05 Jul 2018, 7:500 (https://doi.org/10.12688/f1000research.14619.2)

Revised Amendments from Version 1

This version implements the changes suggested by referees Marik and Visser. There were some key clinical considerations suggested by Dr Visser that required addressing. Dr Marik had several points relating to bundle information that were also taken into account. The conclusions and rationale of the paper remain unchanged, but the revisions help to iron out kinks in the first version and slightly change the substance of the methodology and results. These include changes to the text, to Figure 1, where the decision tree features a revised pathway to appropriately model the influence of renal failure on long term outcomes, and to the supplementary dataset of distributions, which features all transition probabilities, costs, and utilities as per the new decision model.

To read any peer review reports and author responses for this article, follow the "read" links in the Open Peer Review table.

Introduction

Sepsis arises frequently among patients admitted to hospitals in the US and elsewhere. It is often fatal, accounting for 30% to 50% of inpatient deaths¹. Those who survive incur large costs from increased risk of organ damage². A recent paper by Marik et al. (2017) described the effectiveness of a treatment bundle made up of intravenous vitamin C, thiamine and hydrocortisone³. Because of small sample size, non-randomised design, large observed treatment effects and the simple and low cost characteristics of the bundled intervention, there was scepticism among clinicians, administrators and payers regarding adoption^4–6. Rapid adoption without good evidence has risks and costs, yet delaying an effective treatment imposes larger costs to patients and the health system. Waiting for conclusive evidence might be a poor and costly strategy and should be balanced against the likely costs of rapid adoption with uncertain and low quality evidence.

The aim of this paper is to estimate the economic consequences of a decision to adopt the Marik sepsis bundle early, under conditions of large uncertainty. This is compared to the alternative, which is to wait 2.5 years; the time period between publication of the first paper in June 2017³ and the completion of several multi-centre trials of the bundle in late 2019. The trials will ideally reduce the uncertainty in an adoption decision. This paper predicts the cost savings and health benefits of adopting the Marik bundle immediately with uncertain evidence for the entire United States health system, compared to waiting for better quality evidence. Our estimates examine likely changes to costs and health outcomes if the treatment is found to be effective, less effective and ineffective.

Methods

An incremental cost-effectiveness analysis was conducted on the change to costs and Quality-Adjusted Life Years (QALYs) of standard sepsis care compared to the early adoption of the Marik bundle³. Values for expected costs and health utilities were taken from the literature, with probabilities of treatment outcomes taken from Marik’s paper³. The time horizon was 5 years in order to sufficiently measure the long-term outcomes of acute renal failure (ARF), an important and common result of severe septic shock². A decision tree was programmed in TreeAge Pro 2017 R2.1 (Williamstown, MA2017) and prior statistical distributions of costs, outcomes and probabilities used to include uncertainties in the data, see Figure 1. Patients receiving either modality progress through a chance node to ARF or not. Patients progress through a second chance node, where they either recover or die. Surviving ARF patients progress through a final chance node where they may require chronic renal replacement therapy (RRT), which can take the form of either dialysis or organ transplantation².

Figure 1. Sepsis decision tree model using TreeAge Pro 2017.

Costs

The costs of care include the hospital cost of a sepsis episode, the additional cost of the Marik treatment bundle, the additional cost of an episode of ARF, and the annual ongoing cost of RRT. Hospital costs were taken from a systematic review by Arefian et al. (2017) for a US perspective with a mean of $32,421 and standard deviation of $15,051 per stay⁷. For the Marik bundle, we used an estimate of $528 and a standard deviation of 5% at $26.50 per patient per course. The additional cost of a hospital episode of ARF was taken from Silver et al. (2017) as $11,016 with a standard error of $279.59^8,9. The annual cost of RRT for end stage renal disease varies depending on treatment modality, such as dialysis or transplant, and a weighted average of $76,936, with a standard deviation of $15,387 was taken from the 2017 USRDS Annual Data Report¹⁰.

Health utilities

The health utility score of patients reflects the value of their heath state and is a measure of health related quality of life. It is expressed on a range between zero, the worst health state, and one, the best possible health state¹¹. The scores for patients who recover from septic shock depend on whether or not they suffered ARF. Patients who did not suffer ARF still underwent the stress of intensive care unit recovery, and their mean health utility scores and standard deviations were taken from Cuthbertson et al. (2010) at 1, 2.5, and 5 years as 0.666 (0.280), 0.701 (0.281), and 0.677 (0.301), respectively¹². Survivors of acute kidney problems were found to have a utility of 0.40 with a standard deviation of 0.37 at 60 days. We did not find evidence of improved health utilities over the 5 year period¹³. The health utility of death was valued at 0¹¹.

Transition probabilities

The transition probabilities of patients moving through the decision tree were informed by data from Marik’s study. This includes the probabilities of survival and ARF, see Table 1³. In Marik’s study, the Marik bundle’s probability of death and ARF were 0.085 and 0.097, with standard errors of 0.041 and 0.043 respectively. These compared to the standard care mortality and ARF probabilities of 0.404 and 0.234, with standard errors of 0.071 and 0.062 respectively. Full transition probabilities are available in Dataset 1.

Table 1. Transition probabilities from bundle decision to survival and recovery.

	Mean Standard Care (SE)	Mean Standard Care, Scenario 1 (SE)	Mean Marik Bundle (SE)
Mortality	0.404 (0.071)	0.255 (0.064)	0.085 (0.041)
ARF	0.234 (0.062)	0.170 (0.055)	0.097 (0.043)

Modelling additional scenarios

Scenario 1 uses probabilities from the literature for mortality and ARF; costs and utilities were unchanged. The attributable mortality risk and rate of ARF in the general population vary in the literature and are different to those reported in Marik’s control cohort. We used the mortality risk associated with ARF identified in Oppert et al. (2008) of 21.2% and the rate of sepsis-associated ARF in Lopes et al. (2009) of 17.0%^14–17. These are lower than the rates seen in Marik’s observation group. This led to an overall mortality rate of 25.5%, consistent with several studies on septic shock^15,18.

Scenario 2 represents a worst-case scenario, in which Marik’s bundle is ineffective and there is no difference in probabilities between the bundle and standard care. We modelled scenario 2 using the same transition probabilities found in scenario 1 for both arms of the decision tree. As in scenario 1, costs and utilities remained unchanged, though we carried the worst-case QALY outcome to illustrate the unlikely possibility of minor harm.

Probabilistic sensitivity analysis

Probabilistic sensitivity analysis was conducted on the decision tree by taking 1000 random resamples of values from the prior statistical distributions of model parameters. This propagates all uncertainty in the data forward to the results and provides useful information for decision making.

Results

The total economic cost of patients under standard care, including 5 year estimations of RRT, was $41,982 compared to the total economic cost of patients receiving the Marik bundle of $35,867, an expected saving of $6,115 per patient. The annual incidence of severe sepsis in the United States is around 300 cases per 100,000 population, suggesting a total expected cost saving over 2.5 years of $14.9bn USD⁹. Over the same period, patients gain an additional 1.46 QALY per case, or 3.6m QALYs over 2.5 years. Given the reduced costs and improved outcomes, the Marik bundle dominates standard care, as it both saves costs and increase health outcomes at the same time. The probabilistic sensitivity analysis shows this conclusion arises 93.6% of the time.

Scenario 1 reduces the mean expected economic cost of standard care to $38,068 per patient, compared to the Marik bundle at $35,478. The expected cost saving between the bundle and standard care was $2,590 per patient, a total saving of $6.3bn USD. Altering the transition probabilities to values from the literature also reduces the QALYs gained, with patients gaining 0.46 QALY per case, or 1.1m QALY over 2.5 years. In this scenario, the Marik bundle still dominates standard care by saving costs and improving health outcomes. The probabilistic sensitivity analysis shows this conclusion arises 87.8% of the time.

Scenario 2 is that there is no change to outcomes between Marik’s bundle and standard care. The mean cost per patient over 5 years is $37,022USD for standard care and $37,550USD for the Marik bundle, an increase of $528 per patient over a 5 year period. There is no difference in QALYs between treatment alternatives. We were unable to find any evidence of harm to the patient as a result of the bundle. Hydrocortisone and thiamine are already present in routine sepsis care, and the dosage of 6g of vitamin C per day has been shown to be safe unless contraindicated^19–21. In this scenario, assuming the intervention is universally adopted, no patients will have been harmed, but healthcare payers and providers would have spent an additional $1.3bn USD over 2.5 years. For the sake of illustration we have carried the worst possible health outcome from scenario 1 to scenario 2. The probabilistic sensitivity analysis shows a negative health outcome arises 2.5% of the time. Outcomes from the scenario analyses are listed in Table 2 below.

Table 2. Scenario analyses for adoption compared to baseline adoption outcomes.

	Cost savings	95% uncertainty interval	QALYs gained	95% uncertainty interval
Baseline	$14.9bn	[$-7.3bn – $33.9bn]	3.6m	[1.3m – 6.3m]
Scenario 1: less effective	$6.3bn	[-$8.4bn – $18.3bn]	1.1m	[-0.1m – 2.6m]
Scenario 2: ineffective	-$1.3bn	[-$0.9bn – -$0.7bn]	0	[-0.1m – 0]

Table of distributions used in sepsis model

Type	Name	Description	Mean	Standard Deviation
Normal	d_c_marik	Cost of Marik bundle	528	26.4
Normal	d_c_RRT_annual	Annual cost of renal replacement therapy	76936	15387.2
Normal	d_c_AKI	Additional hospital cost of kidney care	11016	279.59

Type	Name	Description	Alpha	Lambda
Gamma	d_c_standard	Cost of a standard sepsis episode in hospital	4.639735749	0.000143109

Type	Name	Description	Alpha	Beta
Beta	d_p_marik_mortality	Probability of death under Marik bundle	2.0909111	43.90913356
Beta	d_p_standard_mortality	Probability of death under standard care	11.49999991	11.49999991
Beta	d_p_ARF_marik	Probability of ARF under Marik bundle	0.712640801	10.45206508
Beta	d_p_ARF_standard	Probability of ARF under standard care	10.76399998	35.23599995
Beta	d_p_death_ARF	Mortality rate from ARF	83.04000003	89.96000004
Beta	d_p_ARF_RRT	Probability of chronic RRT from ARF	93.15109788	581.8568578
Beta	d_u_recover_1yr	Utility of recovery from ICU after 1 year	1.223639082	0.613656837
Beta	d_u_recover_2point5year	Utility of recovery from ICU after 2.5 years	1.159778093	0.494684237
Beta	d_u_recover_5year	Utility of recovery from ICU after 5 years	0.956980497	0.45658006

Dataset 1.Sepsis Distributions Table.

Data on distributions used in the analysis.

Discussion

We found that adopting the Marik bundle has a high likelihood to save billions of dollars and generate millions of extra QALYs under the conditions outlined in Marik’s paper and in an alternate scenario that uses other data. Under the ineffective treatment of scenario 2 costs are increased to health services by $0.5bn per year. These results reveal substantial opportunity costs in dollars and lives if we fail to implement and the bundle is ultimately found to be effective, even if the treatment effect is lower than purported by Dr Marik. If the bundle does not work then some costs have been incurred by hospitals for no heath gain. Not adopting the bundle because the evidence for effectiveness is currently uncertain could well be a poor strategy. The worst possible outcome from the base case is that we spend around $5,700 per QALY; given a conventional $50,000/QALY valuation, this gives us a net monetary benefit of $108bn over 2.5 years²².

The scenario analysis uses values at which the Marik bundle is less effective, specifically by aligning observation group figures with the literature. The mortality rate of sepsis under standard care was suggested by Marik to be 40.4%. Sepsis mortality has been declining in the US, from 46.9% in the early 1990s to 21.2% in 2014, declining by about 3% per annum and driven by improved organ support systems and protocoled early recognition and treatment^17,23,24. It might be that Marik’s study featured unusually sick patients in the control group. Study participants are often unrepresentative of the general population and the intervention group may have been less likely to suffer an adverse outcome²⁵. This progress in sepsis care management is complicated by the fact that claims data has shown concurrently increasing sepsis incidence and decreasing mortality, so the literature is conflicted^26,27. It is possible that the increased incidence from claims data is due to increased reimbursement received by US hospitals for sepsis compared to other diseases, and patients are being misdiagnosed. An increasing rate of misdiagnosis of patients that do not have sepsis increases the denominator of septic cases while mortality stays the same in the numerator, creating the illusion of declining mortality^17,27.

Regarding the mortality of Marik’s treatment group, we note the significant improvement in outcomes recommending the use of steroids in adults with septic shock by Annane et al. (2018)²⁸. Patients randomised to the steroid group (n = 614) showed a 6% absolute reduction in 90-day all-cause mortality when compared to placebo (n = 627). Therefore, the figures in Marik’s patient cohort may have been unusual, but they are plausible.

There is no current body of evidence that suggests this bundle is dangerous. Indeed, the combination of vitamin C and thiamine with steroids would have to cause an attributable death once in every 10 sepsis patients - by a hitherto unimagined and novel mechanism - to negate the modelled benefits of its early adoption.

Limitations

Our study excluded the costs of bundle implementation, including training, labour and the potential for high costs of de-implementing an unsuccessful treatment. These would have increased the cost of bundle implementation, so the incremental cost of the Marik bundle may be understated. We noted that hydrocortisone, as part of the bundle, was found by Venkatesh et al. (2018) to speed up resolution of shock and reduce the need for blood transfusions, so the in-hospital cost-savings for the treatment group may also be understated²¹. We were also unable to quantify the potential QALY gains from reductions in post-sepsis syndrome associated with the Marik bundle, understating the gains in utility from the bundle. There is also considerable uncertainty around the parameter estimates that are available. We did not have access to primary data, including clinical data on costs and transition probabilities for each patient, and were reliant on the literature.

Bundle adoption decision

An average hospital in the US may treat around 230 sepsis patients per year¹⁶. By implementing the bundle, it will spend an additional $528 per patient, or $121,440 per year. Conservative estimates from the scenario analysis shows cost savings of $2,590 and a gain of 0.46 QALYs per patient. If the treatment was effective for 47 patients out of 230, or 1 in 5, it will have paid for itself in terms of total economic costs. Comparing implementing the sepsis bundle to other hospital-based treatment studies shows that for 230 patients, the bundle costs less than a tenth of a standard phase I clinical trial, which run from $1.4m-$6.5m²⁹.

The case for not adopting the Marik bundle has several components. Scientific and empirical evidence is thin, and a single-site Vitamin C trial showing remarkable results only to be proven ineffective after a multi-site RCT is a clinical trope. If the bundle was ineffective, health systems will have added an unnecessary load to clinicians and implementers, which would then have to be de-implemented. Administrators and clinicians may be less likely to adopt novel treatments in the future. Introducing the bundle on the current evidence may also set a bad precedent for novel treatments, eroding the authority of the presiding physician and giving more credence to largely unproven interventions. The proposal in this paper does not replace the need for the clarity provided by good science and empirical research, but these are not always immediately available. We provide an approach to explicitly guide the interim decisions that must be made under these circumstances.

The Marik bundle is a somewhat unusual case relative to most ‘miracle’ interventions later found to be ineffective. As the analysis shows, it is an extremely cheap treatment with the potential to reduce rates of mortality and kidney damage at no risk to the patient. The delay between publication of the pilot study and results of the large RCT due in 2019 could create substantial opportunity costs in dollars and lives, and while there is not perfect evidence, under the circumstances it might be sufficient for hospitals and health systems to choose whether to conduct their own trials and not only independently verify their results, but also publish their findings and improve the availability of evidence around the treatment bundle.

The implementation decision of the Marik bundle relies upon the willingness of health administrators to use the available evidence to influence policy. We attempted to make the bundle choice as intuitive as possible, using straightforward trade-offs, simple modelling techniques, and a realistic decision process. Merlo et al. (2014) showed that for research to be accessible to decision-makers, it must be contextually relevant, contain little jargon, and put the terms of the implementation decision into terms that specify a trade-off that will be familiar to decision-makers³⁰. If the decision maker is unwilling to use the best available evidence now, but rather wishes to wait for definitive evidence they are risking incurring large costs for health care systems and for the patients they serve.

Conclusion

An explicit analysis of uncertain clinical outcomes is a useful adjunct to guide decision making where there is clinical ambiguity. This approach offers a valid alternative to the default of clinical inactivity when faced with uncertainty.

Data availability

Dataset 1. Sepsis Distributions Table: Data on distributions used in the analysis. 10.5256/f1000research.14619.d209630³¹

Competing interests

No competing interests were disclosed.

Grant information

The author(s) declared that no grants were involved in supporting this work.

Faculty Opinions recommended

References

1. Liu V, Escobar GJ, Greene JD, et al.: Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014; 312(1): 90–2. PubMed Abstract | Publisher Full Text
2. Alobaidi R, Basu RK, Goldstein SL, et al.: Sepsis-associated acute kidney injury. Semin Nephrol. 2015; 35(1): 2–11. PubMed Abstract | Publisher Full Text | Free Full Text
3. Marik PE, Khangoora V, Rivera R, et al.: Hydrocortisone, Vitamin C and Thiamine for the Treatment of Severe Sepsis and Septic Shock: A Retrospective Before-After Study. Chest. 2017; 151(6): 1229–38. PubMed Abstract | Publisher Full Text
4. Rezane S: The Marik Protocol: Have We Found a "Cure" for Severe Sepsis and Septic Shock? R.E.B.E.L. EM; 2017. Reference Source
5. Farkas J: PulmCrit - Metabolic sepsis resuscitation: the evidence behind Vitamin C. emcrit.org; 2017. Reference Source
6. SGEM: SGEM#174: Don't Believe the Hype - Vitamin C Cocktail for Sepsis. thesgem.com; 2017. Reference Source
7. Arefian H, Heublein S, Scherag A, et al.: Hospital-related cost of sepsis: A systematic review. J Infect. 2017; 74(2): 107–17. PubMed Abstract | Publisher Full Text
8. Silver SA, Long J, Zheng Y, et al.: Cost of Acute Kidney Injury in Hospitalized Patients. J Hosp Med. 2017; 12(2): 70–6. PubMed Abstract | Publisher Full Text
9. Mayr FB, Yende S, Angus DC: Epidemiology of severe sepsis. Virulence. 2014; 5(1): 4–11. PubMed Abstract | Publisher Full Text | Free Full Text
10. USRDS: 2017 ADR Reference Table K: Economic Costs of ESRD. 2017. Reference Source
11. Torrance GW: Measurement of health state utilities for economic appraisal. J Health Econ. 1986; 5(1): 1–30. PubMed Abstract | Publisher Full Text
12. Cuthbertson BH, Roughton S, Jenkinson D, et al.: Quality of life in the five years after intensive care: a cohort study. Crit Care. 2010; 14(1): R6. PubMed Abstract | Publisher Full Text | Free Full Text
13. Johansen KL, Smith MW, Unruh ML, et al.: Predictors of health utility among 60-day survivors of acute kidney injury in the Veterans Affairs/National Institutes of Health Acute Renal Failure Trial Network Study. Clin J Am Soc Nephrol. 2010; 5(8): 1366–72. PubMed Abstract | Publisher Full Text | Free Full Text
14. Oppert M, Engel C, Brunkhorst FM, et al.: Acute renal failure in patients with severe sepsis and septic shock--a significant independent risk factor for mortality: results from the German Prevalence Study. Nephrol Dial Transplant. 2008; 23(3): 904–9. PubMed Abstract | Publisher Full Text
15. Lopes JA, Jorge S, Resina C, et al.: Acute kidney injury in patients with sepsis: a contemporary analysis. Int J Infect Dis. 2009; 13(2): 176–81. PubMed Abstract | Publisher Full Text
16. Angus DC, Linde-Zwirble WT, Lidicker J, et al.: Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001; 29(7): 1303–10. PubMed Abstract | Publisher Full Text
17. Rhee C, Dantes R, Epstein L, et al.: Incidence and Trends of Sepsis in US Hospitals Using Clinical vs Claims Data, 2009-2014. JAMA. 2017; 318(13): 1241–9. PubMed Abstract | Publisher Full Text | Free Full Text
18. Rhee C, Dantes R, Epstein L, et al.: Incidence and Trends of Sepsis in US Hospitals Using Clinical vs Claims Data, 2009-2014. JAMA. 2017; 318(13): 1241–9. PubMed Abstract | Publisher Full Text | Free Full Text
19. Mallat J, Lemyze M, Thevenin D: Do not forget to give thiamine to your septic shock patient! J Thorac Dis. 2016; 8(6): 1062–6. PubMed Abstract | Publisher Full Text | Free Full Text
20. Ohno S, Ohno Y, Suzuki N, et al.: High-dose vitamin C (ascorbic acid) therapy in the treatment of patients with advanced cancer. Anticancer Res. 2009; 29(3): 809–15. PubMed Abstract
21. Venkatesh B, Finfer S, Cohen J, et al.: Adjunctive Glucocorticoid Therapy in Patients with Septic Shock. N Engl J Med. 2018; 378(9): 797–808. PubMed Abstract | Publisher Full Text
22. Neumann PJ, Cohen JT, Weinstein MC: Updating cost-effectiveness--the curious resilience of the $50,000-per-QALY threshold. N Engl J Med. 2014; 371(9): 796–7. PubMed Abstract | Publisher Full Text
23. Stevenson EK, Rubenstein AR, Radin GT, et al.: Two decades of mortality trends among patients with severe sepsis: a comparative meta-analysis*. Crit Care Med. 2014; 42(3): 625–31. PubMed Abstract | Publisher Full Text | Free Full Text
24. Levy MM, Rhodes A, Phillips GS, et al.: Surviving Sepsis Campaign: association between performance metrics and outcomes in a 7.5-year study. Intensive Care Med. 2014; 40(11): 1623–33. PubMed Abstract | Publisher Full Text
25. Kalata P, Martus P, Zettl H, et al.: Differences between clinical trial participants and patients in a population-based registry: the German Rectal Cancer Study vs. the Rostock Cancer Registry. Dis Colon Rectum. 2009; 52(3): 425–37. PubMed Abstract | Publisher Full Text
26. Glück T: Mortality from Severe Sepsis Decreasing in the U.S. www.jwatch.org: NEJM. 2013. Reference Source
27. Dressler DD: Is Sepsis Incidence and Mortality Changing in U.S. Hospitals?www.jwatch.org: NEJM. 2017. Reference Source
28. Annane D, Renault A, Brun-Buisson C, et al.: Hydrocortisone plus Fludrocortisone for Adults with Septic Shock. N Engl J Med. 2018; 378(9): 809–18. PubMed Abstract | Publisher Full Text
29. Sertkaya A, Wong HH, Jessup A, et al.: Key cost drivers of pharmaceutical clinical trials in the United States. Clin Trials. 2016; 13(2): 117–26. PubMed Abstract | Publisher Full Text
30. Merlo G, Page K, Ratcliffe J, et al.: Bridging the gap: exploring the barriers to using economic evidence in healthcare decision making and strategies for improving uptake. Appl Health Econ Health Policy. 2015; 13(3): 303–9. PubMed Abstract | Publisher Full Text
31. Blythe R, Cook D, Graves N: Dataset 1 in: Scepticaemia: The impact on the health system and patients of delaying new treatments with uncertain evidence; a case study of the sepsis bundle. F1000Research. 2018. Data Source

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 26 Apr 2018

Author details Author details

¹ Australian Centre for Health Services Innovation, School of Public Health and Social Work, Institute for Health and Biomedical Innovation, Queensland University of Technology, Brisbane, Queensland, Australia
² Intensive Care Unit, Princess Alexandra Hospital, Brisbane, Queensland, Australia

Robin Blythe
Roles: Conceptualization, Formal Analysis, Investigation, Methodology, Project Administration, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

David Cook
Roles: Conceptualization, Methodology, Writing – Review & Editing

Nicholas Graves
Roles: Conceptualization, Formal Analysis, Methodology, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

The author(s) declared that no grants were involved in supporting this work.

Article Versions (2)

version 2

Revised

Published: 05 Jul 2018, 7:500

https://doi.org/10.12688/f1000research.14619.2

version 1

Published: 26 Apr 2018, 7:500

https://doi.org/10.12688/f1000research.14619.1

Copyright

© 2018 Blythe R et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

0

SEE MORE DETAILS

CITE

how to cite this article

Blythe R, Cook D and Graves N. Scepticaemia: The impact on the health system and patients of delaying new treatments with uncertain evidence; a case study of the sepsis bundle [version 2; peer review: 2 approved]. F1000Research 2018, 7:500 (https://doi.org/10.12688/f1000research.14619.2)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 1

VERSION 1

PUBLISHED 26 Apr 2018

Views

23

Reviewer Report 04 Jun 2018

Adam Visser, Toowoomba Hospital, Toowoomba, Qld, Australia

Approved

https://doi.org/10.5256/f1000research.15909.r34658

This fascinating paper uses a simple health economics model to argue that delaying the widespread implementation of the Vitamin C protocol as described by Marik and colleagues would lead to the loss of enormous potential savings to the health industry. ... Continue reading

This fascinating paper uses a simple health economics model to argue that delaying the widespread implementation of the Vitamin C protocol as described by Marik and colleagues would lead to the loss of enormous potential savings to the health industry. The argument is well presented, easily followed, and sound, with some exceptions.

There is likely to be some inaccuracy in the figures presented. The authors state that there are 1.5 million cases of sepsis in the United States yearly (no reference supplied, but data easily available via the CDC). Whilst there may indeed be this number who meet the recently updated definition of sepsis, this large cohort of patients are different to those who have a mortality rate of 40% with standard care. This group, as described in Marik’s paper, are a much sicker population. Their number is likely to be significantly fewer than 1.5 million. Whilst it is not known whether Vitamin C may have benefits in a less sick population, it is unlikely that ‘early adopters’ of this therapy will consider it in those that are not towards the severe end of the sepsis spectrum. Reducing the exposed population in the model from 1.5 million will reduce the proposed savings by a proportional amount.

Similarly, the quoted incidence of requirement for renal replacement therapy is overstated. It is true that the Marik Vitamin C-treated group had an incidence of RRT-requiring AKI of 9.7%. However, the Marik paper does not state that these patients all went on to require ongoing RRT for ESRF (at the authors’ quoted cost of $77,000 per year). The majority of patients who require RRT in the setting of sepsis-induced AKI will recover their renal function either completely, or enough that they do not require ongoing dialysis. Again, reducing the stated financial benefits of this aspect of the Vitamin C treatment will reduce the long-term economic attractiveness of the therapy.

Overall however, the authors are correct in their statement that the economic implications of adopting the ‘bundle’ will be positive. Furthermore, the economic implications should be secondary to the beneficial effects the treatment may have on mortality. Again, this therapy is unlikely to be offered to 1.5 million patients per year, but whatever the number, those who receive it are likely to benefit.

A major strength of the paper is their use of a more pessimistic scenario (scenario 1) than published in the Marik study to demonstrate the robustness of their figures. The Marik-stated improvement in mortality, from 40% to 8%, is likely to not be repeated in larger studies, as it is simply too great to be plausible. Reducing the relative risk reduction such that mortality is reduced from 21% to 8% may be more realistic, and yet the economic figures still add up to considerable savings.

It would perhaps also be useful to see a model where baseline mortality of 40% (ie the very sick cohort from Marik) have their mortality reduced to, say, 30% (ie a more plausible magnitude of effect from Vitamin C).

The authors have done well to pick this particular therapy to demonstrate their model. The Vitamin C bundle is unique in recent years, in that it is a cheap and seemingly safe therapy. Their model is unlikely to be able to be used for supporting other therapies that emerge in coming years, given the likelihood that these therapies will come from the pharmaceutical industry and have large price tags attached.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Partly
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Critical care medicine, sepsis

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Respond or Comment

Views

26

Reviewer Report 11 May 2018

Paul E. Marik, Division of Pulmonary and Critical Care Medicine, Eastern Virginia Medical School, Norfolk, VA, USA

Approved

https://doi.org/10.5256/f1000research.15909.r33910

This is an outstanding paper that objectively evaluates the potential benefit of the vitamin C protocol for sepsis.This evaluation was impeccably conducted and reported. The impact of this paper is significant.

A few comments that will improve ... Continue reading

This is an outstanding paper that objectively evaluates the potential benefit of the vitamin C protocol for sepsis.This evaluation was impeccably conducted and reported. The impact of this paper is significant.

A few comments that will improve the paper:

1.The authors state "The delay between publication of the pilot study and results of the large RCT due in mid-2018 could create substantial opportunity costs". This study which the authors cite is the NIH funded study due for completion in mid-2018 which is investigating vitamin c (ALONE) and not the combination AND in patients with ARDS (not specifically sepsis). Secondly, this study is not powered for a mortality difference. Currently according to Clintrials.gov there are 8 RCTS testing the combination and 3 testing Vitamin C alone. This suggests that it may take at least until the end of 2019 before the results of any of these studies are available and we can make firm conclusions.

2. The authors refer to the "sepsis bundle"; I would avoid using this term as it will create confusion with the Surviving Sepsis Bundle.. widely known as the Sepsis Bundle. I would call it the Vitamin C protocol, the Marik Protocol, or something similar.

3. Approximately 50% of sepsis survivors develop the post-sepsis syndrome, defined as severe physical and cognitive dysfunction not unlike PTSD. This syndrome is associated with enormous suffering and high costs. It is likely that the Marik Protocol will reduce the incidence and severity of this syndrome. While the cost savings are difficult to quantitate, I think that this proposition should be mentioned in the discussion section.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests: I am the first author of the primary study under review.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Respond or Comment

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 26 Apr 2018

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 05 Jul 18
Version 1 26 Apr 18	read	read

Paul E. Marik, Eastern Virginia Medical School, Norfolk, USA
Adam Visser, Toowoomba Hospital, Toowoomba, Australia

Comments on this article

All Comments(0)

Add a comment

Sign up for content alerts

Browse by related subjects

Back to all reports

Reviewer Report

23 Views

04 Jun 2018 | for Version 1

Adam Visser, Toowoomba Hospital, Toowoomba, Qld, Australia

23 Views Cite this report Responses(0)

Approved

This fascinating paper uses a simple health economics model to argue that delaying the widespread implementation of the Vitamin C protocol as described by Marik and colleagues would lead to the loss of enormous potential savings to the health industry. The argument is well presented, easily followed, and sound, with some exceptions.

There is likely to be some inaccuracy in the figures presented. The authors state that there are 1.5 million cases of sepsis in the United States yearly (no reference supplied, but data easily available via the CDC). Whilst there may indeed be this number who meet the recently updated definition of sepsis, this large cohort of patients are different to those who have a mortality rate of 40% with standard care. This group, as described in Marik’s paper, are a much sicker population. Their number is likely to be significantly fewer than 1.5 million. Whilst it is not known whether Vitamin C may have benefits in a less sick population, it is unlikely that ‘early adopters’ of this therapy will consider it in those that are not towards the severe end of the sepsis spectrum. Reducing the exposed population in the model from 1.5 million will reduce the proposed savings by a proportional amount.

Similarly, the quoted incidence of requirement for renal replacement therapy is overstated. It is true that the Marik Vitamin C-treated group had an incidence of RRT-requiring AKI of 9.7%. However, the Marik paper does not state that these patients all went on to require ongoing RRT for ESRF (at the authors’ quoted cost of $77,000 per year). The majority of patients who require RRT in the setting of sepsis-induced AKI will recover their renal function either completely, or enough that they do not require ongoing dialysis. Again, reducing the stated financial benefits of this aspect of the Vitamin C treatment will reduce the long-term economic attractiveness of the therapy.

Overall however, the authors are correct in their statement that the economic implications of adopting the ‘bundle’ will be positive. Furthermore, the economic implications should be secondary to the beneficial effects the treatment may have on mortality. Again, this therapy is unlikely to be offered to 1.5 million patients per year, but whatever the number, those who receive it are likely to benefit.

A major strength of the paper is their use of a more pessimistic scenario (scenario 1) than published in the Marik study to demonstrate the robustness of their figures. The Marik-stated improvement in mortality, from 40% to 8%, is likely to not be repeated in larger studies, as it is simply too great to be plausible. Reducing the relative risk reduction such that mortality is reduced from 21% to 8% may be more realistic, and yet the economic figures still add up to considerable savings.

It would perhaps also be useful to see a model where baseline mortality of 40% (ie the very sick cohort from Marik) have their mortality reduced to, say, 30% (ie a more plausible magnitude of effect from Vitamin C).

The authors have done well to pick this particular therapy to demonstrate their model. The Vitamin C bundle is unique in recent years, in that it is a cheap and seemingly safe therapy. Their model is unlikely to be able to be used for supporting other therapies that emerge in coming years, given the likelihood that these therapies will come from the pharmaceutical industry and have large price tags attached.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Partly
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Critical care medicine, sepsis

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

26 Views

11 May 2018 | for Version 1

Paul E. Marik, Division of Pulmonary and Critical Care Medicine, Eastern Virginia Medical School, Norfolk, VA, USA

26 Views Cite this report Responses(0)

Approved

This is an outstanding paper that objectively evaluates the potential benefit of the vitamin C protocol for sepsis.This evaluation was impeccably conducted and reported. The impact of this paper is significant.

A few comments that will improve the paper:

1.The authors state "The delay between publication of the pilot study and results of the large RCT due in mid-2018 could create substantial opportunity costs". This study which the authors cite is the NIH funded study due for completion in mid-2018 which is investigating vitamin c (ALONE) and not the combination AND in patients with ARDS (not specifically sepsis). Secondly, this study is not powered for a mortality difference. Currently according to Clintrials.gov there are 8 RCTS testing the combination and 3 testing Vitamin C alone. This suggests that it may take at least until the end of 2019 before the results of any of these studies are available and we can make firm conclusions.

2. The authors refer to the "sepsis bundle"; I would avoid using this term as it will create confusion with the Surviving Sepsis Bundle.. widely known as the Sepsis Bundle. I would call it the Vitamin C protocol, the Marik Protocol, or something similar.

3. Approximately 50% of sepsis survivors develop the post-sepsis syndrome, defined as severe physical and cognitive dysfunction not unlike PTSD. This syndrome is associated with enormous suffering and high costs. It is likely that the Marik Protocol will reduce the incidence and severity of this syndrome. While the cost savings are difficult to quantitate, I think that this proposition should be mentioned in the discussion section.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests

I am the first author of the primary study under review.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

[1] 1. Liu V, Escobar GJ, Greene JD, et al.: Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014; 312(1): 90–2. PubMed Abstract | Publisher Full Text

[2] 2. Alobaidi R, Basu RK, Goldstein SL, et al.: Sepsis-associated acute kidney injury. Semin Nephrol. 2015; 35(1): 2–11. PubMed Abstract | Publisher Full Text | Free Full Text

[3] 3. Marik PE, Khangoora V, Rivera R, et al.: Hydrocortisone, Vitamin C and Thiamine for the Treatment of Severe Sepsis and Septic Shock: A Retrospective Before-After Study. Chest. 2017; 151(6): 1229–38. PubMed Abstract | Publisher Full Text

[4] 4. Rezane S: The Marik Protocol: Have We Found a "Cure" for Severe Sepsis and Septic Shock? R.E.B.E.L. EM; 2017. Reference Source

[5] 5. Farkas J: PulmCrit - Metabolic sepsis resuscitation: the evidence behind Vitamin C. emcrit.org; 2017. Reference Source

[6] 6. SGEM: SGEM#174: Don't Believe the Hype - Vitamin C Cocktail for Sepsis. thesgem.com; 2017. Reference Source

[7] 7. Arefian H, Heublein S, Scherag A, et al.: Hospital-related cost of sepsis: A systematic review. J Infect. 2017; 74(2): 107–17. PubMed Abstract | Publisher Full Text

[8] 8. Silver SA, Long J, Zheng Y, et al.: Cost of Acute Kidney Injury in Hospitalized Patients. J Hosp Med. 2017; 12(2): 70–6. PubMed Abstract | Publisher Full Text

[9] 9. Mayr FB, Yende S, Angus DC: Epidemiology of severe sepsis. Virulence. 2014; 5(1): 4–11. PubMed Abstract | Publisher Full Text | Free Full Text

[10] 10. USRDS: 2017 ADR Reference Table K: Economic Costs of ESRD. 2017. Reference Source

[11] 11. Torrance GW: Measurement of health state utilities for economic appraisal. J Health Econ. 1986; 5(1): 1–30. PubMed Abstract | Publisher Full Text

[12] 12. Cuthbertson BH, Roughton S, Jenkinson D, et al.: Quality of life in the five years after intensive care: a cohort study. Crit Care. 2010; 14(1): R6. PubMed Abstract | Publisher Full Text | Free Full Text

[13] 13. Johansen KL, Smith MW, Unruh ML, et al.: Predictors of health utility among 60-day survivors of acute kidney injury in the Veterans Affairs/National Institutes of Health Acute Renal Failure Trial Network Study. Clin J Am Soc Nephrol. 2010; 5(8): 1366–72. PubMed Abstract | Publisher Full Text | Free Full Text

[14] 14. Oppert M, Engel C, Brunkhorst FM, et al.: Acute renal failure in patients with severe sepsis and septic shock--a significant independent risk factor for mortality: results from the German Prevalence Study. Nephrol Dial Transplant. 2008; 23(3): 904–9. PubMed Abstract | Publisher Full Text

[15] 15. Lopes JA, Jorge S, Resina C, et al.: Acute kidney injury in patients with sepsis: a contemporary analysis. Int J Infect Dis. 2009; 13(2): 176–81. PubMed Abstract | Publisher Full Text

[16] 16. Angus DC, Linde-Zwirble WT, Lidicker J, et al.: Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001; 29(7): 1303–10. PubMed Abstract | Publisher Full Text

[17] 17. Rhee C, Dantes R, Epstein L, et al.: Incidence and Trends of Sepsis in US Hospitals Using Clinical vs Claims Data, 2009-2014. JAMA. 2017; 318(13): 1241–9. PubMed Abstract | Publisher Full Text | Free Full Text

[18] 18. Rhee C, Dantes R, Epstein L, et al.: Incidence and Trends of Sepsis in US Hospitals Using Clinical vs Claims Data, 2009-2014. JAMA. 2017; 318(13): 1241–9. PubMed Abstract | Publisher Full Text | Free Full Text

[19] 19. Mallat J, Lemyze M, Thevenin D: Do not forget to give thiamine to your septic shock patient! J Thorac Dis. 2016; 8(6): 1062–6. PubMed Abstract | Publisher Full Text | Free Full Text

[20] 20. Ohno S, Ohno Y, Suzuki N, et al.: High-dose vitamin C (ascorbic acid) therapy in the treatment of patients with advanced cancer. Anticancer Res. 2009; 29(3): 809–15. PubMed Abstract

[21] 21. Venkatesh B, Finfer S, Cohen J, et al.: Adjunctive Glucocorticoid Therapy in Patients with Septic Shock. N Engl J Med. 2018; 378(9): 797–808. PubMed Abstract | Publisher Full Text

[22] 22. Neumann PJ, Cohen JT, Weinstein MC: Updating cost-effectiveness--the curious resilience of the $50,000-per-QALY threshold. N Engl J Med. 2014; 371(9): 796–7. PubMed Abstract | Publisher Full Text

[23] 23. Stevenson EK, Rubenstein AR, Radin GT, et al.: Two decades of mortality trends among patients with severe sepsis: a comparative meta-analysis*. Crit Care Med. 2014; 42(3): 625–31. PubMed Abstract | Publisher Full Text | Free Full Text

[24] 24. Levy MM, Rhodes A, Phillips GS, et al.: Surviving Sepsis Campaign: association between performance metrics and outcomes in a 7.5-year study. Intensive Care Med. 2014; 40(11): 1623–33. PubMed Abstract | Publisher Full Text

[25] 25. Kalata P, Martus P, Zettl H, et al.: Differences between clinical trial participants and patients in a population-based registry: the German Rectal Cancer Study vs. the Rostock Cancer Registry. Dis Colon Rectum. 2009; 52(3): 425–37. PubMed Abstract | Publisher Full Text

[26] 26. Glück T: Mortality from Severe Sepsis Decreasing in the U.S. www.jwatch.org: NEJM. 2013. Reference Source

[27] 27. Dressler DD: Is Sepsis Incidence and Mortality Changing in U.S. Hospitals?www.jwatch.org: NEJM. 2017. Reference Source

[28] 28. Annane D, Renault A, Brun-Buisson C, et al.: Hydrocortisone plus Fludrocortisone for Adults with Septic Shock. N Engl J Med. 2018; 378(9): 809–18. PubMed Abstract | Publisher Full Text

[29] 29. Sertkaya A, Wong HH, Jessup A, et al.: Key cost drivers of pharmaceutical clinical trials in the United States. Clin Trials. 2016; 13(2): 117–26. PubMed Abstract | Publisher Full Text

[30] 30. Merlo G, Page K, Ratcliffe J, et al.: Bridging the gap: exploring the barriers to using economic evidence in healthcare decision making and strategies for improving uptake. Appl Health Econ Health Policy. 2015; 13(3): 303–9. PubMed Abstract | Publisher Full Text

[31] 31. Blythe R, Cook D, Graves N: Dataset 1 in: Scepticaemia: The impact on the health system and patients of delaying new treatments with uncertain evidence; a case study of the sepsis bundle. F1000Research. 2018. Data Source

Scepticaemia: The impact on the health system and patients of delaying new treatments with uncertain evidence; a case study of the sepsis bundle

Abstract

Keywords

Revised Amendments from Version 1

Introduction

Methods

Figure 1. Sepsis decision tree model using TreeAge Pro 2017.

Costs

Health utilities

Transition probabilities

Table 1. Transition probabilities from bundle decision to survival and recovery.

Modelling additional scenarios

Probabilistic sensitivity analysis

Results

Table 2. Scenario analyses for adoption compared to baseline adoption outcomes.

Discussion

Limitations

Bundle adoption decision

Conclusion

Data availability

Competing interests

Grant information

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

The problem

How to fix it

Competing Interests Policy

Stay Updated