How adjusting elicited health utilities after the fact can adversely affect shared decision making

Edouard Kujawski

doi:10.12688/f1000research.128862.1

Home Browse How adjusting elicited health utilities after the fact can adversely...

ALL Metrics

-

Views

-

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

How adjusting elicited health utilities after the fact can adversely affect shared decision making

[version 1; peer review: 1 approved with reservations, 1 not approved]

Edouard Kujawski

PUBLISHED 19 Dec 2022

Author details Author details

Independent researcher, Berkeley, CA, 94708, USA

Edouard Kujawski
Roles: Conceptualization, Formal Analysis, Investigation, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the Health Services gateway.

Abstract

Background: The elicitation of inconsistent health-state utility values (HSUVs) is a prevalent problem. There are two approaches to address this problem: (1) intervention during the elicitation process to ensure that patients estimate consistent HSUVs; (2) no intervention during the elicitation process and inconsistent HSUVs are adjusted after the fact. This paper studies three models recently proposed for adjusting inconsistent HSUVs and consistent HSUVs that some may consider unrealistic.
Analysis: The three models are analyzed using a sound theoretical framework: the mathematical equivalence of HSUVs elicited using the standard gamble and probabilities, the Fréchet bounds, and preference theory. It is proven that none of these models accounts for the Fréchet lower bound and health conditions that are preference substitutes.
Results: A clinical vignette proves these models may recommend treatments that result in premature death over treatments that cause acceptable adverse effects.
Conclusions: The three models are incorrect and may mislead patients and physicians to poor medical decisions. In the spirit of shared decision making, patients should be given the opportunity to reassess inconsistent HSUVs and confirm that the revised HSUVs reflect their preferences.

Keywords

Elicitation intervention, Fréchet bounds, Health-state utility values, Preference complements, Preference substitutes, Reasonableness test

Corresponding author: Edouard Kujawski

Competing interests: No competing interests were disclosed.

Grant information: The author(s) declared that no grants were involved in supporting this work.

Copyright: © 2022 Kujawski E. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Kujawski E. How adjusting elicited health utilities after the fact can adversely affect shared decision making [version 1; peer review: 1 approved with reservations, 1 not approved]. F1000Research 2022, 11:1533 (https://doi.org/10.12688/f1000research.128862.1) First published: 19 Dec 2022, 11:1533 (https://doi.org/10.12688/f1000research.128862.1) Latest published: 19 Dec 2022, 11:1533 (https://doi.org/10.12688/f1000research.128862.1)

Introduction

The elicitation of inconsistent health-state utility values (HSUVs) is a prevalent problem. A rational person should not prefer a joint health state (JHS) to any of the constituent single health states (SHSs). This property is termed logical consistency.¹ For instance, at the individual level, 41% of the HSUVs for the JHS (incontinence & impotence) elicited by Dale et al.¹ violated this property. Given the mathematical equivalence of HSUVs elicited using the standard gamble (SG) and probabilities, logical consistency corresponds to the Fréchet upper bound (FUB).² It is highly likely that the percentage of inconsistent HSUVs was higher than 41% because Dale et al. did not consider the Fréchet lower bound (FLB).² The later has significant implications for the disutility of multiple coexisting morbidities: the joint disutility cannot exceed immediate death (ID) or the sum of the individual disutilities.²

Two approaches to the problem of eliciting inconsistent HSUVs have been proposed:

1. Prevent inconsistent elicited HSUVs through interviewers intervening whenever necessary to ensure that patients estimate consistent HSUVs.³
2. No intervention during the elicitation process. Inconsistent elicited HSUVs are adjusted after the fact.⁴^,⁵

The first approach requires trained and knowledgeable interviewers. The second approach requires realistic and mathematically valid models. However, the adjusted HSUVs may not accurately represent a patient’s preferences.

Triantaphyllou and Yanase⁴ (referred to as T-Y in this paper) proposed three models for adjusting inconsistent as well as consistent HSUVs which they say “may still not be realistic”:

– Model (i): “Readjusting the original health utility values via an error minimization approach based on the monotonicity property.”
– Model (ii): “Multiplicative functions for health states and a new model for adjusting the initial health-state utilities.”
– Model (iii): “A combined approach for adjusting the initial health-state utilities.”

This study has a sound theoretical framework: the mathematical equivalence of HSUVs elicited using SG and probabilities.² It uses probability theory and preference theory to prove that the T-Y models are incorrect. A simple clinical vignette (Box 1) demonstrates that these models can be misleading. Therefore, they are inappropriate for shared decision making (SDM) where reliable HSUVs are critical for patients and physicians to decide upon a preferred treatment.⁶

Box 1. Clinical vignette.

At their annual physical examination, patient HP is diagnosed to suffer from the asymptomatic health condition X. HP is otherwise in good health. HP is neither a trained nor innate probability assessor.

The physician is a proponent of SDM, and HP agrees to participate.

The physician informs HP that:

– X would reduce their expected years of life from 15 to 7, unless treated
– Treatment T_x has a 100% success rate in curing X
– T_x has a 100% probability of two side effects: $a_{i}$ and $b_{j} .$
– HP agrees to partake in assessing the HSUVs for $a_{i}$ , $b_{j}$ and $a_{i} & b_{j} .$

This paper proceeds as follows. The theoretical framework used for analyzing the T-Y models is presented. These models are analyzed, and it is demonstrated that they are inappropriate for life-critical SDM. Concluding remarks are presented.

Methods

Theoretical framework

Health-state utility values

Health states are identified by health conditions (HCs) (also termed “attributes” and “dimensions”) and severity levels. Each combination of levels of HCs represents a unique health state. HSUVs measure the strength of a person’s subjective preferences for health-related quality of life (HRQL).⁷ They are cardinal values specified on the (ID = 0.0, perfect health (PH) = 1.0) scale.

Estimating HSUVs is challenging. People are affected by the information which they receive, emotional factors, and elicitation methods.⁷ HSUVs elicited by different methods may not agree and can affect treatment choices.⁸ The SG has a theoretical foundation in von Neumann-Morgenstern expected utility theory⁹, which establishes the validity of HSUVs elicited using the SG as a measure for HRQL.¹⁰ Using arbitrary scales for HSUVs can lead to serious errors.¹¹^,p.17

Probabilities: Fréchet inequalities

Fréchet¹² proved that the joint probability of two events is bounded by the marginal probabilities of each event regardless of the dependence between them. For two events A and B,

(1)

max (0.0, P (A) + P (B) - 1.0) \leq P (A & B) \leq min (P (A), P (B)) .

Few people realize they assign inconsistent values to joint probabilities. Osherson et al.¹³ state: “It is striking to observe, for example, how few people realize that it is inconsistent to attribute probabilities of 0.8 to each of two sentences and probability 0.5 to their conjunction.” From (1), the conjunction of 0.8 and 0.8 cannot be less than 0.6: $P^{FLB} (0.8 \land 0.8) = 0.8 + 0.8 - 1.0 .$

Mathematical equivalence of HSUVs and probabilities

In the SG, an individual is asked to make the hypothetical choice between living for T years with health state $a_{i}$ (health condition $A$ with severity level $i$ ) and a gamble with a binary outcome (probability p of living in PH for T years or ID with probability $(1.0 - p)) .$ The probability p is varied until the individual is indifferent between living T years with $a_{i}$ and the gamble. The indifference probability $p (a_{i})$ corresponds to the individual’s HSUV for $a_{i}$ :¹⁰ $U (a_{i}) = p (a_{i}) .$ Given the mathematical equivalence of HSUVs elicited using the SG and probabilities, probability theory can be applied to the problem of identifying inconsistent HSUVs.²

Consistent HSUVs: Fréchet inequalities

Given that HSUVs are mathematically equivalent to probabilities, the Fréchet inequalities play an important role in identifying inconsistent HSUVs. $U (a_{i} & b_{j})$ is bounded by the FUB and FLB on conjunction irrespective of preference interactions:²

(2a)

max (0.0, U (a_{i}) + U (b_{j}) - 1.0) \leq U (a_{i} & b_{j}) \leq min (U (a_{i}), U (b_{j})) .

(2b)

U^{FUB} (a_{i} & b_{j}) = min (U (a_{i}), U (b_{j})) .

(2c)

U^{FLB} (a_{i} & b_{j}) = max (0.0, U (a_{i}) + U (b_{j}) - 1.0) .

The FUB (2b) ensures logical consistency.¹ The FLB (2c) has significant implications for the disutility of multiple coexisting morbidities. The joint disutility cannot exceed ID or the sum of the individual disutilities:²

(3)

{\bar{U}}^{FLB} (a_{i} & b_{j}) = min (1.0, \bar{U} (a_{i}) + \bar{U} (b_{j}))

where

\bar{U} (\cdot) \equiv 1.0 - U (\cdot) .

Preference interactions

HCs can be mutually utility independent (MUI), preference complements (PCs), or preference substitutes (PSs).² If a patient’s preference for condition A is independent of the level of condition B and vice versa, A and B are said to be MUI: $U^{MUI} (a_{i} & b_{j}) = U (a_{i}) \times U (b_{j})$ . If a patient believes that both A and B need to improve for their HRQL to improve, A and are said to be PCs. PC HSUVs are positively correlated: $U^{MUI} (a_{i} & b_{j}) < U^{PC} (a_{i} & b_{j}) \leq FUB$ . If a person believes that only A or only B needs to improve for their HRQL to improve, A and B are PSs. PS HSUVs are negatively correlated: $FLB \leq U^{PS} (a_{i} & b_{j}) < U^{MUI} (a_{i} & b_{j}) .$

Quality-adjusted life-years

The linear quality-adjusted life-year (simply termed the QALY) is presently the principal model for medical decision making (MDM). The expected number of QALYs for living $y_{i}$ years in a health state $x_{i}$ which has a probability of occurrence $p (x_{i})$ is¹⁰

(4)

EQ (p (x_{i}), x_{i}, y_{i}) = p (x_{i}) \times U (x_{i}) \times y_{i} + (1.0 - p (x_{i})) \times y_{i} .

Analysis of T-Y models

In the following subsections, the T-Y models⁴ are analyzed using the above theoretical framework and data shown in Table 1.¹⁴ The analysis was done using Microsoft Excel 2016.

Table 1. Elicited and adjusted HSUVs using T-Y models.

Methods	U (a_i)	U (b_j)	U (a_i&b_j)
No intervention	0.62	0.73	0.15
Model (i)	0.62	0.73	0.15
Model (ii)	0.62	0.73	0.45
Model (iii)	0.62	0.73	$\in$ [0.45, 0.62]

Model (i):⁴ “Readjusting the original health utility values via an error minimization approach based on the monotonicity property.”

The monotonicity property requires that a rational individual should not prefer a JHS to any of the constituent health states. Hence, Model (i) satisfies the FUB (2b). For consistency with probability theory, HSUVs elicited using the SG are also required to satisfy the FLB (2c). Model (i) does not address the FLB. For instance, it does not identify the HSUVs $U (a_{i}) = 0.62, U (b_{j}) = 0.73,$ $U (a_{i} & b_{j}) = 0.15$ as inconsistent: $U (a_{i} & b_{j}) < U^{FLB} (0.62 \land 0.73) = 0.35 (= 0.62 + 0.73 - 1.0) .$

Model (ii):⁴ “Multiplicative functions for health states and a new model for adjusting the initial health-state utilities.”

Model (ii) posits $U^{(ii)} (a_{i} & b_{j}) = U (a_{i}) \times U (b_{j}) .$ This assumes that health conditions $a_{i}$ and $b_{j}$ are MUI.

Keeney & Raiffa advocated assuming MUI with the significantly important qualifier¹⁵^{,p. 244}: “the utility independence assumptions are appropriate in many realistic problems”. Moreover, their focus was principally on decision making outside of the medical domain. More recently, Howard and Abbas wrote¹⁶^{,p. 578}

“We have several issues with this type of ‘utility independence’ reasoning … Enforcing these ‘utility independence’ assumptions result in functional forms that are simple, but quite frequently they will not represent the preference of the decision maker.”

Experimental studies have concluded that the multiplicative model is not a suitable model for JHSUVs.¹⁷ MUI is a strong assumption that is usually inappropriate for HSUVs.¹⁸

Model (iii):⁴ “A combined approach for adjusting the initial health-state utilities”

Model (iii) posits that JHSs have a level of utility independence controlled by a parameter $0.0 \leq γ \leq 0.1$ . Thus, the JHSUVs lie between the MUI HSUV and the FBU (2b) and they do not account for HCs that are PSs². For instance, given $U (a_{i}) = 0.62$ and $U (b_{j}) = 0.73,$ Model (iii) predicts $0.45 \leq U^{(iii)} (a_{i} & b_{j}) \leq 0.62 .$ This is wrong: HCs can be PSs, in which case $0.35 \leq U^{PS} (a_{i} & b_{j}) < 0.45 .$

T-Y⁴ recommend using Models (ii) and (iii) for JHSUVs that “would easily pass the previous monotonicity test but could still be considered as not realistic.” This can mislead clinicians to recommend and patients to choose unwanted treatments. Case in point, a patient who wants to avoid treatments with HSUVs $\leq 0.45$ chooses treatment T_X based on $U^{(ii)} (a_{i} & b_{j})$ and $U^{(iii)} (a_{i} & b_{j}) \geq 0.45 .$

Results and discussion

The clinical vignette in Box 1 is analyzed assuming HSUVs that are elicited with and without intervention.

HSUVs elicited with intervention

Interviewers intervene when necessary to ensure that patient HP assesses consistent HSUVs which truly represent their preferences. Elicited single HSUVs (SHSUVs) are not always more correct than elicited JHSUVs.¹ HP adjusts the JHSUV and SHSUVs as shown in Box 1: $U (a_{i}) = 0.55,$ $U (b_{j}) = 0.62$ , $U (a_{i} & b_{j}) = 0.38 .$ These HSUVs satisfy the FUB (= 0.55) and FLB (= 0.17).

HSUVs elicited without intervention

Interviewers do not intervene during the elicitation of HSUVs. The no-intervention elicited HSUVs shown in Table 1 violate the FLB (= 0.35). As discussed above, the T-Y models do not identify these HSUVs as inconsistent. T-Y recommend using Models (ii) and (iii) for JHSUVs that “could still be considered as not realistic.”⁴ These models predict the significantly different JHSUVs shown in Table 1.

Decision analysis

For illustration, we consider the clinical vignette and data shown in Box 1 and Table 2, respectively. Patient HP has a complicated decision to make: “to be or not to be” treated with T_X? The expected number of QALYs for each alternative and set of HSUVs is calculated using (4). Table 2 summarizes the results and recommendations. The HSUVs elicited with and without intervention provide contradictory recommendations:

– HSUVs elicited with intervention. The prediction is: 15.0 YLs, 7.8 QALYs. The recommendation is “Yes T_x”.
– HSUVs elicited without intervention and adjusted after the fact. Model (i) predicts 15 YLs and 2.25 QALYs. Models (ii) predicts 15 YLs and 6.75 QALYs. Based on the number of QALYs, Models (i,) and(ii) recommend “No T_x”. Model (iii) recommends either “No T_x” or “Yes T_x” depending on the control parameter $γ .$

Table 2. Impact of HSUV elicitation and T-Y models on treatment recommendation.

Options	YLs	Elicitation	P (a_i& b_j)	U (a_i & b_j)	QALYs	RECs	RT
No T_x	7.00		1.00	1.00	7.00
Yes T_x	15.00
		Intervention	1.00	0.52	7.80	Yes T_x	P
		No intervention
		Model (i)	1.00	0.15	2.25	No T_x	F
		Model (ii)	1.00	0.45	6.75	No T_x	F
		Model (iii)	1.00	$\in$ (0.45, 0.62]	$\in$ [6.75, 9.30]	?	F

Kujawski et al.¹⁹ proposed an intuitive reasonableness test that decision models used for SMD should pass to qualify as SDM tools: “Can a treatment that results in premature death trump a treatment that causes acceptable adverse effects?” A “Yes” answer may mislead clinicians into recommending and patients into choosing decisions with unintended consequences. As shown in Table 2, the three T-Y models fail this test.

Conclusions

The elicitation of reliable HSUVs is critical to ensure medical decisions that patients truly prefer. As shown in this paper, the three T-Y models⁴^,⁵ do not accurately account for individual preferences and the mathematical equivalence of HSUVs with probabilities elicited using the SG. Given consistent elicited HSUVs, it is not the function of clinicians to judge whether these are realistic or unrealistic.

A clinical vignette proves that the three T-Y models may recommend treatments that result in premature death over treatments that cause acceptable adverse effects. This is a sure sign that these models are faulty and can be misleading. Well-trained interviewers are still essential to elicit reliable HSUVs. Practical tools are being developed to assist with the assessment of HSUVs, e.g., Gambler II.²⁰ The uncertainties of elicited HSUVs and calculated QALYs need to be addressed for sound SDM. Assuming point estimates causes false confidence in the analysis results.²¹^,²²

Data availability

Underlying data

Figshare: Elicited and adjusted HSUVs using T-Y models. https://doi.org/10.6084/m9.figshare.21651911.¹⁴

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

References

1. Dale W, Biller P, Basu A, et al.: The prevalence, correlates, and impact of logically inconsistent preferences in utility assessments for joint health states in prostate cancer. Med. Care. 2011; 49(1): 59–66. PubMed Abstract | Publisher Full Text
2. Kujawski E: The importance of preference interactions in joint health-state utility values applied to decision analyses for shared decision-making. IISE Trans. Healthc. Syst. Eng. 2022; 1–14. Publisher Full Text
3. Stewart ST, Lenert L, Bhatnagar V, et al.: Utilities for prostate cancer health states in men aged 60 and older. Med. Care. 2005; 43(4): 347–355. Publisher Full Text
4. Triantaphyllou E, Yanase J: How to identify and treat data inconsistencies when eliciting health-state utility values for patient-centered decision making. Artif. Intell. Med. 2020; 106: 101882. Publisher Full Text
5. Triantaphyllou E, Yanase J: Treatment selection for life-critical shared decision making under ranges of health-state utility scenarios. J. Biomed. Inform. 2021; 115: 103604. Publisher Full Text
6. Lafata EJ, Brown RF, Pignone MP, et al.: Primary care physicians’ support of shared decision making for different cancer screening decisions. Med. Decis. Mak. 2017; 37(1): 70–78. PubMed Abstract | Publisher Full Text | Free Full Text
7. Guyatt GH, Feeny DH, Patrick DL: Measuring health-related quality of life. Ann. Intern. Med. 1993; 118(8): 622–629. Publisher Full Text
8. Elkin EB, Cowen ME, Cahill D, et al.: Preference assessment method affects decision-analytic recommendations: a prostate cancer treatment example. Med. Decis. Mak. 2004; 24(5): 504–510. PubMed Abstract | Publisher Full Text
9. von Neumann J , Morgenstern O: Theory of games and economic behavior. Princeton University Press;1947.
10. Hunink MGM, Weinstein MC, Wittenberg E, et al.: Decision making in health and medicine: integrating evidence and values. Cambridge University Press; 2^nd edition2014.
11. Abbas AE: Foundations of multiattribute utility. Cambridge University Press;2018.
12. Fréchet M: Généralisations du théorème des probabilités totales. Fundam. Math. 1935; 25(1): 379–387. Publisher Full Text
13. Osherson D, Lane D, Hartley P, et al.: Coherent probability from incoherent judgment. J. Exp. Psychol. Appl. 2001; 7(1): 3–12. PubMed Abstract | Publisher Full Text
14. Kujawski E: Elicited and adjusted HSUVs using T-Y models. figshare Dataset.2022. Publisher Full Text
15. Keeney RL, Raiffa H: Decisions with multiple objectives: preferences and value tradeoffs. John Wiley & Sons;1976.
16. Howard RA, Abbas AE: Foundations of decision analysis. Pearson;2015.
17. Fu AZ, Kattan MW: Utilities should not be multiplied: evidence from the preference-based scores in the United States. Med. Care. 2008; 46(9): 984–990. Publisher Full Text
18. Kujawski E: On the appropriateness/inappropriateness of the Keeney-Raiffa multiplicative utility function for medical decision making. PPI SyEN. 2019; 79: 5–17.Reference Source
19. Kujawski E, Triantaphyllou E, Yanase J: Additive multicriteria decision analysis models: misleading aids for life-critical shared decision making. Med. Decis. Mak. 2019; 39(4): 437–449. PubMed Abstract | Publisher Full Text
20. Behney AC: Ignoring uncertainty in predictor variables leads to false confidence in results: a case study of duck habitat use. Ecosphere. 2020; 11(10): 1–13.
21. Adejare AA Jr, Eckman MH: Automated tool for health utility assessments: the Gambler II. MDM Policy & Practice. 2020; 5(1): 1–12.
22. Spiegelhalter DJ, Franklin RC, Bull K: Assessment, criticism and improvement of imprecise subjective probabilities for a medical expert system. Proceedings of the fifth workshop on uncertainty in artificial intelligence. 1989; 335–342.

Comments on this article Comments (0)

Version 1

VERSION 1 PUBLISHED 19 Dec 2022

Author details Author details

Independent researcher, Berkeley, CA, 94708, USA

Edouard Kujawski
Roles: Conceptualization, Formal Analysis, Investigation, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

The author(s) declared that no grants were involved in supporting this work.

Article Versions (1)

version 1

Published: 19 Dec 2022, 11:1533

https://doi.org/10.12688/f1000research.128862.1

Copyright

© 2022 Kujawski E. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

0

SEE MORE DETAILS

CITE

how to cite this article

Kujawski E. How adjusting elicited health utilities after the fact can adversely affect shared decision making [version 1; peer review: 1 approved with reservations, 1 not approved]. F1000Research 2022, 11:1533 (https://doi.org/10.12688/f1000research.128862.1)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 1

VERSION 1

PUBLISHED 19 Dec 2022

Views

5

Reviewer Report 30 Oct 2024

Pierfelice Cutrufelli, University of Catania, Catania, Italy

Approved with Reservations

https://doi.org/10.5256/f1000research.141495.r235544

As a clinician, I will focus my peer review on the clinical aspects of this paper while acknowledging that I am not qualified to evaluate its mathematical components.
I must first clarify that I am not qualified to assess ... Continue reading

As a clinician, I will focus my peer review on the clinical aspects of this paper while acknowledging that I am not qualified to evaluate its mathematical components.
I must first clarify that I am not qualified to assess the mathematical models, Fréchet bounds, or utility calculations presented in this paper. These aspects should be evaluated by experts in mathematical modeling and decision theory.

Clinical Relevance: The paper addresses an important clinical concern: how we measure and use patient preferences in shared decision-making (SDM). Recent work by Rodolico et al. (2023) examining SDM in psychiatric settings found that trust and decision-making involvement alone may not enhance patient empowerment, suggesting that preference elicitation is more complex than previously thought. This complexity makes the current paper's examination of health utility values particularly relevant to clinical practice, especially when discussing treatments with multiple potential side effects.

Clinical Vignette Assessment: The clinical vignette (Box 1) effectively illustrates a common scenario where:

A patient has an asymptomatic condition requiring treatment
The treatment has guaranteed side effects
We need to help patients weigh longevity against quality of life

While the vignette focuses on physical health outcomes, Rodolico's findings suggest we should also consider how patient empowerment and trust levels influence preference expression and stability.

As a clinician reviewing this paper, I strongly recommend a substantial expansion of the clinical components, particularly regarding Shared Decision Making (SDM). The current mathematical focus, while valuable, needs to be balanced with robust clinical context and practical guidance. Here are my detailed suggestions for enhancement. This is a comprehensive outline, and the author should decide what to focus on and what to write about. Not all parts need to be addressed:

1. Comprehensive SDM Framework
The paper would benefit from a dedicated section explaining:

a) Core SDM Components:

Definition and evolution of SDM in clinical practice
Current best practices in SDM implementation
Barriers and facilitators to effective SDM
Role of patient empowerment, as highlighted by Rodolico et al. (2023)
Integration of trust dynamics in the SDM process

b) Clinical Context:

Real-world applications across different medical specialties
Impact of time constraints on SDM implementation
Variation in SDM approaches based on clinical settings (acute vs. chronic care)
Influence of symptom severity on decision-making capacity

2. Expanded Clinical Considerations
The paper should address:
a) Patient Factors:

Health literacy levels
Cultural and linguistic considerations
Role of family and caregivers
Impact of cognitive status
Emotional state and its influence on preference expression
Prior healthcare experiences

b) Provider Factors:

Clinical expertise integration
Time management strategies
Communication skills requirements
Documentation practices
Risk communication approaches

Practical Implementation Guidelines

Add sections on:
a) Clinical Workflow Integration:

Step-by-step guidance for preference elicitation
Time-efficient approaches
Documentation templates
Quality assurance measures
Follow-up protocols

b) Training Requirements:

Clinician preparation needs
Staff role definition
Ongoing education requirements
Competency assessment methods

3. Clinical Vignette Expansion
Current vignette should be expanded to include:
a) Multiple Scenarios:

Varying complexity levels
Different clinical specialties
Range of patient characteristics
Various outcome possibilities

b) Detailed Process Description:

Initial patient engagement
Information sharing methods
Preference elicitation techniques
Decision documentation
Follow-up planning

4. Quality Metrics and Outcomes
Add discussion of:
a) Clinical Outcomes:

Patient satisfaction measures
Treatment adherence rates
Health outcomes correlation
Quality of life impacts
Long-term follow-up results

b) Process Measures:

Time efficiency metrics
Resource utilization
Cost implications
Staff satisfaction

System integration success

5. Special Considerations
Include sections on:
a) Vulnerable Populations:

Elderly patients
Mental health conditions
Language barriers
Cultural differences
Limited health literacy

b) Complex Clinical Situations:

Emergency decision-making
Chronic disease management
End-of-life care
Multiple comorbidities
Conflicting treatment goals

6. Integration with Existing Clinical Systems Address:
a) Electronic Health Record Integration:

Documentation requirements
Decision support tools
Workflow optimization
Data collection methods
Quality monitoring

b) Team-Based Care:

Role definition
Communication protocols
Handoff procedures
Collaborative decision-making
Cross-disciplinary coordination

7. Research Implications
Expand on:
a) Clinical Research Needs:

Outcome studies design
Implementation research
Quality improvement metrics
Patient-reported outcomes
Long-term impact assessment

b) Knowledge Gaps:

Current limitations
Future research directions
Methodology improvements
Validation requirements
Translation to practice

These expansions would significantly strengthen the paper's clinical utility and provide a more balanced perspective between mathematical rigor and practical application. The enhanced clinical content would make the paper more accessible and valuable to healthcare providers while maintaining its theoretical contributions.

Clinical Concerns: The paper's key clinical message - that post-hoc adjustments to patient preferences could lead to recommendations for premature death over manageable side effects - is deeply concerning from a medical ethics perspective. This concern is amplified by Rodolico's finding that higher patient empowerment correlates with better clinical outcomes, suggesting that undermining patient preferences through post-hoc adjustments might adversely affect treatment outcomes.

Practical Clinical Implications: As a clinician, I have reservations about:

The practicality of conducting detailed preference assessments in time-constrained clinical settings
How to balance trust and active patient involvement, given Rodolico's finding that trust may paradoxically decrease patient engagement
The impact on clinical workflow and patient comprehension
How symptom severity might affect preference stability and consistency

Recommendation: While I support publication from a clinical perspective due to its relevance to medical decision-making, I strongly recommend: a) Additional peer review by mathematical experts for the theoretical framework b) Clear separation of clinical and mathematical components in the paper c) More practical guidance for implementing preference elicitation in clinical settings d) Integration of recent findings on patient empowerment and trust dynamics in SDM e) Discussion of how symptom severity might influence preference stability

Limitations of My Review: I explicitly acknowledge that I cannot evaluate:

The mathematical validity of the models presented
The statistical methodology
The theoretical framework of utility values; these aspects require separate review by appropriate experts.

Conclusion: While the clinical implications of this work are important and well-supported by recent research on patient empowerment and SDM, I can only partially approve this paper, specifically regarding its clinical relevance and practical implications. The mathematical components require separate expert review and validation. The integration of recent findings on patient empowerment and trust dynamics in SDM strengthens the paper's clinical relevance while highlighting the complexity of patient preference elicitation in clinical settings.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

I cannot comment. A qualified statistician is required.
Are all the source data underlying the results available to ensure full reproducibility?

No source data required
Are the conclusions drawn adequately supported by the results?

Partly

References

1. Rodolico A, Cutrufelli P, Maccarone G, Avincola G, et al.: Exploring Patient Empowerment in Major Depressive Disorder: Correlations of Trust, Active Role in Shared Decision-Making, and Symptomatology in a Sample of Italian Patients.J Clin Med. 2024; 13 (20). PubMed Abstract | Publisher Full Text

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: As a practicing psychiatrist, I limit this peer review to the clinical aspects of this manuscript, particularly those concerning shared decision-making and patient preference assessment. My expertise in mental health care, where patient preferences and treatment choices are uniquely complex, allows me to evaluate the practical implications of preference elicitation methods in clinical settings.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Respond or Comment

Views

11

Reviewer Report 14 Feb 2024

Michal Lewandowski, Warsaw School of Economics, Warsaw, Poland

Not Approved

https://doi.org/10.5256/f1000research.141495.r235536

Report on the article How adjusting elicited health utilities after the fact can adversely affect shared decision making, written by Eduard Kujawski and submitted to F1000.

The article discusses the consistency of utility values of health states. ... Continue reading

Report on the article How adjusting elicited health utilities after the fact can adversely affect shared decision making, written by Eduard Kujawski and submitted to F1000.

The article discusses the consistency of utility values of health states. In particular, it is noted that the utility values obtained using a standard gamble are the probability equivalent of a gamble in which one can win perfect health or lose by immediately dying. This is the probability at which the decision maker is indifferent whether he will be in a given state of health for sure or whether he will take a risk with this probability of achieving perfect health; otherwise, instant death occurs. Therefore, the (dis-) utility of two individual states a, b (e.g. two side effects) occurring simultaneously must satisfy Frechet's probability inequality: it must be greater than max(0,pa+pb-1) $max(0,p(a)+p(b)-1)$ and smaller than min(pa,pb) $min(p(a), p(b))$ .
The point is, of course, right, but also quite obvious. To make a meaningful contribution, I would expect the paper to clearly show which existing models violate these consistency requirements. Of course, these do not include models that satisfy the mutual utility independence: since they assume independence, they clearly fall within Frechet's limits. However, the author states that these models are not realistic. The author discusses models (i) – (iii). However, they are not even properly defined, only a numerical example is presented. I think a publishable article should clearly define these models and discuss why they are important/where they are used etc.
From the numerical example presented in the article, it can be concluded that model (i) does not satisfy Frechet's inequality (joint disutility/probability is less than Frechet's lower bound), model (ii) satisfies independence (but is called unrealistic), and model (iii) assumes that a and b are preference complements. Well, so what? The author states that none of the models is satisfactory because it may happen that a and b are preference substitutes and their joint probability falls somewhere between Frechet's lower bound and the product of two individual probabilities. The point, however, is that without additional knowledge we cannot say what it really is: whether a, b are preference substitutes, their complements, or whether they are independent. Frechet's inequalities must be explicitly satisfied if utility values are constructed as probabilities, this is clear.
There are many things that can be criticized in this article - for example, the definition given by equation (4). However, since I believe that, for the reasons I have described above, the work is not suitable for publication, even if these defects are removed, I will not mention them and recommend rejection of the article.

Is the work clearly and accurately presented and does it cite the current literature?

No
Is the study design appropriate and is the work technically sound?

No
Are sufficient details of methods and analysis provided to allow replication by others?

No
If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable
Are all the source data underlying the results available to ensure full reproducibility?

No source data required
Are the conclusions drawn adequately supported by the results?

No

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Preference modelling, decision theory under risk, uncertainty, time delay, imprecision, fuzzy preferences.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

CITE

Report a concern

Respond or Comment

Comments on this article Comments (0)

Version 1

VERSION 1 PUBLISHED 19 Dec 2022

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 1 19 Dec 22	read	read

Michal Lewandowski, Warsaw School of Economics, Warsaw, Poland
Pierfelice Cutrufelli, University of Catania, Catania, Italy

Comments on this article

All Comments(0)

Add a comment

Sign up for content alerts

Browse by related subjects

Back to all reports

Reviewer Report

5 Views

30 Oct 2024 | for Version 1

Pierfelice Cutrufelli, University of Catania, Catania, Italy

5 Views Cite this report Responses(0)

Approved With Reservations

As a clinician, I will focus my peer review on the clinical aspects of this paper while acknowledging that I am not qualified to evaluate its mathematical components.
I must first clarify that I am not qualified to assess the mathematical models, Fréchet bounds, or utility calculations presented in this paper. These aspects should be evaluated by experts in mathematical modeling and decision theory.

Clinical Relevance: The paper addresses an important clinical concern: how we measure and use patient preferences in shared decision-making (SDM). Recent work by Rodolico et al. (2023) examining SDM in psychiatric settings found that trust and decision-making involvement alone may not enhance patient empowerment, suggesting that preference elicitation is more complex than previously thought. This complexity makes the current paper's examination of health utility values particularly relevant to clinical practice, especially when discussing treatments with multiple potential side effects.

Clinical Vignette Assessment: The clinical vignette (Box 1) effectively illustrates a common scenario where:

A patient has an asymptomatic condition requiring treatment
The treatment has guaranteed side effects
We need to help patients weigh longevity against quality of life

While the vignette focuses on physical health outcomes, Rodolico's findings suggest we should also consider how patient empowerment and trust levels influence preference expression and stability.

As a clinician reviewing this paper, I strongly recommend a substantial expansion of the clinical components, particularly regarding Shared Decision Making (SDM). The current mathematical focus, while valuable, needs to be balanced with robust clinical context and practical guidance. Here are my detailed suggestions for enhancement. This is a comprehensive outline, and the author should decide what to focus on and what to write about. Not all parts need to be addressed:

1. Comprehensive SDM Framework
The paper would benefit from a dedicated section explaining:

a) Core SDM Components:

Definition and evolution of SDM in clinical practice
Current best practices in SDM implementation
Barriers and facilitators to effective SDM
Role of patient empowerment, as highlighted by Rodolico et al. (2023)
Integration of trust dynamics in the SDM process

b) Clinical Context:

Real-world applications across different medical specialties
Impact of time constraints on SDM implementation
Variation in SDM approaches based on clinical settings (acute vs. chronic care)
Influence of symptom severity on decision-making capacity

2. Expanded Clinical Considerations
The paper should address:
a) Patient Factors:

Health literacy levels
Cultural and linguistic considerations
Role of family and caregivers
Impact of cognitive status
Emotional state and its influence on preference expression
Prior healthcare experiences

b) Provider Factors:

Clinical expertise integration
Time management strategies
Communication skills requirements
Documentation practices
Risk communication approaches

Practical Implementation Guidelines

Add sections on:
a) Clinical Workflow Integration:

Step-by-step guidance for preference elicitation
Time-efficient approaches
Documentation templates
Quality assurance measures
Follow-up protocols

b) Training Requirements:

Clinician preparation needs
Staff role definition
Ongoing education requirements
Competency assessment methods

3. Clinical Vignette Expansion
Current vignette should be expanded to include:
a) Multiple Scenarios:

Varying complexity levels
Different clinical specialties
Range of patient characteristics
Various outcome possibilities

b) Detailed Process Description:

Initial patient engagement
Information sharing methods
Preference elicitation techniques
Decision documentation
Follow-up planning

4. Quality Metrics and Outcomes
Add discussion of:
a) Clinical Outcomes:

Patient satisfaction measures
Treatment adherence rates
Health outcomes correlation
Quality of life impacts
Long-term follow-up results

b) Process Measures:

Time efficiency metrics
Resource utilization
Cost implications
Staff satisfaction

System integration success

5. Special Considerations
Include sections on:
a) Vulnerable Populations:

Elderly patients
Mental health conditions
Language barriers
Cultural differences
Limited health literacy

b) Complex Clinical Situations:

Emergency decision-making
Chronic disease management
End-of-life care
Multiple comorbidities
Conflicting treatment goals

6. Integration with Existing Clinical Systems Address:
a) Electronic Health Record Integration:

Documentation requirements
Decision support tools
Workflow optimization
Data collection methods
Quality monitoring

b) Team-Based Care:

Role definition
Communication protocols
Handoff procedures
Collaborative decision-making
Cross-disciplinary coordination

7. Research Implications
Expand on:
a) Clinical Research Needs:

Outcome studies design
Implementation research
Quality improvement metrics
Patient-reported outcomes
Long-term impact assessment

b) Knowledge Gaps:

Current limitations
Future research directions
Methodology improvements
Validation requirements
Translation to practice

These expansions would significantly strengthen the paper's clinical utility and provide a more balanced perspective between mathematical rigor and practical application. The enhanced clinical content would make the paper more accessible and valuable to healthcare providers while maintaining its theoretical contributions.

Clinical Concerns: The paper's key clinical message - that post-hoc adjustments to patient preferences could lead to recommendations for premature death over manageable side effects - is deeply concerning from a medical ethics perspective. This concern is amplified by Rodolico's finding that higher patient empowerment correlates with better clinical outcomes, suggesting that undermining patient preferences through post-hoc adjustments might adversely affect treatment outcomes.

Practical Clinical Implications: As a clinician, I have reservations about:

The practicality of conducting detailed preference assessments in time-constrained clinical settings
How to balance trust and active patient involvement, given Rodolico's finding that trust may paradoxically decrease patient engagement
The impact on clinical workflow and patient comprehension
How symptom severity might affect preference stability and consistency

Recommendation: While I support publication from a clinical perspective due to its relevance to medical decision-making, I strongly recommend: a) Additional peer review by mathematical experts for the theoretical framework b) Clear separation of clinical and mathematical components in the paper c) More practical guidance for implementing preference elicitation in clinical settings d) Integration of recent findings on patient empowerment and trust dynamics in SDM e) Discussion of how symptom severity might influence preference stability

Limitations of My Review: I explicitly acknowledge that I cannot evaluate:

The mathematical validity of the models presented
The statistical methodology
The theoretical framework of utility values; these aspects require separate review by appropriate experts.

Conclusion: While the clinical implications of this work are important and well-supported by recent research on patient empowerment and SDM, I can only partially approve this paper, specifically regarding its clinical relevance and practical implications. The mathematical components require separate expert review and validation. The integration of recent findings on patient empowerment and trust dynamics in SDM strengthens the paper's clinical relevance while highlighting the complexity of patient preference elicitation in clinical settings.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

I cannot comment. A qualified statistician is required.
Are all the source data underlying the results available to ensure full reproducibility?

No source data required
Are the conclusions drawn adequately supported by the results?

Partly

References

1. Rodolico A, Cutrufelli P, Maccarone G, Avincola G, et al.: Exploring Patient Empowerment in Major Depressive Disorder: Correlations of Trust, Active Role in Shared Decision-Making, and Symptomatology in a Sample of Italian Patients.J Clin Med. 2024; 13 (20). PubMed Abstract | Publisher Full Text

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

As a practicing psychiatrist, I limit this peer review to the clinical aspects of this manuscript, particularly those concerning shared decision-making and patient preference assessment. My expertise in mental health care, where patient preferences and treatment choices are uniquely complex, allows me to evaluate the practical implications of preference elicitation methods in clinical settings.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

11 Views

14 Feb 2024 | for Version 1

Michal Lewandowski, Warsaw School of Economics, Warsaw, Poland

11 Views Cite this report Responses(0)

Not Approved

Report on the article How adjusting elicited health utilities after the fact can adversely affect shared decision making, written by Eduard Kujawski and submitted to F1000.

The article discusses the consistency of utility values of health states. In particular, it is noted that the utility values obtained using a standard gamble are the probability equivalent of a gamble in which one can win perfect health or lose by immediately dying. This is the probability at which the decision maker is indifferent whether he will be in a given state of health for sure or whether he will take a risk with this probability of achieving perfect health; otherwise, instant death occurs. Therefore, the (dis-) utility of two individual states a, b (e.g. two side effects) occurring simultaneously must satisfy Frechet's probability inequality: it must be greater than max(0,pa+pb-1) $max(0,p(a)+p(b)-1)$ and smaller than min(pa,pb) $min(p(a), p(b))$ .
The point is, of course, right, but also quite obvious. To make a meaningful contribution, I would expect the paper to clearly show which existing models violate these consistency requirements. Of course, these do not include models that satisfy the mutual utility independence: since they assume independence, they clearly fall within Frechet's limits. However, the author states that these models are not realistic. The author discusses models (i) – (iii). However, they are not even properly defined, only a numerical example is presented. I think a publishable article should clearly define these models and discuss why they are important/where they are used etc.
From the numerical example presented in the article, it can be concluded that model (i) does not satisfy Frechet's inequality (joint disutility/probability is less than Frechet's lower bound), model (ii) satisfies independence (but is called unrealistic), and model (iii) assumes that a and b are preference complements. Well, so what? The author states that none of the models is satisfactory because it may happen that a and b are preference substitutes and their joint probability falls somewhere between Frechet's lower bound and the product of two individual probabilities. The point, however, is that without additional knowledge we cannot say what it really is: whether a, b are preference substitutes, their complements, or whether they are independent. Frechet's inequalities must be explicitly satisfied if utility values are constructed as probabilities, this is clear.
There are many things that can be criticized in this article - for example, the definition given by equation (4). However, since I believe that, for the reasons I have described above, the work is not suitable for publication, even if these defects are removed, I will not mention them and recommend rejection of the article.

Is the work clearly and accurately presented and does it cite the current literature?

No
Is the study design appropriate and is the work technically sound?

No
Are sufficient details of methods and analysis provided to allow replication by others?

No
If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable
Are all the source data underlying the results available to ensure full reproducibility?

No source data required
Are the conclusions drawn adequately supported by the results?

No

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Preference modelling, decision theory under risk, uncertainty, time delay, imprecision, fuzzy preferences.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

Respond to this report

Responses (0)

[1] 1. Dale W, Biller P, Basu A, et al.: The prevalence, correlates, and impact of logically inconsistent preferences in utility assessments for joint health states in prostate cancer. Med. Care. 2011; 49(1): 59–66. PubMed Abstract | Publisher Full Text

[2] 2. Kujawski E: The importance of preference interactions in joint health-state utility values applied to decision analyses for shared decision-making. IISE Trans. Healthc. Syst. Eng. 2022; 1–14. Publisher Full Text

[3] 3. Stewart ST, Lenert L, Bhatnagar V, et al.: Utilities for prostate cancer health states in men aged 60 and older. Med. Care. 2005; 43(4): 347–355. Publisher Full Text

[4] 4. Triantaphyllou E, Yanase J: How to identify and treat data inconsistencies when eliciting health-state utility values for patient-centered decision making. Artif. Intell. Med. 2020; 106: 101882. Publisher Full Text

[5] 5. Triantaphyllou E, Yanase J: Treatment selection for life-critical shared decision making under ranges of health-state utility scenarios. J. Biomed. Inform. 2021; 115: 103604. Publisher Full Text

[6] 6. Lafata EJ, Brown RF, Pignone MP, et al.: Primary care physicians’ support of shared decision making for different cancer screening decisions. Med. Decis. Mak. 2017; 37(1): 70–78. PubMed Abstract | Publisher Full Text | Free Full Text

[7] 7. Guyatt GH, Feeny DH, Patrick DL: Measuring health-related quality of life. Ann. Intern. Med. 1993; 118(8): 622–629. Publisher Full Text

[8] 8. Elkin EB, Cowen ME, Cahill D, et al.: Preference assessment method affects decision-analytic recommendations: a prostate cancer treatment example. Med. Decis. Mak. 2004; 24(5): 504–510. PubMed Abstract | Publisher Full Text

[9] 9. von Neumann J , Morgenstern O: Theory of games and economic behavior. Princeton University Press;1947.

[10] 10. Hunink MGM, Weinstein MC, Wittenberg E, et al.: Decision making in health and medicine: integrating evidence and values. Cambridge University Press; 2^nd edition2014.

[11] 11. Abbas AE: Foundations of multiattribute utility. Cambridge University Press;2018.

[12] 12. Fréchet M: Généralisations du théorème des probabilités totales. Fundam. Math. 1935; 25(1): 379–387. Publisher Full Text

[13] 13. Osherson D, Lane D, Hartley P, et al.: Coherent probability from incoherent judgment. J. Exp. Psychol. Appl. 2001; 7(1): 3–12. PubMed Abstract | Publisher Full Text

[14] 14. Kujawski E: Elicited and adjusted HSUVs using T-Y models. figshare Dataset.2022. Publisher Full Text

[15] 15. Keeney RL, Raiffa H: Decisions with multiple objectives: preferences and value tradeoffs. John Wiley & Sons;1976.

[16] 16. Howard RA, Abbas AE: Foundations of decision analysis. Pearson;2015.

[17] 17. Fu AZ, Kattan MW: Utilities should not be multiplied: evidence from the preference-based scores in the United States. Med. Care. 2008; 46(9): 984–990. Publisher Full Text

[18] 18. Kujawski E: On the appropriateness/inappropriateness of the Keeney-Raiffa multiplicative utility function for medical decision making. PPI SyEN. 2019; 79: 5–17.Reference Source

[19] 19. Kujawski E, Triantaphyllou E, Yanase J: Additive multicriteria decision analysis models: misleading aids for life-critical shared decision making. Med. Decis. Mak. 2019; 39(4): 437–449. PubMed Abstract | Publisher Full Text

[20] 20. Behney AC: Ignoring uncertainty in predictor variables leads to false confidence in results: a case study of duck habitat use. Ecosphere. 2020; 11(10): 1–13.

[21] 21. Adejare AA Jr, Eckman MH: Automated tool for health utility assessments: the Gambler II. MDM Policy & Practice. 2020; 5(1): 1–12.

[22] 22. Spiegelhalter DJ, Franklin RC, Bull K: Assessment, criticism and improvement of imprecise subjective probabilities for a medical expert system. Proceedings of the fifth workshop on uncertainty in artificial intelligence. 1989; 335–342.

How adjusting elicited health utilities after the fact can adversely affect shared decision making

Abstract

Keywords

Introduction

Box 1. Clinical vignette.

Methods

Theoretical framework

(1)

(2a)

(2b)

(2c)

(3)

(4)

Analysis of T-Y models

Table 1. Elicited and adjusted HSUVs using T-Y models.

Results and discussion

HSUVs elicited with intervention

HSUVs elicited without intervention

Decision analysis

Table 2. Impact of HSUV elicitation and T-Y models on treatment recommendation.

Conclusions

Data availability

Underlying data

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated