Keywords
palliative care, terminal care, end-of-life care, quality indicators, performance measurement, evaluation
-
Views
-
Downloads
How to cite this article
Budhwani S, Gill A, Scott M et al. Measuring indicators of health system performance for palliative and end-of-life care using health administrative data: a scoping review [version 1; peer review: 2 approved with reservations]. F1000Research 2020, 9:1438 (https://doi.org/10.12688/f1000research.27121.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
Export Citation
Sciwheel
EndNote
Ref. Manager
Bibtex
ProCite
Sente
▬
✚
Research Article
Measuring indicators of health system performance for palliative and end-of-life care using health administrative data: a scoping review
[version 1; peer review: 2 approved with reservations]
PUBLISHED 10 Dec 2020
OPEN PEER REVIEW
REVIEWER STATUS
Abstract
Corresponding author:
Suman Budhwani
Competing interests:
No competing interests were disclosed.
Grant information:
This research was supported by a research grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC) to the Health System Performance Research Network Grant (HSPRN #06034). This study was also supported by the Institute for Clinical Evaluative Sciences, which is funded by an annual grant from the Ontario MOHLTC. The views expressed in this paper are the views of the authors and do not necessarily reflect those of the funder.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Copyright:
© 2020 Budhwani S et al.
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
How to cite: Budhwani S, Gill A, Scott M et al. Measuring indicators of health system performance for palliative and end-of-life care using health administrative data: a scoping review [version 1; peer review: 2 approved with reservations]. F1000Research 2020, 9:1438 (https://doi.org/10.12688/f1000research.27121.1)
First published: 10 Dec 2020, 9:1438 (https://doi.org/10.12688/f1000research.27121.1)
Latest published: 10 Dec 2020, 9:1438 (https://doi.org/10.12688/f1000research.27121.1)
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Introduction
Growing health care costs in an environment of tight financial constraints and an aging population are challenging many health care systems globally. Recent health care reform initiatives have underscored the importance of providing quality care to patients at all phases of the disease trajectory, including palliative care (PC) both specialist and generalist, and end-of-life care (EOLC), to improve patient and health system outcomes1,2. The Worldwide Palliative Care Alliance and the World Health Organization recommend provision of PC for all persons with chronic and life limiting/threatening conditions early in the illness trajectory3. However, universal access to quality, integrated and timely PC for patients remains intermittent at best, with PC largely being provided close to the end-of-life (EOL)4. Moreover, concerns about high costs associated with health care at the EOL are prevalent in the literature; studies attribute these costs to intensive, aggressive, and sometimes unnecessary utilization of institutionalized care, such as inpatient hospital services in the last few months of life5.
Monitoring health system performance is one important component of evaluating achievement of quality care for improved patient and health system outcomes. This can be done through health system performance indicators, which summarize directional quantitative information on the quality of health, often evaluated through measurement of structure (inputs) or characteristics of the health care system, process (outputs) or services required to provide health care, and outcomes or measures of ultimate impact of the health care provided6. Health system performance indicators allow comparisons across jurisdictions, organizations, and/or administrative databases to track progress over time in efforts to improve health care quality7. Quantitative information for indicators can utilize health administrative data or data that is routinely collected in the process of delivering health care programs and services, often used due to its extensive reach (many databases are population-based), low cost, and low respondent burden8. Administrative data can also enable contextualization of health system performance indicators due to availability of data on patient sociodemographic and clinical characteristics.
In recent years, jurisdictions in Canada, the United States, and the United Kingdom have focused on defining standards and monitoring health care quality as efforts to expand PC programs intensify9–14, and as a result a plethora of health system performance indicators for PC and EOLC using health administrative data have been proposed13,15–18. These indicators have been suggested based on differing objectives (e.g. quality improvement, performance measurement) and/or for different patient populations (e.g. cancer, intensive care units, long-term care (nursing home) patients)13,19–24. Given the diversity of indicators in the literature, a scoping and cataloguing exercise becomes important in order to consolidate and summarize common measurement concepts (themes) and indicators that can be recommended for use in the performance measurement of PC and EOLC. Such a catalogue can serve as a reference guide for use by policy and decision makers, thereby reducing potential redundant efforts on collecting indicators, and enabling movement to the next step of decision making on what to measure and how best to measure it given each jurisdiction’s individual data systems. Furthermore, categorizing indicators by health care sector or setting of health care delivery can be a helpful demarcation that enables measurement of health system performance to align with funding and accountability structures25. Consideration of health care setting or health sector (e.g., acute care hospitals, long-term care (LTC), home care) becomes important given each sector’s potentially differing care quality aims, target populations, health care processes, and definitions of outcomes. Such organization does not preclude inclusion of indicators of transitional care or coordination of care between health care settings, but rather enables tailoring of indicator definitions that are more reflective of the relevant sector or health setting’s contribution to system-level performance.
As such, the primary objective of this study was to create a catalogue or inventory of health system performance measurement concepts and indicators for PC and EOLC utilizing routinely-collected population-level health administrative databases and as categorized by health sector by conducting a scoping literature review.
Methods
This scoping literature review of health system performance indicators for PC and EOLC followed the Arksey and O’Malley (2005) methodological framework for scoping reviews26. Scoping reviews are a “type of knowledge synthesis, (that) follow a systematic approach to map evidence on a topic and identify main concepts, theories, sources and knowledge gaps”27. The primary objective of this scoping review was to collect and map health system performance indicators for PC and EOLC using routinely collected population-based health administrative data and to organize indicators by health care sector. Since this required consulting a broad array of literature to uniformly collect indicators, rather than appraisal of published indicators, this objective aligned well with the rationale for the use of scoping review methodology in comparison to systematic review methodology, and hence a scoping review methodology was chosen to complete research objectives28. A scoping review protocol was not published a priori for this study. Subsequent sections describe specific steps taken for this scoping review in accordance with the Arksey and O’Malley (2005) methodological framework.
Identifying relevant studies for inclusion
We collected both peer-reviewed and grey literature published in English for this scoping review. For peer-reviewed literature, we conducted a search of MEDLINE and EMBASE databases between years of 2010 to July 2018. A University of Toronto librarian and comparable literature reviews were consulted in the development of the search strategy, which included a combination of the United States National Library of Medicine Medical Subject Headings (MeSH)29 and keywords (see Extended data: S1 Appendix Search Strategy30). For grey literature, we utilized Google search engine using a combination of keywords such as “palliative care”, “end-of-life-care”, “performance indicators”, and/or “quality indicators”, as well as incorporated reference documents made available to us through our knowledge user partners based on their previously conducted literature searches. No jurisdictional restrictions were placed for the grey literature search; however Canadian references were generally better known by study authors and knowledge users, and formed the bulk of included references.
Inclusion/exclusion criteria for study selection
Inclusion and exclusion criteria were developed in accordance with the study objectives and points of inquiry for the review, notably: a) focused on core concepts of quality or performance measurement in palliative and/or end-of-life care, b) measurement conducted using population-level, routinely-collected data, and c) studies with a health system focus. The full set of generated articles was reviewed based on developed inclusion and exclusion criteria (see Extended data: S2 Table Study Inclusion and Exclusion Criteria30).
Reviewers (SB and AG) independently screened all articles using titles and abstracts, with any conflicts resolved through discussion and inclusion and exclusion criteria accordingly updated. This generated a total of 285 articles for full text review (Figure 1). Full text review was also independently conducted by SB and AG, with a total of 54 peer-reviewed studies and 42 grey literature documents being identified for indicator extraction. Additionally, forward reference searching of reference lists was conducted to include any additional relevant articles, as well as any other relevant studies known to the all study authors. Studies outside of the study time period of 2010 to July 2018 were included. This resulted in a total of 32 additional articles included into the study for data extraction.
Figure 1. Article and indicator extraction process.
Charting and summarizing the data
A data extraction tool was developed to effectively chart details on presented and potential indicators (such as indicator definition, numerator and denominator if available). Indicators were extracted if they a) measured health system performance for PC and EOLC, and b) used population-level data using health administrative datasets. Once the full set of indicators were charted, SB, AG and PT removed duplicates and conducted health sector classification. We chose to focus on categorizing our collected indicators by health sectors specifically for the Canadian province of Ontario, with a population of over 14.5 million residents31 with universal health coverage for costs associated with acute care, hospitalizations, physician visits, emergency room visits, long-term care, home care, complex continuing care, and medications for those meeting select age-based and need-based criteria32. Health administrative data is also collected comprehensively and at the population-level for the majority of health sectors with public health coverage, thereby increasing potential for performance measurement with indicators based on routinely-collected administrative data. For the purposes of our study, indicators were classified based on the following Ontario health sectors based on how data from health administrative databases is currently obtained and organized in Ontario administrative databases: hospital care (including emergency department (ED)); home care or care services provided in the home and community;33. LTC (i.e., nursing homes), hospice care, physician services or care services captured through physician billing codes; medications covered through the public health insurance system;34 complex continuing care (CCC) or technology-based care provided to patients with chronic and complex health conditions;35 cancer care or care specifically targeted for cancer prevention and treatment; and other. A category entitled “Cross Sectors” was created to capture indicators transcending more than one health sector (e.g. place of death at various care locations). Further thematic analysis was conducted collectively by the study authors in discussion to group indicators measuring similar constructs by common themes, leading to the creation of measurement themes for indicator classification under each health sector category.
Consultation exercise
Following this initial categorization, a working group with subject matter experts (both researchers and clinicians) was organized to review the final list of indicators. Indicators were cross-checked to ensure a) all relevant concepts related to PC and EOLC had been captured, and b) all indicators were conceivably measurable using (Ontario/Canadian) health administrative data based on the knowledge and expertise of the subject matter experts present.
Results
Studies reviewed
A total of 1111 indicators were extracted from 128 articles. 722 indicators were excluded (due to reasons of duplication and irrelevancy), resulting in a total of 339 indicators included for summarization (Figure 1). Studies and grey literature ranged from publication dates of 1992 to 2018, with only 5 included studies published prior to 2003, and the majority (n=82) being published after 2008 (Table 1).
Table 1. Included articles by year of publication.
| Year of publication | Number of publications |
|---|---|
| No date | 3 |
| <2003 | 5 |
| 2003 | 4 |
| 2004 | 1 |
| 2005 | 5 |
| 2006 | 4 |
| 2007 | 6 |
| 2008 | 14 |
| 2009 | 7 |
| 2010 | 10 |
| 2011 | 11 |
| 2012 | 10 |
| 2013 | 17 |
| 2014 | 15 |
| 2015 | 7 |
| 2016 | 3 |
| 2017 | 4 |
| 2018 | 2 |
| Total | 128 |
The 42 grey literature documents included publications from Canadian provinces (e.g., Ontario, Alberta, Saskatchewan) (n=31), where routinely collected administrative data is readily available, and from the United States (n=6), United Kingdom (n=4), and Australia (n=1). They included documents generated by PC delivery organizations, such as local home care agencies, and also jurisdictional collaborative efforts to improve care.
Categorization of indicators
Ten health care sectors were utilized to categorize indicators, with sector categories of Cross Sector, Cancer and Home Care having the greatest number of indicators (Table 2). Table 3 presents a summary of indicators and key measurement themes by health care sector, and a collective list of all indicators collected can be found in Extended data: S1 Table Detailed List and Information on Collected Indicators30. These findings are discussed next.
Table 2. Extracted indicators by health care sector.
| Health care Sector | Number of Indicators |
|---|---|
| Cross Sector | 58 |
| Cancer Care | 57 |
| Home Care | 48 |
| Long-Term Care | 40 |
| Hospital | 39 |
| Hospice | 36 |
| Physician Care | 29 |
| Medications | 26 |
| Complex Continuing Care | 3 |
| Other | 3 |
| Total | 339 |
Table 3. Summary of indicators by health care sector.
| Indicator theme | Description | References |
|---|---|---|
| Cross Sectors (n=58) | ||
| Place of death and Place of care | Possible locations of death include home, hospital, LTC, unknown. Time spent within a specific location of care, and whether the patient received the appropriate care when they died in a specific care setting. | 9,13–18,20,22–24,36–85,107,108,120–122 |
| Palliative care service utilization, and receipt of services | Length of stay and utilization of a palliative care program, or enrollment in a program within a specific time period, such as 3 or 6 months before death. Total number of individuals formally identified as palliative, and palliative upon pre- admission to hospital. Identify proportion of population who receive PC outside of acute care. | 13–18,21,38,39,43,51,61,62,65,72,76,81,83,84,86–100 |
| Costs | Costs measured by health sector utilization, and percentage of total budget spent on EOLC in last year of life. Costs were also compared between patients who did or did not receive PC, and per episode of care. | 9,43,52,67,93,99,101–105 |
| Aggressiveness of care and burdensome transitions | Composite indicator consisting of: death in acute care hospital, chemotherapy use in the last 14 days, greater than 1 ED visit in the last 6 months and 30 days of life, greater than 1 hospital admission in the last 6 months and 30 days, any Intensive Care Unit (ICU) admission in last 6 months and 30 days of life. Also measured as a rate of aggressive care in the last 3, 30 and 90 days per decedent in potential sectors of: acute care admission, ED visit, Complex Continuing Care (CCC)/rehab, LTC, ICU, or hospice. | 45,48–50,70,71,81,106 |
| Cancer Care (n=57) | ||
| Outpatient visits to a cancer centre | Number of visits (chemotherapy, physician appointments), and patients seen at a PC clinic, referrals to PC clinic with Palliative Performance Scale (PPS) rating of 60 and PPS 10 – 30, and median time between diagnosis and referral to a PC clinic. | 39,68,103,109,123–125 |
| Symptom and Functional Status Assessment | Assessment of complex advanced symptoms including screening rates, whether the symptoms were addressed with treatment, and referral to additional services. Screening rates for cancer patients were also compared across functional status scores, and time until death. | 13,21,41,44,53,61,67,76,78,84,86,90,103,107,120, 126–129 |
| Treatment | The delivery of chemotherapy and radiation within a specific time period before death (i.e., last 6 months, 5 months, 2 months, 30 days, 2 weeks or 14 days). Also, treatment was compared across functional status, to reflect aggressive treatment at the EOL, and the use of palliative radiation and palliative chemotherapy. | 13,17,20,21,37,39,48,50,51,54,56,60,63,64,67, 70,72–74,77–80,91,94,98,103,106–119 |
| Home care (n=48) | ||
| Utilization and length of stay | Utilization measured as the number of visits or hours of home care specified for different time periods (i.e., last 6 months of life), involvement of different health professions [i.e., nursing, personal support worker (PSW), physician, social work, respite care, medication delivery] and disease groups (i.e., cancer, congestive heart failure). Variation in home care support captured by measuring the change in hours of support, and mean hours of home care leading up to death. | 13,16–18,39,42,43,52,58,65,67,68,70,72,80,87,92,99, 103,110,123,128,130–136 |
| Referral and initiation of homecare | Wait times between referral to home care and initiation of services; identifying patients who are approaching EOL and flagged for home palliative care; and how early homecare began in the patient’s care trajectory. Place of death and care, and hospital readmission rate compared to home care received at the EOL. | 14,16,17,39,41,52,57,67,68,70,71,92,103,105,126,132, 134 |
| Patient outcomes and quality of care | Multidisciplinary care as reflected by the range of providers making home visits, and patient outcomes that reflect good clinical practice, such as pain and pressure ulcers. Characteristics of home care were also evaluated, including whether there was 24/7 home PC coverage, prognosis and goals of care documentation, and proportion of patients discharged home from hospital with and without home care. | 18,39,51,61,67,86,91,92,105,132,137,138 |
| Costs | Cost of home PC visits | 99,139 |
| Long-Term Care (LTC) (n=40) | ||
| Referral and length of stay | Referral to LTC, length of stay, and comparison of cancer and non-cancer patients who were in LTC. | 65,67,87,123,139,140 |
| Burdensome transitions at EOL | Number or percent of LTC clients with at least one burdensome transition within a specific time period. Burdensome transitions defined as 2 or more hospital admissions for any reason or 1 hospital admission for pneumonia, urinary tract infection (UTI), dehydration or sepsis in last 90 days of life141; or, admissions or the number of hospitalizations and transfers from LTC within a specific time period (i.e., last 180, 90 or 30 days). | 24,40,98,141,142 |
| Symptom management within LTC | Percentage of deaths with uncontrolled symptoms (including Stage IV pressure ulcers, anxiety, dyspnea, pain etc.). | 24,51,58,67,83,141–143 |
| Costs | Total monthly costs for nursing home residents in the last 6 months of life. | 24 |
| Hospice (n=36) | ||
| Patients’ enrollment and access | Enrollment to hospice across different time periods from death (i.e., last 3/7 days), as well as length of stay in hospice. Hospice care for disease groups, such as cancer and non-cancer were also compared. The number of referrals made to residential hospice, location of where patients were admitted to hospice from (LTC, Home care etc.), were also measured. Wait times of less than 3 days between referral and enrollment in hospice care, and number of referrals to hospice who died while waiting for admission were measures of access to care. | 15–17,20,43,45,48,49,51,54,60,64,66,67,69,74,77,78,86, 99,106,109,111,118,144 |
| Utilization | Hospice care use, bed utilization, and health care provider visits close to death | 16,20,55,106,144,145 |
| Quality of care | Measures of quality of care including screening, assessment and treatment (pain, dyspnea, bowel regimen) and patient/caregiver experience | 145 |
| Hospice quality | Rating of hospice and willingness to recommend | 145 |
| Costs | Funding provided to hospice | 43 |
| Hospital (n=39) | ||
| Emergency Department (ED) Use | ED utilization was measured as individual visits, or an average number of visits within a specific time period. ED use without a hospitalization, and unscheduled ED visits were also captured. | 12,13,15–17,19–21,39,41–43,48–50,54,56,61 ,63–65, 67–70,72–74,78–80,91,92,94,98,99,103, 110–112, 118,119,121,123,133–135,140,146 |
| Costs | Costs associated with an ED visit, inpatient admission, and episodes of care using a diagnostic test, or a specific type of hospital admission (such as Alternate Level of Care or Palliative Care Unit) were captured. | 13,18,67,68,99,102,104,109 |
| Hospital Utilization | Inpatient utilization was measured as number of days for length of stay, or the number of admissions during a specific time period (i.e., last year, 6 months, 3 months, 1 month, 30 days of life). The admitting unit of the admission was also specified, such as rehabilitation, intermediate bed, general bed, or Alternate Location of Care (ALC) bed. Hospitalization rates were also measured, including whether the hospital admission had an admitting diagnosis of palliative care and difference in rates based on those receiving care from a GP vs. not. | 9,12,13,16,17,21,38–40,43,45,47–50,54,56,60,63,64,67, 68,70,72–75,77–79,82,85,87,91,98,99,102,103,105,106, 108,112,118,123,130,131,134,135,139,140,144, 147–151 |
| Intensive Care Unit (ICU) Use | ICU admissions were captured as the number of days spent in ICU, or the number of visits within a specific time frame (i.e., last 6 months, 90 days, 30 days). The rate of deaths associated with an ICU admission was also assessed. | 13,15,17,19–21,39,44,45,47–50,54,59–61,63,64,69,70, 72–74,76–79,81,91,98,103,106,110,112,114,118,131, 133,148,152 |
| Hospital Readmission and Discharge | Hospital readmission rate or the number of readmissions within a specific study period (i.e., last year, 6 months, 90 days, 30 days). Patient discharge disposition, and whether there was home care or an ED visit prior to the admission, was also measured. | 9,18,36,43,51,57,58,67,74,86,92,102,103,140,147 |
| Alternate level of care (ALC) | Number of days spent in an ALC bed, and percentage of acute care/post-acute ALC patients waiting for their most appropriate destination. | 9,18,57,58,67,92,103,105 |
| In-hospital treatment and procedures | Rate of resuscitation for patients coded as palliative and other medical procedures | 73,79,81,98,118,140–142 |
| Physician Care (n=29) | ||
| Multidisciplinary and continuity of care | Multidisciplinary care was reflected by the number of different providers [i.e., social work, general practice (GP)]. Physician care was further scrutinized, including the percentage of physician visits seen that were GPs and non-GPs. The Modified Continuity Index (MMCI) was used to reflect continuity and collaborative care. | 18,21,38,57,65,87,91,98,103,144,148 |
| Home Visits | The number of patients who received home visits at different periods before death (i.e., last 2 weeks). Number of palliative physician home visits, proportion of patients with physician house calls in last 2 weeks of life, and number ofpatients with at least one physician house call. | 13,16,18,39,57,61,65,67,68,70,71,80,91,99,103, 110, 118,123,131,135,140,151 |
| Outpatient Care | Number of visits made to different outpatient health care providers, such as GPs and specialists, including medical oncologists. The percentage of palliative care delivered from primary care and from an outpatient palliative care clinic were also compared. |
16,18,38,39,65,68,83,89,91,94,98,99,118,130,131, 133,139,144 |
| Medications (n=26) | ||
| Cost | Drug costs for the patient, and costs incurred by the government, per prescription. | 9,18,40,46 |
| Symptom Management | Determined if symptoms were managed by the most appropriate medications (and for such specific disease types), such as short- or long-acting opiate within 60 days before death. Medications evaluated included opioids, narcotics, antipsychotics and anxiolytics. Also, overmedication and number of prescriptions, as well as aggressive treatment was also included, such as the prescribing of antibiotics at the EOL. | 21,40,44,46,61,69,74,84,87,98,107,115,118,126,139, 144,153–156 |
| Access to medications | Palliative drug benefits | 46,51,68,87 |
| Complex Continuing Care (n=3) | ||
| Palliative care delivered within Complex Continuing Care (CCC) | Pain management, and the delivery and cost of palliative care services within CCC and rehabilitation hospitals were measured. | 58,68,133 |
| Other (n=3) | ||
| Emergency services | Ambulance utilization | 56,87 |
| Lab/Medical Equipment | Medical and lab equipment expenses; diagnostic testing at EOL | 98,144 |
Cross sector
Within this review, a total of 58 collected indicators were categorized as transcending sectors or cross sector9,13–18,20–24,36–106 and four measurement themes were identified. Indicators classified under the cross sector category tended to focus on how well palliative and EOL patients were being cared for through examination of aggressiveness of care, place of care/death, availability of palliative care services and overall cost; this measurement was to better understand how the overall health care system was performing in the care of palliative and EOL patients, rather than on the performance on any specific health care sector by itself (e.g. the enrollment of patients in a palliative care program and how early this enrollment occurred).
Cancer care
A total of 57 indicators were extracted for measuring performance in the delivery of cancer care13,17,20,21,37,39,48,50,51,54,56,60,63,64,67,70,72–74,77–80,91,94,98,103,106–119. These indicators centered around three key measurement themes which were tied to the overall provision of quality care to cancer patients, including the regular assessment and management of symptoms, wait times for specific cancer care services, and aggressive care near the end-of-life.
Home care
A total of 48 indicators were categorized under the home care sector13,16–18,39,42,43,52,58,65,67,68,70,72,80,87,92,99,103,110,123,128,130–136 with four key measurement themes. Most indicators described patient utilization/cost of, or access to PC and/or home care services. Indicators on patient outcomes and quality of care emphasized specific clinical characteristics (e.g. pain, cognitive function), and involvement of specific health care providers (multidisciplinary care, social workers).
Long-term care
A total of 40 indicators were extracted for the LTC or nursing home sector43,48,54,60,61,67,70,74,76,87,91,100,106,137–144 classified under four key measurement themes. Indicators focused on the quality of care being provided through adequate symptom management and burdensome transitions at the EOL; indicators also focused on length of stay and costs of these patients at the end-of-life.
Hospice
A total of 36 indicators were categorized under the hospice sector25–29,31,34,45–47,51,52,54,57,58,60,63,64,67,69,72,75,76,80,81,90,96,101,116,122,128,141,142 and five key measurement themes were identified. Overall, indicators heavily emphasized access to timely, adequate and good quality hospice care, with a large number of indicators measuring access to hospice care across as measured based on different time periods prior to death, by disease groups, etc.
Hospital
A total of 39 indicators were extracted for the hospital sector19,22,23,25–29,31–35,41–54,56–60,63,64,66–86,89–91,95–108,110–113,116–119,120–122,123,127–133,135–143,145,148 under seven key measurement themes including ED use, intensive care unit use, and cost. Hospital-specific indicators largely focused on utilization, costs related to utilization, and the appropriateness of hospital care at the EOL.
Physician care
A total of 29 indicators were extracted for the physician care sector23,26,29,32,35,40,41,42,60,66,68,70,71–74,76,77,79,80,83,87,89,91,92,95,96,97,99,100,103,106,108,111,116,119,120,130,132,133,135,137,138,141,142,145,148 under three key measurement themes. Indicators largely emphasized understanding the intersection between generalist and specialized PC provided by different physicians (general physicians, PC specialists, oncologists etc.) in outpatient settings.
Medications
A total of 26 indicators were extracted on medication use9,18,21,40,44,46,61,69,74,84,87,98,107,115,118,126,139,144,153–156 under three measurement themes. Indicators were largely focused on the most appropriate medications for symptom management, including overmedication and aggressive treatment at the end-of-life.
Complex continuing care (CCC)
A total of three indicators were measured within the CCC sector that typically focuses on rehabilitation and/or PC. They focused on pain management108, access to PC58, and cost68. This sector was the most underdeveloped in the literature compared to other sectors with respect to performance measurement likely due to its uniqueness to the Ontario context.
Discussion
The purpose of our scoping review was to collect, organize, and share a distilled inventory of sector-specific health system performance measurement indicators for PC and EOLC. Our scoping review revealed 339 indicators, organized across nine distinct and one cross-cutting health care sector categories. Indicators within each sector category were subgrouped by key measurement themes. Collectively, these indicators represent the field’s thinking on how best to measure high health system performance in the delivery of both PC and EOLC.
One of the most commonly occurring measurement themes across sectors focused on health system utilization (e.g., hospital length of stay, ICU admissions, chemotherapy or home care received). Indicators of utilization were collected both for comparison across palliative/end-of-life and non-palliative/non-end-of-life populations (to represent access), to examine cost of delivered care, and also to understand how well the system was performing in the delivery of care (e.g. on wait times). However, emphasis on utilization through measurement of process indicators, such as in the hospital sector, highlighted that measurement themes such as patient experience and effectiveness of palliative care were not captured. Measurement of such themes through indicators can serve as an indication of the achievement of positive patient and health system outcomes.
Another frequent measurement theme across sector categories was the aggressiveness of care at the EOL for palliative and EOL populations. This theme included indicators of inappropriate treatment, medications, and transitions at the end-of-life that would be avoided in a well-performing health system focused on quality patient care. As health system resources become finite, this measurement theme will likely gain prominence in not only helping to improve patient outcomes, but also in reducing costs from unnecessary and aggressive health services.
Despite the wealth of indicators collected, the measurement of patient-centered or patient-reported outcomes was infrequent at the population-level using health administrative databases. Measurement of themes related to unmet care and self-management support needs, advance care planning, goals of care, consideration of patient preferences, and patient and caregiver burden were limited. However, absence of indicators from the review does not mean that measurement is not occurring, but rather, that it may not be occurring in population-level health administrative databases. Given the importance of these indicators, measurement through primary data collection may be required to obtain a comprehensive picture of health system performance.
Additionally, the literature review revealed that although there were a large number of indicators, many of these indicators measured similar constructs, but with different specifications. This includes variations in identifying the time period of measurement and the patient population. There were also differences in how debated concepts in the literature were operationalized such as what constitutes markers of quality care (example, how to define a burdensome transition). S1 Table (Extended data30) includes each of these collected indicator iterations across various data sources in more detail. The multiple variations of indicators – with little justification of why they were chosen – posed challenges in summarizing the data, and in recommending how these indicators should be measured. This difficulty was compounded by the absence of information on how indicators were operationally defined, including lack of information on the numerators and denominators in many of the extracted studies. As such, subsequent iterations of similar indicator themes resulted in issues of comparability across jurisdictions. While noting differences in data availability, efforts are needed to systematically define a set of standardized indicators for use across jurisdictions. There have been efforts by some stakeholders to implement accepted performance measurement frameworks98,151. Subsequent efforts can then be aimed at improving the construction and measurement of these indicators, and also in continuing efforts to evaluate system performance.
Limitations specific to the methods and results of this scoping review exist. Indicators were collected based on their ability to be extracted from a population-based data source, dependent on the judgement of study authors and subject matter experts which may have introduced some bias (e.g. large focus on Canadian grey literature). As one of the reasons for this scoping review was to better understand which health system performance indicators currently exist for local efforts of policy planning, a bias towards indicators that can be readily collected in the Canadian and/or Ontario context may exist. Moreover, quality appraisal of indicators was not conducted. Next, the conflation of the terms, palliative, end-of-life, and terminal care was difficult to operationalize when searching and organizing indicators. For the purposes of this review, all terms were included as a part of the search strategy to ensure indicators reflected all references to care related to advanced disease. When reviewing administrative data sources and medical-based records, health service encounters can be coded either or as palliative or end-of-life depending on the care setting and health service provider157,158. Authors however recognize that palliative care reflects a philosophy of care and service that can be provided at any time during the advanced disease trajectory, whereas end-of-life care is not always palliative, and is provided during the final period of a patient’s life. Lastly, this research study answers the question on the types and varieties of indicators available for measurement of performance in PC and EOLC using administrative data. However, it does not answer the question on what these indicators should be based on quality and policy criteria. Next steps would include conducting a consultation with relevant stakeholders to create a universally accepted performance measurement framework of what health system performance measurement for PC and EOLC should look like, using the list of indicators provided in this study as a reference guide. This would include patient-reported outcome measures that have not been included in this study. While some jurisdictions may not be ready to measure many of these indicators, findings from this study can provide insight on what can be potentially measured.
Overall, this study reviews – by health care sector – population-level health system performance indicators for PC and EOLC that can be measured through administrative databases. Although a large number of indicators have been reported for each sector, these indicators are often variations on the same theme, reflecting a lack of consensus on key debated concepts within the PC and EOLC literature. Future work is needed to achieve consensus ‘best’ definitions of these indicators as well as a universal performance measurement framework, similar to other ongoing efforts in population health121. Our scoping review will reduce duplication of the extensive amount of work that is required when a jurisdiction wishes to make a concerted effort to improve care in palliative and EOL populations through adoption of a performance measurement framework; one of the first steps in such efforts is typically to collect performance indicators from literature. This review can instantaneously inform indicator selection and development for other local, national and international efforts currently underway to improve PC and EOLC. Such performance measurement through indicators can help identify gaps in the access and quality of care and evaluate the impact of PC interventions that aim to bridge these gaps.
Data availability
Underlying data
All data underlying the results are available as part of the article and no additional source data are required.
Extended data
Dataverse: Supplementary files for Measuring indicators of health system performance for palliative and end-of-life care using health administrative data: a scoping review, https://doi.org/10.7910/DVN/R3S0RT30.
This project contains the following extended data:
Reporting guidelines
Dataverse: PRISMA-ScR checklist for ‘Measuring indicators of health system performance for palliative and end-of-life care using health administrative data: a scoping review’, https://doi.org/10.7910/DVN/R3S0RT30.
Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).
Download
Export To
metrics
| Views | Downloads | |
|---|---|---|
| F1000Research | - | - |
|
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Citations
CITE
how to cite this article
Budhwani S, Gill A, Scott M et al. Measuring indicators of health system performance for palliative and end-of-life care using health administrative data: a scoping review [version 1; peer review: 2 approved with reservations]. F1000Research 2020, 9:1438 (https://doi.org/10.12688/f1000research.27121.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.
Open Peer Review
Current Reviewer Status:
Key to Reviewer Statuses
VIEW
HIDE
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations
A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Version 1
VERSION 1
PUBLISHED 10 Dec 2020
Views
9
How to cite this report:
Cohen J and Piette V. Reviewer Report For: Measuring indicators of health system performance for palliative and end-of-life care using health administrative data: a scoping review [version 1; peer review: 2 approved with reservations]. F1000Research 2020, 9:1438 (https://doi.org/10.5256/f1000research.29959.r95959)
The direct URL for this report is:
https://f1000research.com/articles/9-1438/v1#referee-response-95959
https://f1000research.com/articles/9-1438/v1#referee-response-95959
NOTE: it is important to ensure the information in square brackets after the title is included in this citation.
Reviewer Report 28 Oct 2021
Approved with Reservations
VIEWS 9
This study provides a useful and welcomed overview of EOLC quality indicators that have been developed or used in published studies.
Overall the presentation is clear, but we have a number of comments and suggestions the authors may wish ... Continue reading
Overall the presentation is clear, but we have a number of comments and suggestions the authors may wish ... Continue reading
CITE
HOW TO CITE THIS REPORT
Cohen J and Piette V. Reviewer Report For: Measuring indicators of health system performance for palliative and end-of-life care using health administrative data: a scoping review [version 1; peer review: 2 approved with reservations]. F1000Research 2020, 9:1438 (https://doi.org/10.5256/f1000research.29959.r95959)
The direct URL for this report is:
https://f1000research.com/articles/9-1438/v1#referee-response-95959
https://f1000research.com/articles/9-1438/v1#referee-response-95959
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Views
13
How to cite this report:
Evans CJ. Reviewer Report For: Measuring indicators of health system performance for palliative and end-of-life care using health administrative data: a scoping review [version 1; peer review: 2 approved with reservations]. F1000Research 2020, 9:1438 (https://doi.org/10.5256/f1000research.29959.r80857)
The direct URL for this report is:
https://f1000research.com/articles/9-1438/v1#referee-response-80857
https://f1000research.com/articles/9-1438/v1#referee-response-80857
NOTE: it is important to ensure the information in square brackets after the title is included in this citation.
Reviewer Report 06 Apr 2021
Approved with Reservations
VIEWS 13
Thank you, this is an interesting and comprehensive scoping review on health system quality/performance indicators for palliative care and end of life care. The scope of the review is impressive in aiming to encompass all settings and conditions, and inclusion
... Continue reading
CITE
HOW TO CITE THIS REPORT
Evans CJ. Reviewer Report For: Measuring indicators of health system performance for palliative and end-of-life care using health administrative data: a scoping review [version 1; peer review: 2 approved with reservations]. F1000Research 2020, 9:1438 (https://doi.org/10.5256/f1000research.29959.r80857)
The direct URL for this report is:
https://f1000research.com/articles/9-1438/v1#referee-response-80857
https://f1000research.com/articles/9-1438/v1#referee-response-80857
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Comments on this article Comments (0)