Keywords
Pharmacoepidemiology, Registries, Denmark, Outcome measures, Drugs, Protocol
Pharmacoepidemiology, Registries, Denmark, Outcome measures, Drugs, Protocol
Randomised clinical trials (RCTs) are the “gold standard” to demonstrate the efficacy of a medicinal product – “if the product can work” – under controlled conditions.1–3 Randomisation reduces the bias due to baseline confounding and is therefore a major asset of RCTs. However, RCTs are often conducted on a more homogenous patient population in relation to age, disease severity, comorbidities, and comedication compared to those patients seen in clinical practice 3–6 as well as in a more specialised “ideal” setting.3,7 Furthermore, RCTs are often limited in number of participants and duration, which can prevent the observation of rare events or assessment of long-term effects.3,7 These factors can limit the generalisability of RCTs. Real-world evidence (RWE) studies can offer important complementary information to the evidence established in RCTs.1,6,8 RWE studies provide a mean to gain information regarding the effectiveness of a medicinal product – “if the product actually does work” - in routine clinical practice (real-world setting).1–3
The interest and focus on RWE studies on effectiveness and safety of medicinal products has been increasing. The European Medicines Agency (EMA) and the U.S. Food and Drug Administration (FDA) have long had the authorisation to require the pharmaceutical industry to perform both post-authorisation safety studies and post-authorisation efficacy studies. 9–11 However, within the last couple of years, both EMA and FDA have begun the process of developing new regulatory frameworks and guidelines for the use of real-world data (RWD) and RWE to support, for example, determination of product effectiveness and safety, or approval of new indications for the medicinal product.12–15 One important source of RWD is registries, which can provide information about diseases, patients characteristics, utilisation and outcomes of treatments.15,16
Denmark has a long history of maintaining nationwide registries capturing a wide variety of longitudinal population-based data.17–19 In Denmark, information regarding redeemed prescriptions and in-hospital medication use is registered using the Anatomical Therapeutic Chemical (ATC) classification system,20,21 which can be interlinked on individual level through the unique personal identifier, the Central Personal Register number18–21, to data from patient registries, prescription registries, registries containing laboratory data, or contacts with general practitioners and other medical specialists, several registries on sociodemographic and employment status, 84 clinical quality databases as well as several research databases.22 An overview of the Danish health data and more than 150 registries/databases is available from the Copenhagen Healthtech Cluster webpage. These data offer unique possibilities for Danish pharmacoepidemiological research to evaluate the effectiveness and safety of medicinal products in clinical practice.18,20,21 The national registries in Denmark are mainly developed for purposes of financial accounting and quality control.17–21 Therefore, it is important to be aware of the potential limitations of these registries such as uncertain validity of data or missing important outcomes when used for RWE studies.18–21 It can be challenging to identify validated and reliable outcome measures for RWE studies. For example, the limitations of the data available in the registries may necessitate the choice of outcomes which are not clinically most relevant. In order to help identifying potential, but hitherto unexplored registry-based outcome measures, that can better target the clinical scope of RWE studies evaluating the effectiveness and safety of medicinal products, an investigation of the currently used outcomes measures is required.
Thus, the objectives of this systematic mapping review23–25 are: (1) to give an overview of the types of outcome measures currently used in Danish registry-based studies evaluating the effectiveness or safety of medicinal products, and link these to the therapeutic exposure (ATC level 2) and to the registries of origin; and (2) to investigate how the studies address the quality (reliability, validity and responsiveness) of the outcome measures used.
The study is conducted as a systematic mapping review. Systematic mapping reviews provide overviews of research characteristics within a topic area by structuring and categorising the existing literature. The systematic mapping review focuses on the research practices related to the findings,e.g. the methodology or publication characteristics of the included studies.23–25
The screening is expected to begin in December 2020. This will be followed by the data extraction which is expected to begin in March 2021.
Study characteristics
This systematic mapping review will include registry-based studies reported in English and published in the period January 1st, 2018 to December 31st, 2019 describing original research on the relationship between exposure of a medicinal product and an effectiveness/safety outcome, where data on outcomes are obtained from Danish registries or databases.
The terms “exposure of a medicinal product” must comply with the EMA definition of a medicinal product available from the EMA webpage glossary: “A substance or combination of substances that is intended to treat, prevent or diagnose a disease, or to restore, correct or modify physiological functions by exerting a pharmacological, immunological or metabolic action”.
The literature search strategy will be developed and completed first in the electronic database PubMed®/MEDLINE using free text words and Medical Subject Headings (MeSH) related to Danish registry-based research on effectiveness or safety of medicinal products. The development of the search strategy will be achieved by testing different combinations of search terms. For each combination, the resulting number of records (N) will be noted and the relevancy of the search will be determined by screening the first 10-20 article headlines and summaries when the PubMed®/MEDLINE search is sorted by: Best Match. Each identified search term will be continuously evaluated for its relevance in the final search string. In addition, the search string may be further specified using the [Title/Abstract] search field tag. The final search string developed in PubMed®/MEDLINE is stated below.
In order to identify and categorise the currently used outcome measures in Danish registry-based studies, we aim at including approximately 250 articles for data extraction. We believe that the relatively short two-year publication period limiting the inclusion will be representative of the current practice but at the same time reduces the number of Covid-19 related studies included as despite of their relevance, these will potentially skew the picture of outcome measures used to evaluate the effectiveness and safety of medicinal products.
Following the completion of the search strategy in PubMed®/MEDLINE, the same strategy will be applied in the electronic database Scopus® to ensure a more comprehensive literature search. The final search string applied to Scopus® is available below. The results of the literature searches in PubMed®/MEDLINE and Scopus® serve as input for the screening process.
PubMed®/MEDLINE (performed 18th November 2020; 783 records)
("registries"[MeSH Terms] OR "registries"[TIAB] OR "registry"[TIAB] OR "register"[TIAB] OR "registers"[TIAB]) AND ("danish"[TIAB] OR "denmark"[MeSH Terms] OR "denmark"[TIAB]) AND ("therapeutics"[MeSH Terms] OR "therapeutics"[TIAB] OR "drug therapy"[MeSH Subheading] OR "drug therapy"[MeSH Terms] OR "therapeutic use"[MeSH Subheading] OR "therapeutic use"[TIAB] OR "pharmaceutical preparations"[MeSH Terms] OR "medication"[TIAB] OR "medications"[TIAB] OR "drug"[TIAB] OR "Pharmacoepidemiology"[MeSH Terms] OR "Pharmacoepidemiology"[TIAB]) Filters: Humans, English, from 2018/1/1-2019/12/31
Scopus® (performed 6th December 2020; 463 records)
TITLE-ABS-KEY (("registries" OR "registry" OR "register" OR "registers") AND ("danish" OR "denmark") AND ("therapeutics" OR "drug therapy" OR "therapeutic use" OR "pharmaceutical preparations" OR "medication" OR "medications" OR "drug" OR "Pharmacoepidemiology")) AND PUBYEAR > 2017 AND PUBYEAR < 2020 AND (LIMIT-TO (LANGUAGE, "English"))
Validation of selection process
Prior to the screening process, three researchers will conduct a validation of the eligibility criteria to ensure consensus. Each researcher will independently review the title and abstract of 20-40 randomly selected records from the literature search, and categorise them into “relevant”, “unsure”, or “irrelevant”. If agreement is reached, the screening process will proceed. In case of disagreement, the three researchers will decide together and potentially correct or specify the eligibility criteria.
Selection process
The study records obtained from the literature searches in PubMed®/MEDLINE and Scopus® will be screened by one of the three researchers based on title and abstract using Rayyan26 review software, and the records will be categorised as stated above. Records categorised as “unsure” will be screened independently by the two other researchers. The three researchers will discuss the results and decide together how to categorise these records.
Validation of data extraction
Prior to data extraction, the process and the item list (Table 1) will be checked and validated to ensure uniform interpretation. Data extraction of minimum 30 randomly selected eligible studies will be conducted independently by least three researchers. If agreement is reached in ≥80% of the cases, the data extraction process will proceed. If disagreement occurs in >20% of the cases, a larger number of articles will be included in the validation process. If disagreement continues, the entire assessment for eligibility and data extraction will be conducted by more than one researcher. During this validation process, the item list will potentially be corrected or specified.
Data item | Description | Scoring |
---|---|---|
Publication characteristics and eligibility assessment | ||
Author(s) | The author(s) of the article. | Text string |
Title | The title of the article. | Text string |
Journal | The journal in which the article was published. | Text string |
Publication year | The year in which the article was published. If the article has been published in both print and electronic, the print year of publish will be stated. | Number |
Language | Is the language of publication English? | Yes No |
Exposure | Does the exposure of interest comply with the EMA definition of a medicinal product? | Yes No |
Registry-based | Is the exposure-related outcome(s) obtained from a Danish registry/database? | Yes No |
Eligibility | The article will be included for further data extraction if “Language”, “Exposure”, and “Registry-based” are all scored “Yes”. Otherwise, the article will be excluded, and no further data extraction will be performed. | Include Exclude |
Study characteristics | ||
Study design | The type of study design, e.g. cohort study, case-control study, nested case-control study, case-cohort study, cross-sectional study, etc. Descriptive adjectives such as a “matched” cohort study will also be stated. | Text string |
Patient enrolment_start | The year in which patient enrolment begins. | Number |
Patient enrolment_end | The year in which patient enrolment ends. | Number |
Request | Whether the study was imposed by EMA, FDA, or other? | EMA FDA Other Not specified |
Data of interest | ||
Medicinal product_ATC1 | The ATC 1st level of the medicinal product(s) of interest. If the ATC code is not explicitly stated, it will be found based on the generic product name using the searchable version of WHOCC - ATC/DDD Index. In the case, where neither the ATC code nor the generic product name is available, the medicinal product exposure will be categorised as “Other”. | A: Alimentary tract and metabolism B: Blood and blood forming organs C: Cardiovascular system D: Dermatologicals G: Genito urinary system and sex hormones H: Systemic hormonal preparations, excluding sex hormones and insulin J: Anti-infective for systemic use L: Antineoplastic and immunomodulating agents M: Musculoskeletal system N: Nervous system P: Antiparasitic products, insecticides and repellents R: Respiratory system S: Sensory organs V: Various. Other |
Medicinal product_ATC2 | The ATC 2nd level of the medicinal product(s) of interest. If the ATC code is not explicitly stated, it will be found based on the generic product name using the searchable version of WHOCC - ATC/DDD Index. | Text string |
Outcome measure_specific | The specific Danish registry-based exposure-related outcome measure assessed in the study. | Text string |
Outcome measure_subgroup | The Danish registry-based outcome measure assessed in the study are categorised into the subgroup given in the scoring column. If no subgroup is applicable, the outcome measure will be scored as “Other”. See Table 2 for the definition of each subgroup. | Apgar Score All-cause death All-cause death offspring Cause-specific death Cause-specific death offspring Cost Diagnosis Diagnosis offspring Diagnosis/prescription combination Diagnostic measures Drug discontinuation/switch Healthcare utilisation Hospitalisation Physical ability test Preterm birth Prescription Severity scale (Disease) Small for Gestational age (SGA) Surgery and procedures Quality of Life (QoL) Work ability/Productivity/Connection to labour market Other |
Outcome measure_registry | The specific registr(y/ies) from which the outcome measure has been obtained. If more than one registry has been used, an additional column will be added in the extraction sheet. | Text sting |
Outcome measure_quality | If applicable, the sentence(s) in which the article states, comments or consider the quality of the specific outcome measure used or the registry from which it has been obtained will be extracted. Otherwise, the item will be scored as “Not considered”. The following terms will be sought systematically: reliability, reproducibility, validity, specificity, completeness, sensitivity, positive predictive value (PPV), negative predictive value (NPV), responsiveness, and responsive. The terms must be related to the outcome measure or registry. | Text string |
Outcome measure_quality reference | If applicable, the reference used to support the quality of the specific outcome measure or the registry from which the outcome measure has been obtained. Otherwise scored as “NA”. | Text string |
Data extraction
Articles assessed as “relevant” will be full-text reviewed by one researcher for eligibility based on criteria stated under the section “Eligibility criteria”. The excluded articles will be assigned a reason for exclusion. In case of doubt, one or more researchers will be consulted. The result will be reported as a modified PRISMA 2009 Flow Diagram.27 Data from the studies identified as eligible will be extracted and scored according to the item list (Table 1) and the outcome measures will be categorised into the subgroups defined in Table 2. Data related to sensitivity analyses will be ignored. In the prepared data extraction sheet, the data extraction is controlled by the ATC level 2 of the exposure of interest and the outcome measure. Thus, if more than one medicinal product is investigated which differ in ATC level 2, or if more than one outcome measure is presented in a study, a new row will be established for each medicinal product or outcome measure with the same information stated in the previously extracted items (Table 1). The data items in Table 1 were identified based on (1) one researcher performing a data extraction test of relevant articles identified in the early process of search strategy development,28–34 (2) a review protocol on incident- and prevalent-user designs in observational comparative effectiveness and safety studies,35 (3) the hypothetical categorical overview of outcome measures presented in Table 3, and (4) from general discussion between the co-authors of which items could be relevant for this study and how to score them.
In the final report, the results will be presented as a categorical overview in which the outcome measures will be assigned into different outcome categories (with potential subcategories) according to the type of measure (e.g. clinical, work-related, economic and utilisation, humanistic). The hypothetical categorical overview presented in Table 3, a result of preliminary literature research36–39 and conceptualisation, has helped shaping the project idea and served as a starting point for conducting the systematic mapping review. It is important to note that the stated outcome measures cannot necessarily be obtained from Danish registries. Furthermore, the outcome measure categories/subcategories are not completely mutual exclusive.
In addition, the resulting outcome measures subgroups will be presented as absolute numbers and percentages of the total number of studies analysed and will be linked to (1) the therapeutic exposure on ATC level 2, and (2) to the specific registry from which outcome measure has been obtained. In addition, a table summarising the information collected for each study included in the analysis will be made available. How the studies have addressed the quality of the outcome measure used will be used in a narrative interpretation and discussion of the results.
A literature review mapping and categorising outcome measures used in Danish registry-based studies across different therapeutic areas, has to our knowledge never been conducted. However, some limitations associated with the study design are important to notice. The review will only include studies published in English which may introduce language bias, though we expect only a negligible number of publications in Danish, as Danish is a small language and therefore most scientific communication is made in English. The studies included will be limited by a two-year publishing period; January 1st, 2018 to December 31st, 2019. Thus, the results of the systematic mapping review are expected to be representable of the current practice within Danish registry-based studies but will be unable to depict the historical development within the field.
The EQUATOR Network contain no reporting guidelines for systematic mapping reviews. Thus, the importance of disseminating the present study protocol is to achieve optimal transparency and consistency in the review method used.
Views | Downloads | |
---|---|---|
F1000Research | - | - |
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
At the request of the author(s), this article is no longer under peer review.
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Already registered? Sign in
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)