Keywords
drug safety, pharmacovigilance, post-authorisation studies, study design
drug safety, pharmacovigilance, post-authorisation studies, study design
We have made edits to the Introduction, to provide a more detailed background to post authorisation studies; the Methods, to more clearly specify what we did; and also to the Discussion, to better highlight limitations of the study as suggested by the reviewers.
See the authors' detailed response to the review by Xavier Kurz and Thomas Goedecke
See the authors' detailed response to the review by Lex Bouter
Randomised trials are considered the gold standard in evaluating the efficacy of new healthcare interventions. However, despite their ability to potentially provide causal estimates of the efficacy of new treatments, their generalisability to patients in the real world is often unclear. Strict criteria excluding patients with comorbidities or those above a certain age may lead to participants in trials differing from the general clinical population1,2. Observational studies offer a means of further characterising the safety and effectiveness of new healthcare interventions in real world clinical settings3.
Post-authorisation studies (PAS), also referred to as ‘postmarketing surveillance studies’ or ‘phase IV trials’, are an example of this type of research which aims to demonstrate the utilisation and safety profile of drugs following their regulatory approval. While these studies are predominantly non-interventional, PAS can include interventional study designs (as per the definition of post-authorisation safety study in GVP VIII). Historically, observational PAS have been criticized to be of poor quality and open to bias in order to increase sales at the cost of scientific rigor4. European pharmacovigilance legislation was introduced in an effort to increase transparency and ensure such studies are methodologically robust5.
Since July 2012 all PAS that are imposed as a condition of granting marketing authorisation are required to be published in a publicly available register. In Europe, the main register for PAS is the EU PAS Register and is held by the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP). While one previous study has examined broad characteristics of studies registered on the EU PAS, the current study aimed to build on this work by further characterising studies requested by the EMA to understand trends in study design, data sources utilised, and the relationship to therapeutic area. As PAS are becoming increasingly mandated by the European Medicines Agency (EMA) and compliance is high6, approved PAS protocols can likely provide insights into the successful design of future studies.
Analysis focused on all PAS requested by the EMA which were recorded on the EU PAS Register as of 18th October 2016. The EU PAS Register is accessible via the ENCePP website which is hosted and maintained by the European Medicines Agency (EMA). All studies in category 1 (imposed as a condition of marketing authorisation), category 2 (obligation of marketing authorisation), and category 3 (required by the risk management plans) on the EU PAS register were included. These studies currently (as at 6th November 2017) represent 40% of studies recorded on the register. Our study results are therefore not generalisable to studies not requested by regulatory authorities.
Data from the Register on study status (finalised, ongoing or planned) were collected to allow comparison of studies across time. Information was also collected from the Register on medical condition to be studied (i.e. the disease area of interest listed by the study investigators), data sources used (if applicable), and study scope (effectiveness evaluation, drug utilisation or risk assessment studies). These data were collected from the publicly available information published on the PAS register (http://www.encepp.eu/encepp/studiesDatabase.jsp).
Studies were classified as either using primary data capture, secondary data capture, or a hybrid approach as used by Engel and colleagues7 who studied PAS protocols and assessments submitted from July 2012 to July 2015 to the EMA Pharmacovigilance Risk Assessment Committee (PRAC). Primary data capture refers to the collection of data specifically for the study. Secondary data is the use of data already collected for another purpose (e.g. administrative or claims databases; medical charts). Studies using a combination of primary and secondary approaches were classed as employing a hybrid approach. Studies using alternative approaches (e.g. interventional studies) to those described were not included, limiting our findings to the aforementioned study designs. It should be noted that study details on the register is not quality checked and therefore misclassification is possible.
Descriptive analysis was performed. Counts and percentages were used for categorical variables.
As of 18th October 2016, a total of 314 studies were identified on the EU PAS register. These studies included 81 (26%) finalised, 160 (51%) ongoing and 73 (23%) planned (Table 1).
Risk Assessment | Effectiveness Evaluation | Drug Utilization | Total | |||||
---|---|---|---|---|---|---|---|---|
Finalized | 42 | 20% | 11 | 12% | 47 | 35% | 81 | 26% |
Ongoing | 112 | 55% | 60 | 64% | 59 | 44% | 160 | 51% |
Planned | 51 | 25% | 23 | 24% | 27 | 20% | 73 | 23% |
Total | 205 | 100% | 94 | 100% | 133 | 100% | 314 | 100% |
Of the total studies identified, 205 (65%) included risk assessment in their scope, 133 (42%) included drug utilisation and 94 (30%) included effectiveness evaluation. The same study could cover more than one objective: 6 (2%) studies included drug utilisation and effectiveness evaluation, 36 (11%) drug utilisation and risk assessment, 32 (10%) risk assessment and effectiveness evaluation and 22 (7%) risk assessment, drug utilisation and effectiveness evaluation.
For the risk assessment studies, finalised studies were less common (42; 52% of all 81 finalised studies) than ongoing (112; 70% of all 160 ongoing studies) and planned (51; 70% of all 73 planned studies). For the effectiveness evaluation studies, again finalised studies were not as frequent (11; 14% of all 81 finalised studies) as ongoing (60; 38% of all 160 ongoing studies) or planned (23; 32% of all 73 planned studies). Finally, for the drug utilisation studies, finalised studies were more common (47; 58% of all 81 finalised studies) with 59 (37% of all 160 ongoing studies) ongoing and 27 (37% of all 73 planned studies) planned.
Just over half of the studies (175, 56%) used a primary data capture design, 135 (43%) used a secondary data capture design and 4 (1%) used a hybrid design combining both approaches.
More primary data capture studies were ongoing 102 (64% of all 160 ongoing studies) or planned 49 (67% of all 73 planned studies) than finalised 24 (30% of all of all 81 finalised studies). For secondary data capture, finalised studies were more frequent (57; 70%), as compared to 55 (34%) ongoing and 23 (32%) planned.
Risk assessment and effectiveness studies were more likely to use primary data capture (60% and 85% respectively for primary data capture as compared to 39% and 14% respectively for secondary data capture). The converse was true for drug utilisation studies, where 59% used secondary data capture vs. 39% for primary data capture.
Of the secondary data capture studies, 117 (87%) used an existing claims or electronic medical record database (the remainder using a chart review approach). 93 (79%) studies used an existing real-world data source based in Europe alone, 9 (8%) studies used European and US data, 14 (12%) studies used US data alone and 1 (1%) study used Canadian data.
A single database was used in 58 (50%) studies, with the remainder using two or more. Where more than one data source was used, this was always from two or more countries. The most frequent established data source used was the United Kingdom’s (UK) Clinical Practice Research Datalink (CPRD, 31%), followed by Nordic (Denmark, Finland, Norway or Sweden) National registries (29%), and The Health Improvement Network (THIN, UK, 18%).
A total of 30 (10%) studies were in the field of type 2 diabetes mellitus, 29 (9%) in cardiovascular disease, 27 (9%) studies in oncology, 6 (2%) in chronic obstructive pulmonary disease (COPD) and 6 (2%) in multiple sclerosis (MS).
For type 2 diabetes mellitus, database studies were more commonly used (80% vs 3% chart review, 3% hybrid and 13% primary data capture). Similar patterns were seen for COPD (83% database vs 17% primary data capture) and cardiovascular disease (59% database vs 38% primary data capture and 3% hybrid design).
For studies in oncology, primary data capture was more likely to be used (85% vs 4% chart review, and 11% database study designs). A similar relationship was seen for MS (83% primary data capture vs 17% database study design).
Our analysis reveals a number of distinct characteristics of studies recorded on the register that can likely be used as a reference for future PAS designs.
Overall there appears to have been an increase in the number of risk assessment studies, as planned and ongoing studies were far more likely to include a risk assessment element than studies that had already been finalised.
Primary data capture study designs were the most commonly implemented study type, and again these were more frequently planned or ongoing, suggesting an increasing use or reflecting the longer nature in general to execute these types of studies. Furthermore, primary data capture were more commonly used for effectiveness and risk assessment studies. The predominance of primary data capture designs is similar to results observed by Engel et al., who studied 189 PAS protocols submitted to the EMA PRAC7.
Secondary data capture was more likely to be used for drug utilisation studies, and this was also described by Engel et al.7 Of the secondary studies, the CPRD in the UK was used in nearly a third of all database PAS. Registry data from the Nordic countries was only slightly less commonly used, with the remainder of existing data sources (eg. THIN, in the UK) being less frequently used.
Of the therapeutic areas investigated, type 2 diabetes mellitus, cardiovascular disease and COPD tended to use secondary data capture, whereas for oncology and MS, primary data capture studies were favoured.
Some limiting characteristics of current data sources in Europe are that they may only cover a specific type of patient care (e.g. primary or secondary) and not the full patient pathway. They may not collect data on everything that is needed for PAS objectives, for example in-hospital treatments, clinical endpoints or markers of disease severity. Given PAS studies are often requested on the grounds of exploring sub-groups of patients who may have been treated outside of these settings8, this is a major limitation of some data sources. Indeed, this may explain why primary data capture study designs are becoming more frequent, especially in the field of oncology. Furthermore, that a handful of data sources are very commonly used highlights the limited data landscape in Europe, and the need to improve coverage and access to routinely collected medical data for observational research. On the other hand, given some similarity in data elements captured in currently available datasets, studies using only a single database may have potentially missed a chance to validate any findings or increase statistical power to detect relevant effects. Additionally, some data sources (eg. CPRD) can be linked to other datasets or have an integrated data collection component to provide additional data. Nevertheless, the tendency for effectiveness and risk assessment studies to use primary data capture is also likely a reflection of the fact that this methodology can potentially overcome problems such as confounding, often associated with some secondary designs due to a lack of data availability on potential confounders.
In summary, we show here that using data from the EU PAS Register, study scope and therapeutic area are associated with the choice of study design agreed by health authorities and therapeutic drug makers for the conduct of PAS. While researchers should consider relevant guidelines (ENCePP Guide on Methodological Standards in Pharmacoepidemiology, Guidelines for Good Pharmacoepidemiology Practices of the International Society of Pharmacoepidemiology (ISPE GPP)) when designing PAS, the results of this study can contribute to the planning of future successful PAS.
Dataset 1: The full dataset used for analysis, extracted from the PAS register (http://www.encepp.eu/encepp/studiesDatabase.jsp) on the 18th of October 2016.
RC, JC and DL are full-time employees of Evidera. SR and LM are full-time employees of Bristol-Myers Squibb.
Views | Downloads | |
---|---|---|
F1000Research | - | - |
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Epidemiology, methodology, research integrity
Competing Interests: No competing interests were disclosed.
Is the work clearly and accurately presented and does it cite the current literature?
Partly
Is the study design appropriate and is the work technically sound?
Partly
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
No
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
No
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Epidemiology
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Partly
If applicable, is the statistical analysis and its interpretation appropriate?
Partly
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Partly
Competing Interests: The reviewers are employees of the European Medicines Agency, which hosts the EU PAS Register.
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | ||
---|---|---|
1 | 2 | |
Version 2 (revision) 10 Nov 17 |
read | read |
Version 1 14 Aug 17 |
read | read |
Click here to access the data.
Spreadsheet data files may not format correctly if your computer is using different default delimiters (symbols used to separate values into separate cells) - a spreadsheet created in one region is sometimes misinterpreted by computers in other regions. You can change the regional settings on your computer so that the spreadsheet can be interpreted correctly.
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Already registered? Sign in
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)