Regulatory gaps and research waste in clinical trials involving women with metastatic breast cancer in Germany

Clinical trials of investigative medicinal products

Since 2014, sponsors have been obliged to disclose the results of Clinical Trials of Investigative Medicinal Products (CTIMPs) on the European EudraCT trial registry within one year of trial completion. This long-standing regulatory guidance became legally binding under German national law when the EU Clinical Trials Regulation became fully applicable in the wake of the launch of Europe’s successor registry CTIS during 2022-2023.¹⁵ With minor exceptions, sponsors of CTIMPs are legally obliged to disclose the results of all CTIMPs registered on CTIS within one year of trial completion at most.¹⁶ Under German law, sponsors that fail to do so risk a fine of up to 25,000 Euros.¹⁷ Depending on the type of CTIMP, either BfArM or Paul-Ehrlich-Institut (PEI) is the responsible German regulator, with BfArM regulating the large majority of CTIMPs. Thanks to strong engagement by both regulators, future compliance with CTIMP reporting requirements is likely to be very strong.¹⁴

Clinical trials of medical devices

The EU Medical Device Regulation requires the results of certain trials of medical devices to be made public within one year of study completion on the new Eudamed database.¹⁸ This provision has been integrated into German law.¹⁷ However, the European Commission has repeatedly delayed the full launch of Eudamed, leaving sponsors in Germany and elsewhere in Europe confused about regulatory expectations. German device studies falling within the scope of the European regulation are overseen by BfArM, which seems likely to face an uphill struggle in securing compliance in coming years.

All other clinical trials

For the purpose of this study, ‘other trials’ are defined as interventional studies listed on a clinical trial registry that fall outside the scope of European drug and medical device regulations. Examples include trials of surgical techniques, physiotherapy, or behavioural interventions. There is no legal requirement to make public the results of such trials under European law or German federal law. Those trials are instead regulated by provisions set out individually by Germany’s 16 federal states, none of which appears to mandate or monitor results reporting.¹⁷ Therefore, it is currently completely legal in Germany for sponsors of such trials to recruit and treat patients and later not make study results public. The Working Group of Medical Ethics Committees in Germany has called for lawmakers to adopt legislation that fully reflects Declaration of Helsinki requirements.¹⁹

Data acquisition and cohort generation

German sponsors commonly use three WHO primary trial registries. CTIMPs are directly registered on EudraCT (which is now being replaced by CTIS) by the responsible German regulator. All other trials, including device trials, are commonly registered by sponsors or by individual investigators either on the American ClinicalTrials.gov or the German DRKS registries. Double and triple registrations of the same trial on different trial registries are discouraged by the WHO but are common.

In October 2022, a team member (DG) searched all three trial registries for interventional studies with at least one study site in Germany that had been completed or terminated between 01 January 2016 and 31 December 2020 that contained the term “metastatic breast cancer” in either the condition/disease, trial title, or ‘other terms’ field in their registry entries. He then downloaded all available trials that matched these criteria and removed duplicates, generating a cohort of 358 trials (flowchart in the protocol). We manually reviewed all 358 trials and removed those that had not enrolled women with metastatic breast cancer inside Germany, arriving at a final study cohort of 70 trials.

Search strategy

We (DS,TJ,OB,TIB,CC,KL,TP,AS,MV,RB,SR) then performed two rounds of systematic registry and literature search for the results of these 70 trials during November and December 2022. The first round involved: 1) checking ClinicalTrials.gov records for native tabular results and any linked journal articles either uploaded by the sponsor or automatically linked to the registry; 2) searching Google Scholar for the ClinicalTrials.gov NCT ID; and 3) searching Google Scholar using the trial title and listed primary investigators. A team member not involved in the original search (TB) reviewed the data and followed up on selected publications, including all that had been marked as difficult to classify. We then conducted a second round of systematic literature searches using PubMed covering all trials for we had not located any results. The search strategies used are described in detail in the study protocol.

Stakeholder outreach

In December 2022, a team member (TS) contacted the sponsors of all trials for which we had been unable to locate outcomes on the registry or in the scientific literature to check whether relevant publications had been overlooked. We used sponsors’ press offices as point of contact whenever possible. In January and February 2023, we sent reminder emails to those sponsors who had not responded to our initial outreach. All sponsors we contacted eventually responded.

Quality assurance and final updates to the dataset

In February 2023, a team member (DS) double-checked all data points against registry data and updated our dataset in line with recent changes. We double-checked all DRKS trial numbers and removed those no longer available on the DRKS registry due to a recent decision by the registry to remove legacy double registrations. On 08 March 2023, a team member (TB) performed a final search for the outcomes of the 4 trials that remained listed as unreported.

Outcomes

As per study protocol, we documented whether and when the 70 trials in our cohort had made their results public either on trial registries or in the scientific literature. For all trials, we tabulated the number of enrolled patients based on registry data as of February 2023.

Results availability

Out of the 70 trials in our cohort, 66 (94.3%) had made their results public either on a trial registry or in the scientific literature by March 2023. The outcomes of the remaining 4/70 trials (5.7%) remained completely unreported.

Publication speed

The average time from trial completion to results publication was 584 days (median: 406 days). Only 13/70 trials (18.6%) of trials had published results within one year as recommended by the WHO. In two cases, results had only been made public after more than five years.

In total, 22,907 women participated across all 70 clinical trials. Results for 3,657 participants (16.0%) had been made public within one year as recommended by the WHO. Results for 19,120 participants (83.5%) were made public after a delay of more than one year. Results for 130 participants (0.6%) remained unpublished (Table 2).

Summary of findings

We identified 70 clinical trials involving 22,907 women with metastatic breast cancer with study sites within Germany. Of those 70 trials, only 13 trials (18.6%) involving 3,657 trial participants (16.0%) had made their results public within one year of trial completion as recommended by the WHO.

Context

The nonpublication rate of 5.7% that we found in this cohort is significantly lower than that found in other recent large-scale studies of German academic trials.^9,10,21 Our study does not call these previous findings into question. Our far smaller cohort of only 70 trials is dominated by commercial sponsors and includes a higher proportion of drug trials; both factors are associated with higher reporting rates. We did not disaggregate reporting performance by type of sponsor, funding source or trial within our small cohort because we did not pre-specify such an analysis.

Unreported and terminated trials

We located only 4 unreported trials. The sponsor of 1 trial (Merrimack Pharmaceuticals, NCT02213744 and 2014-003159-73, 133 participants) pledged to attempt to recover and report its results. As of 02 March 2024, Merrimack has yet to publish those results. The sponsors of the other 3 trials stated that they did not plan to make results public due to low recruitment numbers (3-11 patients); all 3 trials had been terminated early. In the latter 3 cases, the potential scientific value added by uploading the results onto a registry arguably does not justify the investment of finite researcher time. However, neither the Declaration of Helsinki nor current European and American reporting laws make any exceptions for terminated trials, and waiving reporting requirements for all terminated trials seems inappropriate considering that some trials may enrol hundreds of participants before being terminated. Going forward, a discussion regarding the appropriate ethical and legal rules for terminated trials might add value.

Late reporting constitutes research waste

Discussions about research waste in clinical trials commonly focus on weakly designed or unreported trials. We believe that late reporting of results is another important form of research waste. Delays in making clinical trial results public can slow down the development and adoption of new or improved medical interventions. Late reporting lengthens the intervals between successive trials of the same intervention, and makes it impossible to for guideline developers, systematic reviewers, meta-analysts, and other medical researchers to access and integrate recent scientific evidence. Arguably, the value added of any given clinical trial result progressively depreciates over time. The one-year registry reporting timeframe recommended by the WHO is already a legal requirement for some trials in the United States and across all European Union member states, including Germany. In Europe, the reporting timeframe for paediatric drug trials is only six months. The largest commercial sponsors and many large non-commercial sponsors now comply with these rules, demonstrating that faster reporting of results is feasible. At least one European non-commercial sponsor has voluntarily committed to reporting all trial results within 12 months, regardless of the trials’ legal status.²²

Regulatory gaps in Germany

There is currently no legal or regulatory requirement to register or make public the results of around half of all clinical trials run in Germany, probably over 1,000 trials per year.¹⁴ Only 33/70 (47.1%) trials in our cohort were CTIMPs. From a patient, clinician and scientific perspective, the current regulatory approach of limiting reporting requirements to only drug and certain device trials is irrational, as ‘other trials’ for the same indications can be of equal or greater clinical and scientific importance. The UK has adopted a national strategy that sets out to ensure that every clinical trial in the country is prospectively registered and rapidly makes its results public.²³ German policy makers should put into place a similar system.

Methodological innovation

Our use of a methodology standardised across multiple projects and cohorts of clinical trials²⁴ enabled us to increase the efficiency of protocol development, project management and results reporting. We urge other researchers in this field to integrate outreach to trial sponsors into their methodologies, both to improve the quality of their own data²⁵ and to drive improvements in actual reporting performance. Our small project may yet lead to the publication of trial data for 133 women with metastatic breast cancer that would otherwise have been lost forever.

Strengths

This study provides the first overview of the publication status and speed of clinical trials involving German women with metastatic breast cancer. The study was pre-registered and the underlying materials and datasets made public. Data quality was safeguarded through two independent rounds of structured literature searches combined with direct outreach to sponsors that generated a 100% response rate. We verified the reporting status of every unreported trial in our cohort with responsible parties.

Limitations

Our cohort is limited to 70 trials. Our methodology was unable to capture trials that were not registered on one of the three registries commonly used by German researchers. Trial recruitment figures are taken from registry data, which aggregates recruitment numbers over multiple countries and in some cases may be inaccurate. We may have failed to locate the earliest publication for a minority of trials whose outcomes were reported more than once.