Keywords
research and development, product development, non-commercial R&D, commercial R&D, neglected diseases
This article is included in the TDR gateway.
This article is included in the Neglected Tropical Diseases collection.
research and development, product development, non-commercial R&D, commercial R&D, neglected diseases
In response to comments received, we made a number of edits to version 1 of the manuscript, in summary:
We added more information about our parameter of comparison and how the averages used in the P2I Model compare to other estimates available in the literature (new Table 1).
We added additional information about neglected diseases and product development partnerships (PDPs).
We added a definition of timeline and attrition rates, and of NCE Simple and NCE complex.
We added additional references, including a groundbreaking study conducted by Moran et al. in 2005 about non-commercial pharmaceutical R&D.
We added a table summarizing our quantitative data on costs and timelines (new Table 5).
We clarified that the data regarding costs refers to direct costs per candidate per phase.
We reviewed the conclusion regarding attrition rates, as the collected data was not suitable for hypothesis generation, and added additional discussion on the implications for use of the P2I Model and analysis of overall efficiency of non-commercial R&D.
We revised the abstract to reflect the changes made in the text.
See the authors' detailed response to the review by Dzintars Gotham
See the authors' detailed response to the review by Lia Hasenclever
See the authors' detailed response to the review by Claudia Vaca González
The costs and efficiency of biomedical research and development (R&D) have long been of interest to scholars, practitioners and policymakers alike. These questions have recently gained increased salience in light of concerns about the potentially declining productivity of commercial R&D; missing technologies such as products for neglected diseases of poverty, antibiotics or outbreak-prone pathogens; and the high and rising prices of medicines such as those for cancers or rare diseases1. Improved understanding of the biomedical R&D process is essential to address these societal challenges.
Relatedly, the question has arisen as to whether different approaches to organizing, financing or incentivizing R&D – sometimes referred to as “alternative” or “new” business models of R&D – can address some of the shortcomings of the traditional approach. One area where there has been significant experimentation in alternative business models is neglected diseases (ND) that predominantly affect people in low- and middle-income countries (LMICs). It has long been recognized that commercial R&D models did not and would not generate innovative technologies for these diseases because the market incentive is inadequate to do so2. In addition to (usually early-stage) research taking place in academic or public institutes, later-stage product development for NDs has received increased funding and attention through the creation of about two dozen product development partnerships (PDPs) to spur R&D into medicines for neglected diseases, such as malaria or sleeping sickness3. While there is significant variation in how they operate, a PDP is usually a non-profit organization with a separate and distinct legal identity that enables collaboration to advance the R&D of drugs, vaccines, diagnostics and other health technologies directed at unmet health needs. PDPs are generally funded by public and philanthropic contributions, which allows R&D to focus on health rather than market outcomes. PDPs usually bring together academic, government, industry and philanthropic actors to jointly develop new health technologies. Often, PDPs operate as “system integrators“ that coordinate several partners who perform R&D activities3.
With at least two decades of significant non-commercial ND R&D efforts behind us, it is an opportune moment to examine more closely how they compare to traditional commercial R&D on costs and efficiency. Several studies of specific non-commercial R&D initiatives have been published4–9, but we did not find any recent research examining costs, timelines or attrition rates across more than one initiative (a ground breaking study was conducted by Moran et al. in 200510, but with a very small dataset given that these organizations were only a few years old at the time).
This study sought to contribute to the knowledge base by gathering evidence on the costs, timelines and attrition rates of non-commercial ND R&D initiatives and analyse how they compared to estimates of commercial R&D. In general, timeline is defined as the time spent to develop a given product and attrition rate is defined as the proportion of projects that did not pass to the subsequent phase out of the total number of projects that entered a particular phase (a wide range of terms is used in the literature to refer to this concept, including “failure rate” and the converse concept of success rate, phase transition rate, approval rate, likelihood of approval, probability of success). Furthermore, we define “non-commercial” R&D as that undertaken primarily with a not-for-profit purpose. Often, the lead organizations of such initiatives are academic or governmental in nature, or non-profit PDPs. For-profit firms frequently play a collaborating role by providing access to compound libraries, technical expertise, and/or products for use in testing, among other in-kind contributions. However, these initiatives are not part of the firm’s core commercial portfolio or strategy, as they are not expected to generate significant (if any) market returns. Therefore “non-commercial” should not be interpreted as excluding the private sector. Furthermore, we prefer to use the broader term “non-commercial” rather than “non-profit” as in some cases a developer may earn profit or revenue on a product, but that is not the main purpose for their activities.
The Portfolio-to-Impact (P2I) tool was used as a parameter of comparison in the study. P2I is a modelling tool initially developed by TDR (Special Programme for Research and Training in Tropical Diseases) co-sponsored by the United Nations Children’s Fund (UNICEF), the United Nations Development Programme (UNDP), the World Bank and World Health Organization (WHO), to estimate funding needs to accelerate health product development from late stage preclinical study to phase III clinical trials, and to model potential product launches over time. The tool is based on averages of costs, length of phase and probability of success derived from historical data on health product development for all diseases and all product developers. A description of the tool is available elsewhere11. This study was undertaken as part of a TDR-led consortium of organizations that conducted further analysis of the P2I model throughout 2019.
The study was designed to collect: 1) quantitative data from non-commercial R&D initiatives on costs, timelines and attrition rates, and 2) qualitative data from non-commercial R&D initiatives and/or experts on such initiatives to explain costs, timelines and attrition rates and reasons why these might or might not differ from commercial R&D. Data was collected between June and September 2019. Participants were given the opportunity to review and comment on a first draft of the research report but were not allowed to withdraw their data (a copy of the consent form is available with the full research report as extended data; Annex 4, pp. 91–9412). All data has been aggregated and anonymized.
The study population was selected using the database of pipeline technologies for neglected diseases developed by Duke University and Policy Cures Research13. The database does not include all non-commercial R&D initiatives – for example, it excludes biodefense projects that are largely publicly-funded – but it is the most comprehensive database of which we are aware focusing on R&D for neglected diseases, which is by nature largely non-commercial. We used a version of the spreadsheet “Candidates in the pipeline for neglected diseases, as of August 31, 2017” sent to us by the authors, which included a categorization of the organizations by developer type. We selected all not-for-profit organizations that were directly involved in conducting R&D, which included product development partnerships (PDPs), academic and research institutions and public research institutes and other public sector organizations. There were a total of 443 candidate products and 285 organizations that fit this initial criterion. All 16 PDPs were included given their organizational focus on non-commercial R&D, and the list of PDPs was complemented by other studies3,14. In addition to PDPs, we included organizations that had at least one product that had reached Phase 3. Another 32 organizations were included after this second selection, and three additional organizations were included through snowball sampling. The final list of organizations is available in the full research report (extended data; Annex 1, pp. 77–7812). We contacted each organization by email, with the initial request addressed to the organization’s most senior executive (e.g. Chief Executive Officer, Executive Director, Managing Director) to ensure leadership was aware of and agreed to our interview request, as recommended by the ethical review process. In specific cases, where we had reason to know another employee would be relevant to or aware of our research project, we copied other individuals on the initial email. The senior executive often delegated the interview to one or more staff, such as the lead staff person responsible for R&D, finance, policy and/or external relations.
We developed a questionnaire using MS Excel to collect quantitative data from the organizations in our sample on the costs, timelines, and attrition rates for a given organization or project (extended data; Annex 2, pp. 79–8912). We also included quantitative data on costs, timelines or attrition rates published in reports or articles prior to the start of this study from four organizations6–9. For timelines, we consulted data available on the organizations’ websites and in the clinical trials database ClinicalTrials.gov.
We created a quantitative dataset in Excel combining data provided by respondents with publicly available information pertaining to the costs, timelines, and attrition rates of non-commercial R&D initiatives. Data was anonymized and combined by product archetype. Due to the limitations of our dataset, we limited our analysis to only two P2I archetypes: simple and complex new chemical entities (NCE-Simple, NCE-Complex), defined as those with or without validated target or mechanism of action respectively15. Only one organization provided information about diagnostics and two about vaccines (one only included aggregated totals for one product), and we excluded these from the analysis as it would be impossible to protect the anonymity of the organizations.
Several assumptions were made to standardize the data and allow comparison (see detailed methodology in extended data; pp. 31–3212). For costs, we assumed that money was spent at a steady rate across the time period of each phase. For data in currencies other than USD, a yearly exchange rate from the year in which the cost was incurred was used to make the conversion into USD. Totals were calculated in 2017 dollars to facilitate comparison with P2I model figures. Inflation and deflation adjustments used the standard consumer price index.
For timelines, we calculated total time spent in development as the sum of time spent in Pre-clinical, Phase 1, 2 and 3. Phase 1a and 1b trials were counted as phase 1. Phase 2a, 2b, and 2c trials were counted as phase 2, as were phase 2/3 tests. Phase 3a and 3b are both counted as phase 3. An average of time per trial for each phase was taken to estimate the amount of time required for a given clinical phase. For some data points, early stage testing was aggregated for multiple drug candidates and total time spent in a phase was divided by the number of candidates. Many candidates have multiple trials in each phase and the average of all trials was used.
A list of questions for semi-structured interviews was developed to collect the qualitative data (extended data; Annex 3, p. 9012). The questions were not pilot tested. Most interviews lasted about one hour and were either conducted in person in Geneva or using videoconferencing software and were recorded upon agreement of the participant. Interviews were held with individuals with a high degree of familiarity about product development from the organizations included in the study. Interviews were held by the three authors (MV- MPH, researcher, female; RK – researcher, male; SM – PhD., director of research, female) and no one else was present besides the participants and the researchers.
Transcription or notes from the interviews were analysed and coded using NVivo 12. Interviews were coded by MV based on themes derived from the data, and reviewed by SM. A description of the coding tree is not available. Interviews were anonymized both at individual and organization level and each was given a number (PO - participant organization) for quotation identification. Given the small sample size, data saturation was not reached.
We used parameters from the Portfolio-to-Impact (P2I) tool v2, which was initially developed by TDR11 and updated by Duke University and Policy Cures Research13. The P2I Model is based on assumptions of costs, timelines and attrition rates. Assumptions on development costs at each phase were based on clinical trial costs from Parexel’s R&D cost sourcebook, derived from historical data on health product development of more than 25,000 candidates for all diseases, and further refined and validated by interviews. The underlying data used to construct those assumptions is not publicly available and it was not possible to disaggregate costs, timelines or attrition rates by commercial vs non-commercial developer. Given that non-commercial R&D (in particular, non-commercial late-stage product development) is both relatively recent and small in scale, we assume that the vast majority of the data used to construct the P2I averages comes from commercial R&D. We compared our quantitative data to P2I averages, and our qualitative data compared non-commercial with commercial R&D.
We conducted a literature review on costs, timelines and attrition rates of biomedical R&D to compare other estimates with those of the P2I Model (available at the Knowledge Portal on Innovation and Access to Medicines). There is a wide range of estimates, most focusing on the development of new chemical entities (NCEs) by pharmaceutical companies (commercial R&D). The literature on this topic has advanced considerably over the past decade, with new estimates for costs and risks based on larger, more representative datasets than earlier studies. The estimated average cost to develop a new drug ranges widely, from $43.4 million to $4.2 billion16. Similarly, estimates for timelines also vary widely between studies, and according to type of technology and indication (Kimmitt et al. 2020). Estimates available for success rate – that is the percentage of projects that entered a particular phase of development and passed to the subsequent phase – show overall success rate for the clinical stage (from Phase 1 of clinical trials to successfully ending Phase 3) ranges from 6% to 26% for new drugs, with important variation by therapeutic indication and technology type17. Differences in methodology, data sources (many confidential), rate of cost capitalization, types of expenses included as R&D (e.g., mergers and acquisitions, expenses related to marketing of the product or tax deductions), and other methodological approaches provide a wide range of estimates.
P2I averages are overall at the low end of the range of cost estimates available for commercial R&D18–21. They are similar to studies by Sertkaya et al. (2014) and Jayasundara et al. (2019), and far lower than the studies by Mester-Ferrandiz et al. (2012) and DiMasi et al. (2016). P2I timelines are higher for phase 1, lower for phase 2 and within the same range for phase 319,22–24 and P2I success rates are lower for complex NCEs and in the same range for simple NCEs22,23,25–28. In comparison to the few estimates available for non-commercial R&D initiatives6,29, the P2I Model assumptions estimate lower success rates in all clinical development phases, while preclinical is higher. A more detailed literature review is available in the full research report. This study’s findings regarding non-commercial R&D should be read in view of the particular estimates for commercial R&D that we used, which are significantly lower than some widely-cited estimates of R&D costs.
Table 1. Recent estimates of new drug development costs by phase for commercial R&D and non-commercial R&D compared to P2I averages (millions of USD). The table lists recent estimates of costs of pharmaceutical research and development (R&D) by phase, specifically for the development new drugs. It provides a comparison of estimates for commercial R&D, non-commercial R&D and a comparison with P2I averages. The table contains 8 columns containing the title of the study, the period covered, the sample size, whether costs are capitalized or not and at which rate, and the cost estimates for the stages preclinical, phase I, phase II and Phase III. Figures are in millions of USD. There are 7 lines, one for each study, with the first 4 being representative of commercial R&D, one being non-commercial R&D and the last two being the averages from the P2I Model.
Paper | Period | Sample size | Capitalized/ rate | Preclinical | Phase I | Phase II | Phase III |
---|---|---|---|---|---|---|---|
Mestre-Ferrandiz et al. (2012)23a | In clinical development between 1998-2002 | 97 projects conducted by pharmaceutical companies | No | $76.5 | $236.3 | $316.9 | $235.9 |
Yes, 11% | $207.4 | $468.1 | $501.6 | $293.8 | |||
DiMasi et al. (2016)24 | Compounds first test in human from 1995–2007 and R&D expenditures from 1990–2013 | 106 new drugs from 10 multinational biopharmaceutical companies of varying sizes | No | $430 | $25.3 | $58.6 | $255.4 |
Yes, 10.5% | $1,098 | $49.6 | $95.3 | $314 | |||
Sertkaya et al. (2014)25b | 2004–2012 | Industry-sponsored trials conducted in the US | No | - | $ 3.4 | $13.6 | $21.8 |
Jayasundara et al. (2019)26 | Drugs approved between 2000-2015 | 100 non-orphan drugs / 561 trials (only NMEs) – all sponsors (mostly industry) | No | - | $2.81 | $7.0 | $25.75 |
Yes, 10.5% | - | $5.16 | $11.11 | $34.82 | |||
100 new orphan drugs and / 602 trials (only NMEs) - all sponsors (mostly industry) | No | - | $4.27 | $20.86 | $20.02 | ||
Yes, 10.5% | - | $9.61 | $39.87 | $30.35 | |||
DNDi (2019)14c | 2003–2019 | 2 new chemical entities | No | $ 14.9 | $ 4.3 | $ 32.9 | |
P2I Model (NCE-Simple)6 | 2007–2014 | 3,655 candidates | No | $ 5.0 | $2.2 | $5.8 | $32.8 |
P2I Model (NCE-Complex)6 | 2007–2014 | 18,851 candidates | No | $ 10.0 | $7.4 | $ 6.4 | $36.1 |
Legend: White = commercial or mostly commercial; Yellow = non-commercial; Green = P2I Model averages
(a) Mestre-Ferrandiz et al. does not use the standard division of the R&D into phases, but instead use the following intervals: Interval 1. Pre-first toxicity dose, Interval 2. First toxicity dose to first human dose, Interval 3. First human dose to first patient dose, Interval 4. First patient dose to first pivotal dose, Interval 5. First pivotal dose to first core submission, Interval 6. First core submission to first core launch. For the purpose of this table, we considered interval 1 as preclinical, intervals 2 + 3 as phase I, interval 4 as phase II, interval 5 as phase III and interval 6 as approval. Total includes approval, not included in the table. (b) Original figures from Sertykaya et al. are broken down in 13 different therapeutic areas; we calculated a simple average to include all drugs. (c) From DNDi paper, we included a simple average of the costs provided for two new chemical entities. Preclinical includes discovery costs. Costs for phase II and III are combined and include registration costs. Total costs are only for two candidates and does not include cost of failures. Original figures are in euros; exchange rate: 1.12 USD/EUR.
We contacted a total of 48 organizations: 23 did not respond, 12 declined and 13 participated in some way (not all participating organizations provided both quantitative and qualitative data) - a participation rate of 27%. In total, we obtained quantitative data regarding 8 organizations and 83 products - 37 drug candidates (13 NCEs, 8 repurposed drugs and 16 not specified), as well as 19 vaccine and 27 diagnostic candidates. Qualitative data was obtained from 14 interviews with 20 individuals from 12 organizations; out of those, 18 individuals provided their perspective based on projects conducted within their own organizations and two were experts with knowledge of a range of organizations.
Quantitative data on non-commercial R&D costs were largely in line with the P2I model estimates, with some variation by phase. For simple NCEs, total costs for non-commercial R&D were 13% higher than the P2I estimates (51.87 million USD for non-commercial vs 45.84 million USD for P2I) (Figure 1). The largest differences were in pre-clinical and phase 1 – where the costs in our sample were more than double the P2I model estimates. Conversely, phase 2 and 3 trials were less expensive for simple NCEs in our data, but by a small margin. The sample size is too small for statistical significance or to generalize to non-commercial R&D more broadly; rather, the findings merely suggest a hypothesis that costs per candidate per phase to develop simple NCEs are similar between non-commercial R&D initiatives and P2I averages.
The figure compares the development costs collected from the organizations in the study and the averages in the P2I Model for the archetype “new chemical entities – simple”. The comparison is made for different stages of development preclinical, phase 1, phase 2 and phase 3 and total costs. Data shows higher costs for collected data at preclinical, phase 1 and total, and lower costs at phase 2 and phase 3. Collected data for preclinical is based on 4 data points (n=4), for phase 1 n=3, for phase 2 n=3 and for phase 3 n=2.
For complex NCEs, total costs were similar to the P2I model, 8% lower (53.98 million USD for non-commercial vs 58.93 million USD in P2I) (Figure 2). In contrast with simple NCEs, for complex NCEs non-commercial preclinical and phase 1 costs were lower than the P2I model. Notably, phase 2 costs were much higher in our dataset (12.65 million USD vs 6.39 million USD in P2I). This could be in part because of the high proportion of phase 2/3 trials in the dataset, as well as the ratio of phase 2b to 2a tests being higher than the P2I data. Phase 3 costs were substantially lower than the P2I estimates, which may be explained by the fact that many pivotal trials were in phase 2. The opportunity to forgo phase 3 testing would drive up phase 2 costs while lowering phase 3 costs. The proportion of pivotal phase 2 tests may be different between P2I and our dataset. As with simple NCEs, the findings merely suggest a hypothesis that should be tested against a larger dataset – that costs per candidate per phase to develop complex NCEs are similar between non-commercial R&D initiatives and P2I averages.
The figure compares the development costs collected from the organizations in the study and the averages in the P2I Model for the archetype “new chemical entities – complex”. The comparison is made for different stages of development preclinical, phase 1, phase 2 and phase 3 and total costs. Data shows lower costs for collected data at preclinical, phase 1, phase 3 and total, and higher costs at phase 2. Collected data for preclinical is based on 8 data points (n=8), for phase 1 n=6, for phase 2 n=4 and for phase 3 n=4.
To assess how sensitive our results were to coding by archetypes (i.e. characterizing a product as “simple” or “complex” NCEs), we combined our data from both categories and found that total costs (48.9m USD) lay between P2I’s NCE-simple and NCE-complex estimates (45.8m-59.9m USD) (Figure 3). This sensitivity analysis suggests that total non-commercial costs are largely in line with P2I parameters, even if there are some differences in coding of archetypes.
The figure compares the development costs collected from the organizations in the study combining the archetypes “new chemical entities – simple” and “new chemical entities – complex” combined and the averages in the P2I Model for the archetypes “drug repurposed complex”, “new chemical entities – simple” and “new chemical entities – complex”. The comparison is made for different stages of development preclinical, phase 1, phase 2 and phase 3 and total costs.
The qualitative data identified 12 factors that drove costs up or down in the different phases of product development within non-commercial R&D initiatives12 (Table 2). Most responses focused on the clinical stages of development rather than pre-clinical or earlier. Three factors pushed costs upward, and five factors pushed costs downward for non-commercial R&D in comparison with commercial. Four factors were categorized as indeterminate as they would affect both non-commercial and commercial R&D in the same way. The table below presents a summary of the factors influencing costs. A description of the factors and sample quotes are available in the full research report.
The table lists the factors influencing costs for non-commercial research and development (R&D) in relation to commercial R&D. Factors are classified in three columns: “costs pushed upward”, “indeterminate” and “costs pushed downward”. There are three factors listed for pushing costs upwards, four factors as indeterminate and five factors pushing costs downward.
There were more factors that would push costs for non-commercial R&D down vis-à-vis commercial models. However, as the qualitative data does not tell us about the magnitude of the effects, no conclusions can be drawn based on these qualitative data alone on whether non-commercial R&D would generally cost the same, less or more than commercial R&D. Rather, they provide these factors provide potential explanations that merit further exploration
The quantitative data showed that for simple NCEs, timelines were roughly similar between non-commercial R&D and P2I averages. Non-commercial R&D had shorter preclinical times (1.65 years vs 2.49 years in P2I), and longer phase 1 times (2.61 vs 1.80 years in P2I). Non-commercial R&D also had much shorter phase 2 times (1.75 vs 3.38 years in P2I), while phase 3 times were slightly higher (3.67 vs 3.18 years in P2I). Overall, our dataset suggested modestly faster timelines for non-commercial simple NCE development, taking 9.67 years vs. 10.85 years in the P2I model (Figure 4).
The figure compares the development times collected from the organizations in the study and the averages in the P2I Model for the archetype “new chemical entities – simple”. The comparison is made for different stages of development preclinical, phase 1, phase 2 and phase 3 and total costs. Data shows shorter times for collected data at preclinical, phase 2 and total, and longer times at phase 1 and phase 3. Collected data for preclinical is based on 2 data points (n=2), for phase 1 n=3, for phase 2 n=3 and for phase 3 n=2.
For complex NCEs, non-commercial pre-clinical testing was much shorter (1.00 vs 2.87 years in P2I), phase 1 testing slightly shorter (1.67 vs 1.93 years in P2I), phase 2 longer (4.25 vs 3.51 years in P2I), and phase 3 longer (4.0 vs 2.8 years in P2I) (Figure 5). Overall, non-commercial development time was nearly identical, at 10.92 compared to 11.11 years for the P2I model. Possible explanations for these differences could be a result of the ratio of phase 2a to phase 2b tests included in phase 2. The P2I model does not suggest what proportion of its phase 2 tests are 2a compared to 2b, but for our data set, there were many phase 2b tests, which may have increased the amount of time in this phase. Pre-clinical time may have been shorter due to our decision to divide the time in this stage of development among multiple candidates, as that is how some data was shared with us. It is unclear how many preclinical studies in the P2I dataset would have been calculated in this way.
The figure compares the development times collected from the organizations in the study and the averages in the P2I Model for the archetype “new chemical entities – complex”. The comparison is made for different stages of development preclinical, phase 1, phase 2 and phase 3 and total costs. Data shows shorter times for collected data at preclinical, phase 1 and total, and longer times at phase 2 and phase 3. Collected data for preclinical is based on 1 data point (n=1), for phase 1 n=6, for phase 2 n=4 and for phase 3 n=1.
The qualitative data identified 12 factors influencing timelines for non-commercial R&D (Table 3). As with costs, the identified factors were categorized by their potential to push timelines up or down for non-commercial R&D in comparison to commercial R&D. Seven factors were likely to lengthen timelines for non-commercial R&D, no factors were likely to shorten timelines and five factors were categorized as indeterminate. The table below presents a summary of the factors influencing timelines. A longer description of the factors and sample quotes are available in the full research report12.
The table lists the factors influencing timelines for non-commercial research and development (R&D) in relation to commercial R&D. Factors are classified in three columns: “timelinesf longer”, “indeterminate” and “timelines shorter”. There are seven factors listed for longer timelines, five factors as indeterminate and no factors for shorter timelines.
There were a number of factors that would lengthen timelines for non-commercial R&D vis-à-vis commercial models, or that were indeterminate (Table 3). Notably, in none of the interviews did a respondent argue that non-commercial R&D would move faster. As the qualitative data does not tell us about the magnitude of the effect, no firm conclusions can be drawn on whether non-commercial R&D would take generally the same amount of time or more than commercial R&D. We note that Moran (2005) found that pure public-sector R&D took longer than industry averages, while PDP R&D progressed at a similar pace.
The quantitative data on attrition rates was the most difficult to obtain, and there did not appear to be a standard methodology nor practice of calculating such rates within participating organizations. As all non-commercial initiatives in our sample had relatively small portfolios (compared to large commercial firms), attrition rates might not be meaningful as just one or two candidate successes or failures could lead to large swings in attrition rates. Some organizations provided quantitative data only for costs and timelines of specific products by phase, and not for their overall portfolio, and did not include any information on attrition rates. We judged that the data we received could not be aggregated across organizations as there was limited information for each type of product, nor was it adequate for hypothesis generation. Further research is needed in this area.
Interviewees were also asked about the main factors that drive attrition rates up or down in the different phases of product development. This question generated a wide range of responses, and different organizations took quite different approaches to conceptualizing – let alone calculating – attrition rates. It should be noted, however, that most interviewees expressed the view that P2I averages for attrition rates seemed to be in line with their experience conducting project development in their organizations. There was also reasonable disagreement as to whether or when a higher attrition rate is undesirable. Some interviewees argued that it is beneficial for an organization to “fail early and fail fast” – that is, to have a high(er) attrition rate in pre-clinical or Phase 1. Too low of an attrition rate could also suggest an organization is not taking enough risk, particularly in the earlier and lower-cost phases of R&D.
The qualitative data identified nine factors influencing attrition rates for non-commercial R&D (Table 4). As with costs and timelines, the identified factors were categorized as likely to drive attrition rates higher or lower for non-commercial R&D in comparison to commercial R&D. Three factors were identified as pushing attrition rates higher for non-commercial R&D, one factor as pushing attrition rates lower and five factors were categorized as indeterminate. The table below presents a summary of the factors influencing attrition rates. A longer description of the factors and sample quotes are available in the full research report12.
The table lists the factors influencing attrition rates for non-commercial research and development (R&D) in relation to commercial R&D. Factors are classified in three columns: “attrition rate higher”, “indeterminate” and “attrition rate lower”. There are three factors listed for higher attrition rates, five factors as indeterminate and one factor for lower attrition rates.
There were more factors that would raise attrition rates for non-commercial R&D vis-à-vis commercial models, than would lower them, but most of the factors raised by respondents were indeterminate. As the qualitative data does not tell us about the magnitude of the effect, no conclusions can be drawn on whether non-commercial R&D would be characterized by higher, lower or equivalent attrition rates as commercial R&D.
Table 5. Summary of qualitative data. The table provides a summary of the qualitative data for costs (in millions of USD) and timeframe (in years), per phase of development (preclinical, phase I, phase II, phase III, approval and total). Data for costs are provided for NCE Simple, NCE Complex and NCE combined. Data for timeframe is provided for NCE Simple and NCE Complex, and includes only products with no ongoing clinical trials at the time of data collection.
The quantitative and qualitative data combined paint a complex, if grainy, picture. Keeping in mind the very small sample of quantitative data, the following hypotheses emerge from the analysis.
Regarding costs, the quantitative data suggest that non-commercial R&D total costs are about the same overall as P2I averages for NCEs. It should be noted that in comparison to other estimates available in the literature for commercial R&D, P2I averages are on the low end of the spectrum. The qualitative data identified many more reasons why non-commercial costs would be lower than commercial R&D, but did not shed light on the magnitude of the effects. The overall emerging hypothesis is that total direct costs per candidate per phase of non-commercial R&D are expected to be equivalent or somewhat lower than commercial. Indirect costs for commercial R&D are expected to be higher due to higher overhead and capitalization costs.
Regarding timelines, the quantitative data suggest that non-commercial R&D timelines would be slightly shorter for simple NCEs and equivalent for complex NCEs in comparison to P2I averages. Yet the qualitative data identified many more reasons why non-commercial timelines would be longer than commercial; the data did not shed light on the magnitude of the effects. The overall emerging hypothesis is that timelines of non-commercial R&D are expected to be equivalent to commercial.
Regarding attrition rates, the quantitative data was not adequate for analysis. The qualitative data uncovered more reasons why attrition rates might be higher in non-commercial R&D, but also provided a number of reasons why they might be lower or there might be no difference. Again, the magnitude of the effects is not quantified. The data on attrition rate is not adequate for hypothesis generation.
If non-commercial R&D is characterized by equivalent or lower costs, and equivalent timelines to commercial R&D, the final expected direct costs and time to develop products resulting from a pipeline of non-commercially developed candidate technologies would be equivalent to those resulting from commercial R&D. However, given the lack of suitable data regarding attrition rates for non-commercial R&D, we cannot assess the number of products estimated to be developed from a portfolio of candidates – that is, the cost for a non-commercial R&D model to develop one successful product including the cost of failures. One organization (DNDi) has estimated their costs, including failures, to develop an NCE at 60–190 million EUR5, based on industry averages for attrition rates in anti-infectives29. Attrition rate is a key variable required to calculate the magnitude of the cost of failures – and therefore the overall costs per successful product of a non-commercial model – rather than just the cost per candidate per phase. However, we were unable to find adequate data for this variable. If attrition rates for non-commercial R&D are higher than P2I estimates, the number of products predicted by the P2I model would be lower, and vice versa.
This study also identified a number of significant differences between non-commercial and commercial R&D. The many variables that affect cost, timelines and attrition rates also highlight that caution is merited when comparing any single trial, product or organization against average benchmarks, as there are many legitimate reasons for departure from the mean. Therefore, the P2I model may need to be modified when applied more narrowly. While differences may get averaged out when the model is applied to a pipeline of nearly 450 candidates across a broad range of diseases (the intended use of the P2I model), they may be magnified in the narrower context of a single disease, technology type, or organization.
Finally, we re-emphasize that the small size and heterogeneity of the dataset means that these are tentative conclusions. Further quantitative research is needed to test these hypotheses against larger datasets. And further qualitative research is needed to deepen our understanding of the strengths and weaknesses of non-commercial R&D initiatives, and how well they function as alternatives to the traditional commercial model, especially beyond neglected diseases where commercial interests are higher.
This was an observational, descriptive and analytic study of non-commercial R&D initiatives. The main limitations of the study were the small non-random sample size and the short period of time in which the study was conducted, which can partially explain the limited amount of quantitative data received. We also recognize that respondents may have had incentives to report costs, timelines or attrition rates that were favourable to their organizations. Although we sought to check quantitative data against publicly available sources, in general very little relevant data was in the public domain or it was only available at a high level of aggregation. As a result, we have sought to be cautious in drawing inferences from the data.
Given the nascent nature of the area, with almost no prior literature focusing on costs, timelines or attrition rates of non-commercial R&D initiatives, we see the merits of this study as generating hypotheses for further testing against a larger sample of quantitative data, and for providing intuition regarding reasons underlying any significant differences between non-commercial and commercial initiatives. The emerging hypothesis is that non-commercial R&D is comparable to commercial initiatives in direct costs per candidate per phase and in timelines. The limited information regarding attrition rates limits any hypothesis generation regarding overall efficiency, however.
It is also important to highlight that many non-commercial R&D initiatives arose because the commercial model did not meet important global public health needs. This study did not compare the patient, population-level, equity or health system benefits offered by the products emerging from non-commercial vs commercial initiatives – only the costs, timelines and attrition rates to develop those products. A fuller comparison could take both into account.
For future research, it may be useful both to expand the dataset on NCEs and also dedicate special attention to improving our understanding of non-commercial vaccine and diagnostics R&D, recalling that we excluded vaccines and diagnostics from our quantitative analysis due to very limited data, and were only able to examine a small sample for simple and complex NCEs.
Finally, in future research it would be useful to interview a broader range of stakeholders. Our interviews focused on practitioners with direct knowledge of non-commercial R&D initiatives involved in product development for neglected diseases, usually employees of the initiatives themselves. A more thorough picture is likely to emerge through interviews with additional non-commercial initiatives, and a broader range of their partners and funders.
Zenodo: Quantitative data: costs and timeframes_ non-commercial pharmaceutical R&D. https://doi.org/10.5281/zenodo.451970915.
This project contains the following underlying data:
The qualitative data is confidential to protect participant confidentiality as required by the Ethics Review Committee given the small number of organizations active in the field. Selected quotes from the interviews are available in the full research report at the Graduate Institute Institutional Repository at https://repository.graduateinstitute.ch/record/29883412.
The Graduate Institute Geneva Institutional Repository: Do costs, timeframes and attrition rates differ between non-commercial and commercial biomedical R&D ? A study of neglected diseases R&D and the P2I model. https://repository.graduateinstitute.ch/record/29883412.
This report contains the following extended data:
- Consent form (Annex 4, pp. 91–94)
- The final list of organizations (Annex 1, pp. 77–78)
- Questionnaire for quantitative data (Annex 2, pp. 79–89)
- Assumptions made to standardize the data and allow comparison (pp. 31–32)
Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).
Marcela Vieira - Conceptualization, Funding Acquisition, Methodology, Investigation, Data Collection, Data Curation, Formal Analysis, Writing – Original Draft Preparation
Ryan Kimmitt - Investigation, Data Collection, Data Curation, Visualization, Formal Analysis, Writing – Review & Editing
Suerie Moon - Conceptualization, Funding Acquisition, Methodology, Investigation, Data Collection, Formal Analysis, Supervision, Validation, Project Administration, Writing – Review & Editing
We thank Temmy Sunyoto, post-doctoral researcher at the Global Health Centre, for revision and comments that greatly improved the manuscript.
Views | Downloads | |
---|---|---|
F1000Research | - | - |
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Competing Interests: No competing interests were disclosed.
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Pharmaceutical policy, health economics, global health
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Partly
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Not applicable
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Yes
Competing Interests: No competing interests were disclosed.
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Not applicable
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Partly
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Pharmacoepidemiology, regulatory issues and access-to- medicines public policy-making processes.
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Yes
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Partly
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Pharmaceutical policy, health economics, global health
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | |||
---|---|---|---|
1 | 2 | 3 | |
Version 2 (revision) 16 Aug 21 |
read | read | |
Version 1 08 Mar 21 |
read | read | read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Already registered? Sign in
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (1)