Analysis of the health product pipeline for poverty-related and neglected diseases using the Portfolio-to-Impact (P2I) modeling tool

Background: To estimate how much additional funding is needed for poverty-related and neglected disease (PRND) product development and to target new resources effectively, policymakers need updated information on the development pipeline and estimated costs to fill pipeline gaps. Methods: We previously conducted a pipeline review to identify candidates for 35 neglected diseases as of August 31, 2017 (“2017 pipeline”). We used the Portfolio-to-Impact (P2I) tool to estimate costs to move these candidates through the pipeline, likely launches, and additional costs to develop “missing products.” We repeated this analysis, reviewing the pipeline to August 31, 2019 to get a time trend. We made a direct comparison based on the same 35 diseases (“2019 direct comparison pipeline”), then a comparison based on an expanded list of 45 diseases (“2019 complete pipeline”). Results: In the 2017 pipeline, 538 product candidates met inclusion criteria for input into the model; it would cost $16.3 billion (B) to move these through the pipeline, yielding 128 launches. In the 2019 direct comparison pipeline, we identified 690 candidates, an increase of 152 candidates from 2017; the largest increase was for Ebola. The direct comparison 2019 pipeline yields 196 launches, costing $19.9B. In the 2019 complete pipeline, there were 754 candidates, an increase of 216 candidates from 2017, of which 152 reflected pipeline changes and 64 reflected changes in scope. The complete pipeline 2019 yields 207 launches, costing $21.0B. There would still be 16 “missing products” based on the complete 2019 pipeline; it would cost $5.5B-$14.2B (depending on product complexity) to develop these products. Conclusion: The PRNDs product development pipeline has grown by over a quarter in two years. The number of expected new product launches based on the 2019 pipeline increased by half compared to 2017; the cost of advancing the pipeline increased by a quarter.


Introduction
A growing body of evidence has shown that the ambitious global health goals established by the international health community, including the health-related Sustainable Development Goals (SDGs), are unlikely to be achieved with today's health technologies alone [1][2][3][4] . New breakthrough technologies will be needed to accelerate the current rate of mortality reduction.
For example, the 2013 report of the Lancet Commission on Investing in Health (CIH) showed that the development and widespread deployment of new technologies would be needed to achieve a "grand convergence" in global health 1 . The CIH defined grand convergence as a universal reduction in deaths from infections and maternal and child health conditions down to the mortality levels achieved in 2011 by the bestperforming middle-income countries (e.g. China and Chile) 1 . Unfortunately, progress on mortality from several conditions has been slow since this report was published. If the global trends in maternal mortality and mortality from tuberculosis (TB) achieved in 2010-16 were to continue, the convergence targets would not be achieved until 2067 for maternal mortality and 2074 for TB mortality, respectively 4 . This slow progress underscores the need for new medicines, vaccines, diagnostics, and other disease control tools.
There are several signs suggesting that policymakers at global and national levels do recognize the importance of funding product development for poverty-related and neglected diseases. First, funding levels for health research and development (R&D) were included in the indicators that are linked to the SDG health targets. For example, SDG Indicator 3.b.2 includes "official development assistance (ODA) for medical research and basic health sectors as a % of gross national income (GNI) and as a % of all ODA, by donor country." SDG Indicator 9.5.1 is "gross domestic R&D expenditure on health (health GERD) as a % of gross domestic product (GDP)." Second, the new Global Action Plan for Healthy Lives and Wellbeing for All, which commits 12 multilateral health and development organizations to improving their collaboration on achieving the SDGs, includes an effort to scale up health R&D. Third, several new initiatives have been launched in recent years to mobilize additional funding for global health product development, such as the Global Health Innovative Technology Fund and the Coalition for Epidemic Preparedness Innovations 5 . Fourth, a number of new global health R&D coordination mechanisms have also been launched, such as the Global Antimicrobial Resistance R&D Hub and the World Health Organization's Global Observatory on Health R&D 5 .
To estimate how much additional funding is needed for global health product development and to target any new resources effectively, policymakers need up-to-date information on the status of the development pipeline and the estimated costs to fill gaps in the pipeline. In particular, as we have previously argued, they need information on "which candidates are currently in the pipeline and at what development phase; the estimated costs to accelerate this portfolio of candidates to production; the anticipated product launches that would result from such acceleration; and the critical, highly needed products that would still be "missing" under the status quo." 6 In order to generate such information, in 2018 we published the first ever study to use a new financial modeling tool, called the Portfolio-to-Impact (P2I) tool, that can be used to estimate the costs of moving a portfolio of candidates along the pipeline 6 . In that 2018 study, we first conducted a comprehensive, rigorous pipeline portfolio review to identify product candidates for 35 neglected diseases as of August 31, 2017. We found 538 candidates that met inclusion criteria for input into the modeling tool. Using this tool, we estimated that it would cost about $16.3 billion (range $13.4B-19.8B) to move these 538 candidates through the pipeline, with three-quarters of the costs incurred in the first 5 years, resulting in about 128 (89-160) expected product launches by 2029.
Among these launches, our modeling suggested that there would be few launches of complex new chemical entities (NCEs). We also found that, based on the pipeline as of August 31, 2017, launches of highly efficacious HIV, TB, or malaria vaccines would be unlikely. We estimated that the additional costs to launch one of each of 18 priority "missing" products would be $13.6B, assuming lowest product complexity or $21.8B assuming highest complexity ($8.1B-36.6B).
In this current study, we have conducted a new pipeline portfolio review and we have updated the cost modeling. There are several reasons why this updated study was needed. First, the product development pipeline is never static-it changes frequently-and so we wanted to gain an understanding of how the pipeline has changed. We conducted a new pipeline review up to August 31, 2019, i.e. exactly two years on from our last study, allowing us to show how much change can be expected over such a two-year period. Second, we wanted to assess whether the prospects have improved for launching critically needed products, such as vaccines for HIV, TB, and malaria. Third, our new study was timely given the launch of a new target product profile (TPP) directory developed by the Special Programme for Research and Training in Tropical Diseases (TDR). We hoped that we could potentially compare our new findings on product gaps with the gaps identified in the new TPP directory. Finally, we wanted the new findings to help identify the most promising ways in which the pipeline can be better managed to accelerate the development and deployment of critically needed health technologies for poverty-related and neglected diseases.

Methods
In this section, we begin, in sub-section (i), by summarizing the development of the P2I tool. While the tool development has previously been published in detail 6,7 , we wanted to summarize the key steps so that readers could understand the underlying principles of-and the assumptions and data sources underlying-the tool. In sub-section (ii), we describe how we conducted the pipeline portfolio review and in (iii) we explain how we classified each candidate into its product type (which we call "archetype", e.g. simple NCE, complex NCE). In sub-section (iv) we describe how we inputted the pipeline of candidates into the P2I model. In sub-section (v), we explain how we estimated the costs of priority "missing" products. Finally, in sub-section (vi), we describe our sensitivity analysis.
(i) Summary of how the P2I.v2 model was developed The P2I model is a deterministic modeling tool that enables users to estimate funding needs to move a portfolio of candidate health products through the pipeline from advanced preclinical to phase III clinical trials, as well as potential product launches over time. The first version of the P2I tool, called P2I version 1 (P2I.v1) is a Microsoft Excel-based tool that is freely available online 7 . In the first study to apply this tool to the pipeline of candidate products for neglected diseases, we made a number of adaptations to P2I.v1, and called the adapted version P2I version 2 (P2I.v2); this adapted version is also an open access Microsoft Excel-based tool 6 . Both the P2I.v1 and P2I.v2 models are based on assumptions for costs, attrition rates (probability of success), and cycle times per phase for four phases of research-advanced preclinical to phase III-for different product types (archetypes). Figure 1 shows a conceptual overview of the P2I model, and Figure 2 shows which phases are included.
The P2I.v1 model has 11 archetypes: simple or complex vaccines; simple or complex NCEs; simple, innovative, or complex repurposed drugs; simple or complex biologics; and two diagnostic archetypes-assay development or simple technical platform development. We made a number of adaptations in the development of P2I.v2, including the addition of "unprecedented vaccines" (for HIV, TB, and malaria) and vector control products. Table 1 describes the 14 archetypes in the P2I.v2 model, which is the version we have used in this new study.
For each of the 14 different archetypes in P2I.v2, the model has assumptions on costs, attrition rates, and cycle time per phase. As described in detail in Terry et al, assumptions on development costs for each phase of product development for the archetypes included in the P2I.v1 model were initially based on an analysis of clinical trial costs from Parexel's R&D cost sourcebook 7 . The assumptions on attrition rates and cycle times for each phase were initially based on the historical attrition rates and cycle times of more than 25,000 development candidates. All of the assumptions were further refined and validated based on (i) academic literature, (ii) industry publications and databases, and (iii) 133 stakeholder interviews with a wide variety of product development partnerships (PDPs), pharmaceutical companies, and major funders of global health R&D 7 . As   Any drug that has already been approved for market use in humans and still requires clinical trials to verify safety in humans.

New chemical entity (NCE)
Simple Validated target/ mechanism of action primaquine Any chemically synthesized drug that is part of a well-established class of drugs or has a mechanism of action that has already been approved for market use in humans. It is not the first of its class to be approved.
Simple -TB Validated target/ mechanism of action Sutezolid (PNU-100480) Any candidate that meets criteria for a simple NCE and is developed for TB.

Complex
Novel target/ mechanism of action without understanding of disease pathogenesis imatinib Any chemically synthesized drug that is first in its class as determined by having a novel target/mechanism of action regardless of the drug's indication.

Biologics
Simple Validated target/ mechanism of action human monoclonal antibody m102.4 Any drug that is synthesized from a living organism that is part of a wellestablished class of drugs or has a mechanism of action that has already been approved for market use in humans. It is not the first of its class to be approved.
Simple -TB Validated target/ mechanism of action CPZEN-45 Any candidate that meets criteria for a simple biologic and is developed for TB.

Complex
Novel target/ mechanism of action polyclonal IgG antibodies Any drug that is synthesized from a living organism that is first in its class as determined by having a novel target/ mechanism of action regardless of the drug's indication.
described in detail in reference 6, additional sources of assumptions for the new archetypes in P2I.v2, such as vector control products and unprecedented vaccines, were the McKinsey Risk-Adjusted Portfolio Model and clinical trial data shared by the Bill & Melinda Gates Foundation 6 .
In order to use the model to analyse the pipeline of product candidates for poverty-related and neglected diseases, each candidate must first be classified into its archetype and phase, and then these candidates can be inputted into the model. There are two main model outputs. The first is "launches"; in this paper, the term launch refers to a candidate making it through phase III and thus being ready for the next steps, e.g. the regulatory and manufacturing steps. The second is the total costs to move all candidates through the pipeline from their current phase from now to 2031 (the model also gives a breakdown of these total costs into annual costs by year, from 2019 to 2031).
The model only includes candidates in advanced preclinical to phase III research, and thus the cost estimates are an under-estimate of the full costs of product development.
In particular, the model excludes all costs related to basic research through lead optimization; chemistry, manufacturing, and controls; good manufacturing practice; manufacturing, build-up, and scale-up costs; regulatory or registration fees (post-phase III); and all post-market commitments (e.g., phase IV pharmacovigilance studies).
(ii) Pipeline portfolio review We undertook a global review of candidate products for neglected diseases, Ebola and sexual & reproductive health issues in order to capture a snapshot of the R&D pipeline as of August 31, 2019 ("2019 review"); this followed a previously published review 6 of the R&D pipeline as of August 31, 2017 ("2017 review").
The scope of candidate inclusion aligned with the G-FINDER survey on global funding for R&D for neglected diseases, emerging infectious diseases and sexual & reproductive health issues, conducted by Policy Cures Research. The G-FINDER scope is based on three key principles: that the disease or health issue disproportionately affects low-and middle-income countries; that there is a need for new products (i.e. there is no existing product, or improved or additional products are needed); and that there is a market failure (i.e. there is insufficient commercial market to attract private R&D investment).
The scope of the G-FINDER project is reviewed annually in consultation with an expert Advisory Committee. As a result of these consultations, and following an initiative to survey the landscape of sexual and reproductive health R&D funding, the disease scope used for the pipeline review in 2019 was wider than in 2017, with the addition of the following diseases: chlamydia, gonorrhea, hepatitis B, HPV-cervical cancer, human T-cell lymphotropic virus-1, herpes simplex-2, mycetoma, pre-eclampsia, post-partum hemorrhage and syphilis. Snakebite envenoming was added to the G-FINDER scope after the conclusion of the 2019 pipeline review and was therefore not included in the scope of this analysis. "Reproductive health"-which included both contraception and the prevention of sexually transmitted diseases through the use of multipurpose prevention technologies (MPTs) that prevent pregnancy and sexually transmitted infections-was included as a single health issue.
Accordingly, the R&D pipeline review presented here comprises 45 individual disease or health issues and six multiple-disease groups (Underlying data: Data File 1 shows the full list of diseases included in the pipeline reviews for both 2017 and 2019).
The G-FINDER scope includes drugs, microbicides, biologics, vaccines, diagnostics, vector control products, devices and MPTs. Not all product types are included for every disease or health condition: the product category is excluded if a viable commercial market is thought to exist, and additional restrictions are applied to some disease and product categories if there is potential commercial (high-income country market) overlap. For example, the G-FINDER scope excludes dengue vaccines, and HIV/AIDS drugs are included only if they are label-extensions or reformulations specifically intended for developing country use (e.g., pediatric or slow-release formulations; fixed dose combinations; low-dose formulations for prophylaxis; long-acting injectables for treatment or prophylaxis). Medical devices (except for diagnostics and contraceptives), and general or supportive therapies (e.g., oral rehydration or nutritional supplements) are not included in the scope of G-FINDER and were therefore not included in the pipeline reviews. Due to changes in the G-FINDER scope between 2017 and 2019, as described above, two disease-specific product areas were included for the first time in 2019: vaccines for leprosy, and biologics for HIV.
The pipeline review included candidates at all stages of development, from discovery through to product registration. Drugs, vaccines, and reproductive health candidate products were classified in the following five development stages: discovery; pre-clinical studies; and clinical development (Phase I, Phase II and Phase III). Diagnostics and vector control products follow different product development and regulatory pathways, so these candidate products were classified under the following stages: concept and research; feasibility and planning; design and development; and clinical validation and launch readiness.
Candidates were no longer considered to be part of the R&D pipeline -and therefore were excluded from this analysis -once granted regulatory approval by a stringent regulatory authority, or if their development had been placed on hold indefinitely. Candidates in the discovery phase were also excluded, as only candidates undergoing advanced pre-clinical development up to Phase III are included in the P2I.v2 model.
Further details on the diseases and product areas within the scope of the pipeline analysis are provided in the G-FINDER R&D matrices and scope documents.
We reviewed and cross-referenced all major sources of available data on the R&D pipeline within the scope outlined above. (iii) Classification of candidates into archetypes In the pipeline portfolio review, we identified a total of 1160 product candidates for neglected diseases, Ebola and sexual & reproductive health as of August 31, 2019. We excluded 406 of these from the model because (a) they were already marketed, or in a development phase that is excluded from the P2I. v2 model (n=230), (b) there was insufficient information about the candidate from an authoritative source to be able to confirm its development phase or classify it into an archetype (n=10), or (c) the candidate was no longer in development (n=166). After these exclusions, 754 candidates were included in the model.
We then classified each of the included candidates into six primary archetypes: repurposed drugs, NCEs, vaccines, biologics, diagnostics, and 'other products' (which included only vector control products). Candidates classified as repurposed drugs, NCEs, or biologics were further classified as either simple or complex (with a further distinction for simple NCEs and simple biologics targeting TB); vaccine candidates were classified as either simple, complex, or unprecedented; and diagnostic candidates were classified as either assay development or simple platforms. This resulted in each candidate being assigned to one of the 14 unique archetypes outlined in Table 1.
Contraceptives, microbicides and MPTs were classified according to the constituent molecule (NCE, repurposed drug, or biologics). If there was more than one active drug ingredient in the MPT, the candidate was classified according to the most complicated component; we did not consider if the polymer or technology itself was innovative. TB drug candidates beyond Phase IIa were listed as regimens instead of individual compounds because TB treatment is regimen-based rather than monotherapy. Eight diagnostics candidates that were classified in the 2017 review as simple platforms were reclassified as assay development, reflecting the fact that ongoing activity was restricted to disease-specific assay development based on a previously validated proprietary diagnostic test technology.
The evidence used to support the classification of candidates into archetypes came from a wide range of sources, including academic literature (both original research and review articles); relevant publicly available product databases such as NCATS Inxight: Drugs; information from international clinical trials registries, including ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform; websites of PDPs; patent databases; and relevant reports and news releases from bilateral and multilateral funding agencies, companies, PDPs, other product developers, and non-government organizations. In assigning each candidate product to an archetype, we documented any relevant source material that guided the classification, such as a peer-reviewed research article on the candidate's mechanism of action. Regimens were classified according to the complexity of the most complicated component.
(iv) Inputting the pipeline of candidates into the adapted P2I model Following the pipeline portfolio review and classification of product candidates into their archetypes, the candidates that met our inclusion criteria were inputted into P2I.v2. For each disease and archetype, we entered the number of candidates that were in each phase of development at the time of our review. For the candidates classified as repurposed drugs, NCEs, biologics, vaccines, and "other products" (vector control products), the phases were preclinical, phase I, phase II, or phase III. For diagnostic candidates, the phases were concept and research, feasibility and planning, design and development, or clinical validation and launch readiness. The analysis was conducted in November 2019, and hence we selected 2019 as the start year for the modeling.
Based on the previously established parameters for cost, attrition rate, and time per phase for each archetype, the model estimates the costs and outcomes of moving product candidates through the pipeline from their current phase. When a candidate is classified into a particular phase in the model, it is assumed that it is at the beginning of the phase, which means that the cost of moving that candidate through to the end of its current phase is included in the total estimated costs for any given product.
We did not apply a discount rate to our cost estimates. Probability of success, length of phase, cost variables, and archetype were assumed to remain constant throughout the lifecycle of the model. Only candidates that met our inclusion criteria at the time of the pipeline portfolio review were included, meaning that the analysis did not include new candidates entering the pipeline over time.
For this paper, we have chosen a conservative approach to presenting the launches-we have considered a launch to be a binary event, i.e., we have rounded down (in this case, 2.7 rounds down to 2 launches). However, in Underlying data: Data File 2, we also present the results without any rounding (e.g., in this example, 2.7 launches) and with rounding to the nearest integer (2.7 would round to 3). Both of these other approaches give less conservative estimates of the number of launches.
Where rounding was used, it was only applied at the very end of the model. For example, for disease X, if there were 3 simple vaccine candidates at Phase II that led to 1.3 expected launches and 3 simple vaccine candidates at Phase III that led to 1.4 expected launches, we rounded the cumulative total (in this case 2.7).
In this paper, we call the current pipeline of candidates the "2019 pipeline" (i.e. the pipeline of candidates as of August 31, 2019). We call the previously identified pipeline, published by Young et al, the "2017 pipeline" (i.e. the pipeline as of August 31, 2017) 6 . As described below, we conducted two distinct analyses. First, we did an analysis to obtain a time trend-that is, we examined changes over time in the number of candidates in the pipeline, the estimated costs to move them through the pipeline, and the likely resulting launches. For this time trend analysis, we used the exact same scope of diseasesan "apples to apples" comparison (we call this the "direct comparison" between the 2017 and 2019 pipelines). We call the 2019 pipeline used for the direct comparison analysis the "2019 direct comparison pipeline." Second, for the 2019 pipeline, we also used an expanded list of diseases. We call the pipeline used for the complete 2019 analysis the "complete 2019 pipeline."

The direct ("apples-to-apples") comparison analysis between the 2017 and 2019 pipelines.
The direct comparison analysis included product candidates in the 2019 pipeline that were identified using the exact same scope and search methodology used in 2017 6 . As described in the Results section, we found a higher number of candidates in the 2019 direct comparison pipeline than in the 2017 pipeline. Since we used the same approach to identify candidates for both the 2017 and 2019 direct comparison pipelines, the higher number of overall candidates identified in 2019 was due to "true" changes in the pipeline since 2017. Despite the overall increase, when disaggregated by disease and by archetype the pipeline showed both increases and decreases in the numbers of candidate products across different diseases/archetypes. The product candidates included in the 2019 direct comparison pipeline belonged to the exact same 35 diseases and 5 multiple disease categories as those included in the 2017 pipeline analysis 6 .
The complete 2019 pipeline analysis. The complete 2019 pipeline analysis included all candidates from the pipeline portfolio review that met our inclusion criteria based on an expanded scope of diseases (45 diseases and 6 multiple disease categories). Again, as described in the Results section, we found a higher number of candidates in the complete 2019 pipeline than in the 2017 pipeline, but in this case the increase was partly due to the expansion of the disease scope, as well as true changes in the pipeline. Additionally, eight diagnostic candidate archetype classifications were changed between the 2019 direct comparison pipeline and the complete 2019 pipeline. These were existing candidates captured in the 2017 pipeline, and so their archetype classification was left unchanged in the direct comparison analysis to allow a true apples-to-apples comparison. This classification was updated for the complete pipeline analysis to reflect the fact that ongoing development activity for these candidates was now restricted to disease-specific assay development based on a previously validated proprietary diagnostic test technology.
(v) Estimating the costs of priority "missing" products As described in the Results section below, the complete 2019 pipeline is unlikely to lead to launches of several critically needed technologies (e.g. a hepatitis C vaccine). To estimate the costs to develop products that are likely to be "missing" but are highly needed, we used the same methodology adopted by Young et al. 6 Young et al. used the list of "important" or "game changing" diagnostics, drugs, and vaccines prioritized by the CIH, which was developed through expert consensus 1 . We examined the overlap between the CIH's list of needed products and those that our modeling suggested would still be missing by 2031 based on the complete 2019 pipeline. For each missing product, we used the P2I.v2 model in a retrospective manner to estimate the number of additional candidates that would be needed at preclinical phase-over and above the existing candidates-to lead to one expected launch of that product, and the associated additional cost.
For example, we found 39 malaria vaccine candidates in the pipeline and the modeling suggested that these would result in 0.41 launches. Thus, to estimate the additional costs to reach one launch, we estimated the number of additional candidates needed at preclinical phase and the associated additional costs to achieve an additional 0.59 launches (in this case, an additional 144 candidates would be needed at preclinical phase to achieve 0.59 launches, at an additional cost of $3.3 billion).

(vi) Sensitivity analysis
For the sensitivity analysis we adopted a method detailed by Mestre-Ferrandiz et al. at the United Kingdom Office of Health Economics in their study "The R&D Cost of a New Medicine" 8 . Young et al. also used this method 6 . Based on this methodology, we altered the model's parameters for cost per phase and attrition per phase for each archetype by 10 percent higher and 10 percent lower. We also examined the impact of all possible combinations of these changes (e.g., 10% higher probability of success per phase and a 10% higher cost per phase, 10% higher probability of success per phase and a 10% lower cost per phase, etc.). We conducted this sensitivity analysis for moving current candidates through the pipeline both in our direct comparison analysis and our analysis of the complete 2019 pipeline. Additionally, we conducted this sensitivity analysis for the costs of developing priority "missing" products.
Since the length of time is independent of the cost variables in P2I model, we did not conduct a sensitivity analysis varying the length of time per phase.

(i) Candidates identified by the 2017 and the direct comparison 2019 pipelines
For the direct comparison analysis (the "apples-to-apples" approach), we only included candidates that were identified using the same method and scope used in 2017, as described in sub-section (ii) above. Using these inclusion criteria, we identified 690 candidates in the 2019 direct comparison pipeline that met the inclusion criteria for being entered into the P2I.v2 tool, an increase of 152 candidates from 2017 (see Underlying data: Data File 3).    Table 4 and Table 5 show the product candidates in the 2017 and the complete 2019 pipelines, broken down by archetype (Table 4) or by disease (Table 5). In both tables, the difference in the number of candidates between 2017 and 2019 is further broken down into "difference due to changes in the pipeline" and "difference due to changes in scope." Of the 216 additions from 2017 to 2019, 152 candidates were added from a "true" change in the pipeline and 64 product candidates were added due to a scope expansion. Out of these 64 product candidates, 31 products were identified due to either including a new archetype in the scope or due to change in the inclusion criteria of existing archetypes ( Table 6). The remaining 33 candidates identified due to the scope expansion belong to new diseases included in the 2019 pipeline. These were 5 candidates for chlamydia, 11 candidates for gonorrhea, 8 candidates for hepatitis B, 7 candidates for herpes simplex-2, and one candidate each for human papilloma virus-cervical cancer and mycetoma. These candidates are included in Table 4 under scope expansion. Table 7 summarizes the changes in classification for the eight diagnostic product candidates based on the updated information available in 2019.  Reproductive health New chemical entity 7

Repurposed drugs 6
Leprosy Vaccine 1 A key finding of the direct comparison analysis is that the 2019 pipeline is expected to lead to launches of HIV and TB vaccines. In our previous study, based on the 2017 pipeline, launches of HIV and TB vaccines were not expected (i.e. there was a probability of launch of under 1.0 for each) 6 . Table 9 summarizes the expected number of launches based on the 2017 and the direct comparison 2019 pipeline, broken down by disease. The difference in expected launches between the 2017 and the direct comparison 2019 pipelines are summarized under the fourth column, labeled "Expected launches: difference due to changes in the pipeline," in Table 9.

Multi-disease
(v) Expected launches based on the complete 2019 pipeline analysis As shown in Table 8 Table 5 and  Table 6), and (b) the reclassification of eight diagnostics, as summarized in Table 7. The reclassification of diagnostics increased the number of candidates classified as assay development for some diseases resulting in additional launches compared to the 2019 direct comparison pipeline. The expected launches resulting from the complete 2019 pipeline analysis, which had an expanded scope of diseases, include six additional expected launches for diseases not included in the 2017 pipeline review. These launches are one NCE for hepatitis B; three diagnostics for chlamydia; and two diagnostics for gonorrhea.
The scope expansion also added an expected launch of one biologic for HIV/AIDS and one repurposed drug for reproductive health. The remaining three additional launches were all diagnostics-for hepatitis C, TB, and malaria-and these additions are related to the changes in classification for diagnostic candidates.
The expected additional launches based on the complete 2019 pipeline analysis compared to the direct comparison 2019 pipeline are summarized under "Difference in expected launches due to: scope expansion" and "Difference in expected launches due to: classification changes" in Table 10.     Figure 4 compares the costs of moving candidates for HIV/AIDS, TB, Ebola, and malaria through the pipeline to launch based on the 2017 versus the direct comparison 2019 pipelines. Table 11 compares the costs to move candidates through the pipeline based on the 2017 and direct comparison 2019, disaggregated by disease (the cost differences are summarized under the column "Cost difference due to changes in the pipeline").   Table 12 gives the individual costs). This $1. The changes in classification for the eight diagnostic candidates (Table 7) from simple platform development in the direct comparison 2019 pipeline to assay development in the complete 2019 pipeline reduced the overall cost by $719 million. This cost reduction is due to the cost per phase being lower for assay development classification than the cost per phase for simple platform development classification for diagnostics in the P2I.v2 model.
Based on the complete 2019 pipeline analysis, Table 12 summarizes the changes in cost by disease based on the changes in the pipeline, scope expansion and classification changes. In the complete 2019 pipeline, of the total cost, 62% ($12.9 billion) would be for phase III candidates, 22% ($4.7 billion) for phase II candidates, 4% ($921 million) for phase I candidates and 12% ($2.5 billion) for pre-clinical candidates. Furthermore, in the complete 2019 pipeline, over 75% of the cost ($15.9 billion) would be incurred during the first five years.
(viii) Results of the sensitivity analysis for moving current candidates through the pipeline For the direct comparison 2019 pipeline, the sensitivity analysis estimated that the total cost to move the current candidates through the pipeline ranges from $16.4 billion to $23.8 billion and the expected launches range from 150 to 240 (Table 13). For the complete 2019 pipeline, based on the sensitivity analysis results, the total cost to move the current products to launch ranges from $17.3 billion to $25.2 billion and the expected launches range from 159 to 256 (Table 14).
(ix) Estimated costs to launch missing products Our first study using the P2I.v2 tool, based on the 2017 pipeline portfolio review, suggested that there would be 18 "missing products", which were identified using the list of needed priority products outlined in CIH report 6 . In the 2017 analysis, the estimated cost for launching these 18 missing products was between $13.6 billion and $21.6 billion, depending on the complexity of the product.
In the 2019 analysis, the model projects that there are likely to be launches of HIV and TB vaccines and, as a result, there would now be 16 missing products. Since the number of candidates do not change for the 16 missing product categories between the 2019 direct comparison pipeline and the complete 2019 pipeline, the missing products analysis is applicable to both pipelines. To launch one of each of the 16 key missing products, starting from pre-clinical phase, we estimate the costs to be between $5.5 billion (assuming lowest complexity products) and $14.2 billion (assuming highest complexity products). Table 15 shows the summary of these costs by disease and archetype. The ranges are calculated using the sensitivity analysis described in sub-section (vi) of the Methods.
For the 16 missing products, about 70% of the costs would be incurred during the first five years of product development. Therefore, the costs for the first five years range between $3.85 billion and $9.94 billion.
Over the next five years, the combined cost to move all current candidates through the pipeline and to launch a product each for 16 "missing products" ranges between $19.7 to $25.8 billion, or $3.9 to $5.2 billion per year.

Discussion
Our study found that the product development R&D pipeline for poverty-related and neglected diseases has grown by more than a quarter in just two years, driven in particular by the global response to 2018-19 outbreaks of Ebola virus disease (EVD), as well as reproductive health, TB and malaria. As a result, the number of expected new product launches expected based on the 2019 pipeline increased by half compared to 2017, while the cost of advancing the pipeline increased by a quarter. Most of the increase in the number of pipeline candidates and the number of launches was in diagnostics, but most of the increased (and overall) costs were associated with vaccine development. The distribution of the R&D pipeline broadly aligns with the landscape of funding going to neglected diseases as assessed in the G-FINDER reports.
Changes in the pipeline from 2017 to 2019 At August 31, 2019, there were 754 product candidates that fell within the study scope and had sufficient publicly available information to allow them to be entered into the P2I.v2 model. Of these 754 candidates, 690 were in disease and product areas that were included in our 2017 review, representing an increase of 152 candidates (or 28%) since 2017.
The pipeline growth was primarily in just four areas. Ebola, reproductive health, TB and malaria collectively accounted for 130 (86%) of the 152 additional candidates. The increase in pipeline candidates occurred for all archetypes except for NCEs, but the largest increases were in diagnostics (up by 67 candidates, a 66% increase) and vaccines (up by 48 candidates, a 23% increase), which together accounted for more than three quarters (76%) of the overall increase in product candidates.
Ebola alone was responsible for a significant proportion (41%) of the total growth of the R&D pipeline between 2017 and 2019. The number of candidates in the pipeline for Ebola increased fourfold in this period, reflecting the robust international response to two outbreaks of EVD in the Democratic Republic of the Congo (DRC) in 2018-19, the second of which -originating in North Kivu province -became the second largest EVD outbreak on record 9 .
Although TB and malaria were among the diseases to see the most significant pipeline growth, the surge in candidates for It is concerning that the size of the R&D pipeline for diarrheal diseases, salmonella infections, helminth infections and kinetoplastid infections either fell or was at best unchanged from 2017 to 2019. Of the 35 diseases included in the 2017 review, 15 still had fewer than four candidates each in the pipeline in 2019. Many of these are neglected tropical diseases, whose funding has remained basically flat over the course of the last decade, while funding for HIV/AIDS, TB, malaria and Ebola has grown significantly. Also concerning is the fact that there was no increase in the number of NCE candidates in the R&D pipeline (in fact there were two fewer in 2019 than in 2017), despite the overall pipeline growing by more than a quarter. There was a roughly even split in the number of new candidates in early-stage development and those in clinical trials or field development.

Launches and missing products
The model suggests that the 754 candidates currently in the pipeline will deliver 207 launch-ready health technologies targeting poverty-related and neglected diseases. Looking just at the "apples-to-apples" subset, the 690 candidates in the direct comparison pipeline would be expected to deliver 196 launches, an increase of more than half (53%, an additional 68 launches) compared to the 2017 review.
Diagnostics accounted for the vast majority of the increase in expected launches in 2019 over 2017, mostly for assays One of the most significant changes in the 2019 review compared to the 2017 review is the estimated launch of vaccines for HIV/AIDS and TB by 2031 (the 2017 pipeline would not have led to these two launches 6 ). This is a positive reflection of progress in the pipeline for these two areas since our last review, but it is worth emphasizing that the P2I model makes no assessment of the appropriateness of the projected launch-ready candidate, nor of any additional research that may be required to support product introduction. So while the model predicts the likely launch of a TB vaccine, the profile of the most advanced candidates in the pipeline suggest that -at the date of the launch projected by the model -such a vaccine may only be approved for use in limited populations. In the case of HIV/ AIDS, caution may be warranted given that thus far no candidate has been able to match even the 31% efficacy achieved in the 2009 RV144 Thai Phase III clinical trials, and that during the preparation of this report one of the two most advanced HIV vaccine candidates in the pipeline (a modified version of the RV144 vaccine regimen) was found to be ineffective.

TDR Health Product Portfolio Directory
In May 2019, TDR launched the Health Product Profile Directory (HPPD), described as an "online database describing 8-10 key characteristics (such as target population, measures of efficacy and dosage) of product profiles for medicines, vaccines, diagnostics and other products that are intended to be accessed by populations in low-and middle-income countries." 10 Out of the 50 WHO-authored target product profiles (TPPs) contained in the HPPD, 28 align with the scope of this study, covering 11 diseases. Whilst not all areas with the largest pipelines have WHO-authored TPPs, most of the areas for which WHO-authored TPPs exist have healthy pipelines.
The continued expansion of the TB diagnostics pipeline (which grew by 50% between 2017 and 2019) is a case in point, but also one that highlights a potential imbalance in the current pipeline. In 2014, the WHO published a report of a consensus meeting highlighting high-priority TPPs for new TB diagnostics, divided into four use cases: (a) a sputumbased test to detect pulmonary TB; (b) a rapid biomarker-based non-sputum-based test; (c) a drug-susceptibility test; and (d) a community-based triage or referral test for identifying people suspected of having TB. In 2019, almost two-thirds (21, 64%) of all diagnostics in late-stage development for TB are rapid drug susceptibility tests; this is compared to just four rapid biomarker-based non-sputum-based test, two triage tests and a single sputum-based test to detect pulmonary TB (the remaining five late-stage candidates are not indicated for a specific TPP).
Further work to map the pipeline against the TPPs in the HPPD would provide useful insights into the nature of the R&D pipeline.

New cost estimates
Overall, the P2I.v2 model suggests that advancing the current pipeline of 754 candidates to deliver 207 launch-ready health technologies would cost an estimated $21 billion by 2031. More than half of the total amount ($12 billion, 59%) of the cost of advancing the candidates is projected to be spent during the first three years (2020-2022), at an average cost of $3.9 billion per year.
Looking just at the "apples-to-apples" subset, the P2I.v2 model suggests that advancing the pipeline of 690 candidates to deliver 196 launch-ready health technologies would cost an estimated $19.9 billion by 2031. This represents an increase of $3.5 billion (up 22%) over the estimate from the 2017 pipeline review.
The bulk of this $3.5 billion cost increase was due to an additional $3.1 billion in the cost of advancing vaccine candidates, and almost half (47%) was from the more than doubling of projected costs for advancing Ebola candidates (up $1.7 billion, 145%). This makes sense, as Ebola was responsible for a massive increase in the number of vaccine candidates, many of them in pre-clinical development; however the fate of these early-stage candidates is likely to be affected by the approval of Merck's Ervebo (and pending likely approval of Janssen's MVA-BN-Filo), as well as the waning of the current DRC outbreak.
Product development costs also increased dramatically over 2017 for meningitis (up $857 million, from a low base; although this was partly due to better sources, primarily the draft publication of the new 2030 global roadmap for meningitis vaccines) and pneumonia (up $605 million, 94%; reflecting new vaccine candidates advancing to Phase III clinical trials). Although malaria and TB account for a significant proportion of the projected costs, there was no significant increase in projected costs for these diseases between 2017 and 2019, with pipeline growth in both diseases being mainly in diagnostics rather than in other more costly archetypes.
The additional cost to launch all 16 "missing products" ranges from $5.5 billion to $14.2 billion, depending on the complexity of the products, around 70% of which would be spent in the first 5 years. Combined with the cost of advancing the current pipeline, this means that in the first 5 years the total estimated costs to move all current candidates through the pipeline and develop the 16 missing products would be around $3.9 billion to $5.2 billion per year. This total does not include funding for basic research, discovery, and post-registration studies, all of which are outside the scope of the P2I model cost projections.
How does this compare to current global funding levels? According to the most recent G-FINDER neglected disease report and two soon to be published analyses of R&D for sexual and reproductive health and emerging infectious diseases, total funding for the poverty-related and neglected diseases included in the scope of this paper totaled $3.6 billion in 2018, once funding for basic research and post-registration studies is excluded. This suggests an annual product development funding gap of at least $0.3 billion to $2.6 billion even at current record levels of neglected disease R&D funding.

Strengths of our new study
To the best of our knowledge, this is the first study to provide time trends for the R&D pipeline of poverty-related and neglected diseases, as well as for product launches and cost estimates. Information on these trends is valuable for R&D funders, researchers, product development partnerships, and policymakers more broadly. Additional diseases were included in this new analysis including the WHO Product Development for Vaccines Advisory Committee's priority pathogens, such as HSV-2 and gonorrhoea.
A major strength of this study, as well as our previous study using the P2I model to examine the 2017 pipeline 6 , is that it goes beyond analysing a single therapeutic portfolio (e.g. the pipeline of drugs for TB control 11 ) to examine the full portfolio of candidates for a very broad range of poverty-related and neglected diseases. This broader approach "aligns with the way in which funders pursue a diversified portfolio of product development projects." 6 Our study helps to identify critical gaps in the global health product development pipeline by showing (a) which diseases and archetypes have few candidates, (b) which product launches are most likely, and (c) which products will probably still be missing based on existing candidates.
Strengths of the P2I.v2 tool We have previously described the strengths of the P2I.v2 tool in detail 6 . In brief, we highlight three particular strengths here. First, the tool is publicly available online, as are the full details of the pipeline review, model assumptions, and model inputs and outputs. Thus readers can replicate, improve on, and further adapt our work. Second, the tool is highly flexibleusers can make modifications to the underlying assumptions to see how such changes would influence the results. Third, the underlying model assumptions are likely to be realistic. These assumptions are based on a very large number of data points (e.g., assumptions on success rates and cycle times were based on data from of 25,000 development candidates), and were validated through examining peer-reviewed literature, industry reports/databases, and expert interviews 7 . Our assumptions are roughly in line with reported industry standards (the amount of variation differs by product types) 12,13 .

Limitations of our new study
The pipeline data used in this study represents a snapshot of the product development pipeline at a single point in time (up to August 31, 2019). Since the pipeline is constantly evolving, any changes-such as the advancement or termination of pipeline candidates-that occurred after this date are not reflected in the pipeline count or the resulting analysis.
Our research was limited to publicly available sources and relied on confirmation from authoritative sources, including the website of the candidate developer, recent reports from international organizations such as WHO and Unitaid, and clinical trial portals. Where we could not confirm the status of a candidate, we excluded it from the active candidate list. The lack of publicly available information on some products under development means that our count of active candidates may be an underestimate. In particular, there is comparatively less information available on pre-clinical candidates, due to the lack of dedicated public registries mirroring those available for clinical trial candidates, and limited sharing of proprietary data by companies. Our reliance on English language sources also means that the undercounting of early-stage R&D activity may disproportionately affect particular countries or regions. Early-stage candidates from the Middle East, Russia and China are likely to be underrepresented.
Our reliance on publicly available data sources also meant that we were unable to identify when each candidate had entered its current phase of development, meaning that from a model input perspective, a candidate that was close to the conclusion of its phase was treated identically to a candidate that had just entered it.
As in our 2017 study 6 , our new study uses the consensus prioritization exercise conducted by the 2013 Lancet Commission on Investing in Health to identify the critical "missing" products that the current pipeline of product candidates would fail to deliver, according to the P2I. v2 model. While we believe this remains a robust global consensus, it is now seven years old. In addition, we note that the P2I. v2 model does not account for whether sufficient research and trial site capacity exists to conduct the additional studies that would be needed to deliver these missing products in the timeframe projected by the model.

Limitations of the P2I tool
We have previously described the limitations of the P2I.v2 tool in detail 6 . In brief, we highlight six particular limitations here.
First, the P2I tool is a simple, static, and deterministic model that does not consider (a) possible changes in the product development process, such as improved techniques, novel clinical trial designs (such as highly efficient trials 14 ), new regulatory pathways, or (b) product-related decisions, such as the decision to discontinue candidates.
Second, the tool requires users to classify every candidate based on the categories available in the model, yet it can be challenging to categorize some candidates into the P2I. v2 archetypes. In particular, there are too few archetypes for diagnostics and vector control products-for example, there are no specific archetype fields for vector control products, with both chemical and biological products included as a single archetype.
Third, the assumptions on probabilities of success for unprecedented vaccines in phases II and III are based on a relatively small number of data points-around 10-25 data points per estimated value.
Fourth, the model assumptions used for each archetype are the same for almost every disease, with just a few exceptions ("unprecedented vaccines" refers to HIV, TB, and malaria vaccines, and there are two TB-specific archetypes, "simple NCE for TB" and "simple biologic for TB"). Therefore, the model does not take into account the specificities involved for developing a product against a specific disease. So, for example, in the model, the costs, success rates, and cycle times per phase for developing a simple vaccine for schistosomiasis would be the same as those for developing a simple vaccine for hepatitis C.
Fifth, the model does not take into account the public health value and clinical utility of a candidate, or the impact of successful product launches on the remaining candidates in the pipeline.
Finally, the model does not include discovery, early pre-clinical development and post-phase III costs such as regulatory review, marketing authorization, Phase IV and pharmacovigilance and other implementation costs. These other costs can be substantial-for example, funders have already committed $49 million to the first phase of implementation studies for the RTS,S malaria vaccine, with further funding required.
Despite these limitations, efforts are now underway, funded by TDR, to address these weaknesses and to develop further iterations of the tool. TDR is funding many different product development partnerships, including MMV, FIND, and DNDI, to use the tool to analyze their own portfolios; these analyses will also help to refine and improve the tool. TDR is also funding research institutions, such as the Graduate Institute of Geneva, to collect data that can help make further tool modifications. While we did reach out to PDPs to ask them to share new data on product development that could potentially be used to develop a version 3 of the tool (P2I.v3), the PDPs did not have large enough numbers of data points to meaningfully make tool refinements.

Conclusions
This study has provided additional evidence that the P2I tool can be used effectively to estimate the costs and probable launches associated with moving a portfolio of current candidates for neglected diseases through the pipeline. It can help to identify gaps in the R&D pipeline and to estimate the size of the gap between funding requirements and current levels of R&D investment. Importantly, this study has also shown how the model can be used to estimate the impact of changes in the pipeline over time.

Data availability
Open As the other reviewer has noticed, the authors did not apply discounting or 'cost of capital'. In my view, this is appropriate, as there is the (implicit or explicit) assumption that the gaps in funding will/should be filled by government or philanthropic funding. In other words, COC estimation is appropriate for a business proposal, but this is not a business proposal. That said, the authors should give their reasoning behind not including COC in calculations.
One interesting output -which I did not see in the current draft -would be an estimate of the total cost per approved product (including failures

Mikel Berdud
Office of Health Economics, London, UK The work by Gavin Yamey and colleagues 'Analysis of the health product pipeline for povertyrelated and neglected diseases using the Portfolio-to-Impact (P2I) modelling tool' is a relevant and original research on an important topic: the additional funding needed for poverty-related and neglected disease (PRND) product development.
Based on a modelling exercise, the study provides evidence to answer two relevant questions: How much funding is needed to progress the PRND related 2019 pipeline to the market based on an analysis of the period 2019-2031.

○
How large is the funding gap needed to cover to develop and produce at least one launch in each disease listed for the modelling -based on a list of a total of 45 diseases? ○ Methods are sound and well-described, and data are transparent and publicly available. The model is also accessible and therefore analyses can be easily replicate and scrutinised by readers. My first comment is that this study is a follow-up analysis of the previous work by Young et al.
(2018) 1 . The current analysis replicates the same estimates using the same methods but with an updated pipeline data and expanded disease list -from 35 diseases to a 45 diseases. It is important to contextualise the present work this way in order to correctly assess its contribution.
In that sense, I would like to highlight which in my opinion are the two the contributions that can be drawn from the reviewed article: The study demonstrates that PRNDs product R&D pipeline is a vivid organism which only in two years has experienced a huge evolution. The analysis that authors call "2019 direct comparison pipeline" which is based in the same list of 35 diseases of Young et al. 1 shows that the pipeline has increased in 152 new drugs (+28%) with vaccines (+48) and diagnostics (+67) leading this increment and diseases like Ebola (+62), TB (+22) and Malaria (+18) being the most benefited. This can be considered an endogenous change of the PNRDs related pipeline. A change of this magnitude in just two-year period evidences the necessity for policymakers to track the evolution of the pipeline closely and frequently to update the expected launches and required funding to see where and by how much the focus has to be placed to encourage the most needed and impactful innovation. Learnings from continuous analysis of the pipeline may help to shape the Pharmaceutical market regulation and incentives (i.e. push and pull funding programmes).

○
The PRNDs is also by itself an evolving set of diseases. The previous work in which this new study is based was only published two years ago. In that study a total number of 35 PNRDs were listed in 2017. Replicating criteria, a mix of clinical and socio-economic ○ variables, the number has been increased to 45 in 2019. This shows that the changing socioeconomic environment can affect the pipeline for PNRDs by "exogenously" affecting the list PNRDs. This is also an extremely relevant finding as it suggests that the optimal policy to tackle public health issues related to PNRDs requires a continuous update of what is included in the PNRDs list and which implications these inclusions may in terms of the required funding to progress the pipeline or to guarantee at least one treatment option in a reasonable time horizon.
In general, results are clear and well explained and easy to follow even for a non-expert reader, discussion is well written and goes through the most relevant findings and methodological details and the conclusion draws form results and discussion. There are many comments that I would like to provide to authors though, most of the requests for clarification and some other points that will require authors' attention and response. In order to be "indexed" i.e. listed in academic research output databases, I think authors should respond to all of them: 1.
Are attrition rates being applied by development phase? Or are authors basically applying the overall likelihood of approval (LOA) to each archetype without phase specific discrimination? I think this is important to clarify.

3.
○ Comment 2: This is a question for clarification. How are the probabilities of success for unprecedented HIV, TB, and malaria vaccines calculated/estimated/assumed? There is no approved HIV vaccine yet so how can one assume a positive probability alpha and consequently estimate the minimum necessary products in the pipeline required to have at least one launched in the future? Are authors applying the archetype probability of success (PoS) which means that HIV vaccines have the same POS than vaccines for meningitis B or mRNA vaccines? This is what I understand is the approach, otherwise PoS would be equal to zero for these unprecedented vaccines. therefore, If the complex vaccine archetype PoS is taken as reference for unprecedented vaccines, should not we have a HIV vaccine approved already given the huge investment devoted to the disease? But we don't and the analysis predicts one is expected to be launched based on the current pipeline, is not it tricky to assume? This is discussed by authors in the discussion section (page 22) where evidence that the HIV vaccine projects are highly uncertain to success and being launched is provided. In my opinion these specific cases should be subject to more detailed discussion perhaps acknowledging that it is likely that the estimated HIV vaccine will never progress to launch.
Comment 3: In the methods section, (ii) Pipeline portfolio review, page 7, authors explain the review and cross-reference all major data sources used. This seems to be a huge work producing results by itself. As a reader, I would benefit from an explicit/descriptive summary explanation of the most relevant findings in each. It is also very useful for further work that other researchers would want to start. Also, a summary of what was used from each of the data sources would help understand the data. Which universities, companies, references, governments, and non-profit organizations were checked? Might this information be presented in a table, an annex or as another supplementary material file? More details will provide more credit to methods. Authors should at least consider and assess this possibility. Comment 4: In the next paragraph of the above, authors explain a validation process leading to the inclusion of candidates. They were included only "if an authoritative source could confirm they were in active development". As a reader I miss more details about both, the process and the authoritative sources. Which clinical trial portals were used and for which products? How was the process of correspondence interaction with product developers? How many products were (in numbers and in percentage) checked by each authoritative source? What was the percentage of testing? Were all products checked for inclusion/exclusion by these means?
Comment 5: In the methods section, (iv) inputting the pipeline of candidates into the adapted P2i model, page 8, authors explain that "when a candidate is classified into a particular phase In the model, it is assumed it is at the beginning of the phase […]". This approach overestimates the cost of progressing the pipeline as many of the included products can be at mid-or late point of their current development phase at the time of the inclusion hence the full phase cost should not be considered. How large is the size of this overestimation is uncertain and will depend on the distribution of the actual degree of progress within each phase of the products? Why do authors take this approach? Why don't take the approach of assuming that all products are at the midpoint of each development phase? Some explanation and discussion of the degree of overestimation are missed.
Comment 6: In the following paragraph of the above's, authors recognise that they do not apply a discount rate to the cost estimates. They neither apply the rate of the cost of capital as the usual practice is in this type of estimation exercises (see DiMasi et al.  6 ). Authors then neglect an important and well-established driver of the R&D cost: the cost of capital. An omission that is not properly discussed in the paper. Additionally, they neither simply discount costs although the timelines for cost estimation used is over a full decade and the impact of the discounting significant. This approach implies that the time of development for each phase and archetype is only relevant for the cost stream and time horizon calculation, but it does not have any effect on the total funding required. I suggest to at least discuss this caveat in depth and in relation with what R&D cost estimation literature have been doing to this respect, which as I have said is opposite approach.
Comment 7: In the following paragraph of the above, authors discuss on the conservative rounding approach they use to estimate number of launches. They round down all expected launches number estimates the model provides. Are all launches between 0 and 1 (if any) considered equal to 0? If the estimate is an expected output and cost, I suggest (especially for the cost estimate) to consider results with any rounding to be the base case-explicitly mentioning the integer component of the number as the minimum expected launches -and then present results rounded down for comparison.
Comment 8: On page 9, in the paragraph 'The complete 2019 pipeline analysis' authors explain that there are eight diagnostic candidate archetype that were changed between the 2019 direct comparison pipeline and complete 2019 pipeline. My opinion is that relevant assumptions applied to the different diagnostic archetypes vary significantly, this approach may result in misleading. If these changes are rationalised then comparison should be done by assuming them. Alternatively, should not be a better approach to change the archetype in the 2017 pipeline for direct comparison? Or is it equivalent? Comment 9: in results section, subsection '(vii) Costs to move candidates through the pipeline based on the complete 2019 pipeline", page 18, authors assess the share of total cost / funding that belongs to each phase of development. For example, it is said that 62% of the cost is attributable to phase III. But language is loose as the reader may get confused. Is it 62% of the cost attributable only to projects currently in phase III? or is it the total cost of phase III attributable to all projects, those already in phase III and those that are not yet, but which they will be once they progress until to? I guess it is the second case but the dynamics should be explained, which means, the 62% also includes phase III cost of those projects still in phase I and phase II but which the P2I model predicts they will pass to phase III in the future.
Comment 10: In the results section, subsection '(viii) Results of sensitivity analysis…' authors briefly describe the result of the sensitivity analysis and refer the reader to check these results in tables 13 and 14. I consider this discussion on the sensitivity analysis to be insufficient. Sensitivities should be discussed appropriately by parameter and also for all parameters as a whole. Assessment of how sensitive the model and the estimation to the different parameters are and why is necessary so as a reader one can get an idea of what is driving the cost and why. Authors should speculate on the potential implications of these sensitivities and the robustness of their assumptions. In addition, given the dynamic and evolving nature of pipelines and pharmaceutical R&D processes, it should be also discussed the implications of potential changes on the parameters that may likely happen during the time horizon of the estimation exercise. Such discussion will provide valuable information to assess the model's robustness of the estimations and the appropriateness of updating the estimates recurrently.
Comment 11: In tables 13 and 14 )and related to comment 1/second bullet/point 1.), sensitivity analysis of the PoS result in an increase of the cost when PoS increases and vice versa. My concern here is whether the cost of failures of a given phase of development are included or not. For example, if for a given archetype, the pipeline is composed of 10 products and the PoS is 50%, then are the ten considered to calculate the cost of phase II of that archetype, or only the successes are considered? Can authors explain, please? I assume the cost of the ten products is included.
Comment 12: In the discussion section, page 20, authors state that "As a result, the number of expected new product launches expected based on the 2019 pipeline increased by half compared to 2017, while the cost of advancing the pipeline increased by a quarter". I guess the answer is in the model but the first question that comes to the reader's mind is, how an increasing return relationship may be possible. How an increase in the input of 25% can produce an increase in the output of 50%? It's true that we are measuring input in value and output in units, but even if this is taken into account, it is an input-output relationship that requires attention and explanation. Assuming the probabilities of success have not changed, the other possible explanations are (i) the rounding of the expected launches (ii) biased increase of the pipeline for archetypes with higher PoS or lower cost of development. I suggest expanding the discussion on this result to explain why these non-linear relationships happens. This is also important to find out which of the archetypes may be the most efficient to fund (note we are no taking the effectiveness considerations which in a value based assessment should also be included).
Comment 13: I discussion section, in the subsection "new cost estimates", page 23, authors explain the additional cost to launch "missing products". These estimates depend on the complexity of the products. It seems like the complexity has an impact of around +158% in terms of the cost increase. However, what 'complexity' means exactly in this context is not discussed, described or explained. As a reader, I miss more details here about what authors mean by complexity in this context, how they model it, and how such complexity translates to higher cost. Comment 15: I'm not the right person to judge the English language as I am not a native English speaker. As a non-native English reader, my assessment is that the manuscript is very well and clearly written and it does not require any work to improve apart from the usual proofreading and QA.