COVID-19 vaccine developers are racing against time to develop, test, approve and produce an effective vaccine. Currently more than 200 vaccines are under development¹ versus the 80 to 100 programs that were underway in April 2020. Many programs are still in the preclinical stage, but more than 40 candidates are in clinical trials.² It is expected that several vaccines may be approved within months. Each vaccine development program belongs to one of eight technology platforms. Vaccine manufacturers have announced cumulative manufacturing capacity that could produce as many as one billion doses by the end of 2020 and up to nine billion doses by the end of 2021.3

Instead of managing COVID-19 vaccine development programs individually, it was proposed in April 2020,⁴ and in May 20205that portfolio concepts and methodologies⁶ can be applied to the programs. This approach would lead to more coordinated planning and management of development projects, and a more efficient way to achieve strategic goals.

The paper will describe portfolio simulation as an effective tool to forecast achievability of COVID-19 portfolio strategic goals, including risk mitigation to boost vaccines development and production.

Portfolio strategic objectives are:

1. The manufacture of two billion doses by the end of 2021, earmarked for health system workers and for people over 65.
2. The manufacture of 11.2 billion doses in the shortest time with minimal risk (11.2 billion doses would vaccinate 70% of the world population of 8 billion, at two doses per person).7The authors8assume production of 18 billion doses, taking into consideration 20% loss; global distribution companies are planning for 20 billion doses.⁹
3. Other intermediate goals could be established as well.

In March and April 2020 it became obvious that COVID-19 vaccine development programs became numerous enough to be analyzed as a portfolio.4The model, published in April 2020, presented rather conservative estimates for the portfolio risk. At that time, “there is a ~40% chance that no vaccine is approved within 18 months, a ~67% chance that no more than one vaccine is approved, and a ~93% chance that no more than two vaccines are approved.”⁴ Then the Center for Global Development (CGD) followed, and published several blogs on the topic between June and September,^10-12 and a report,⁸ describing a very similar methodology.

Since April 2020, the portfolio of COVID-19 vaccines moved fast.¹³ Hence, the R&D portfolio risk profile is more favorable than in the model published in April.⁴ At the same time, new questions emerged related to the model’s granularity, manufacturing of vaccines, and risk mitigation strategies.

Methodology and the model

The COVID-19 vaccine supply chain includes three major blocks (Figure 1): (1) R&D and clinical trials; (2) manufacture of the vaccines, and (3) distribution of the vaccines. Each block has multiple components or sub-blocks. This paper covers only two sub-models: R&D, and clinical research and manufacturing due to absence of data about vaccines distribution.

Figure 1.COVID-19 supply chain. Source:Shnaydman, 2020

The R&D/clinical research sub-model simulates a vaccine development workflow³ from preclinical to approval. The simulator utilizes decision tree diagram with binary stochastic outcome (success or failure).¹³ The model¹³ was modified to simulate the portfolio of COVID-19 vaccines. Comparing to the current version of the model⁴ presents a more granular view of the vaccine R&D development process at a technology platform level and simulates production of the vaccines. It also incorporates algorithms for simulating interdependence between vaccines and speeding up Phase III trials across the portfolio to mitigate production risk and reduce production time to reach vaccination goals. Modeling capabilities are limited by data availability and data accuracy across the portfolio.

Input data for the model was collected from a variety of sources, including databases, academic and trade publications, webinars, podcasts, private communications, and others. Some data reflect a subjective view of the researchers. Therefore, the requirements for data granularity should be balanced with its uniformity across portfolio and sub-models.

Major data blocks

1. Portfolio landscape—number of preclinical, Phase I, II, and III candidates (approximately mid-July - August 2020) across eight platforms.¹
2. Planning horizon is 60 months. Modeling interval is a month.
3. Platform specific data for each vaccine
   • Cycle time for each clinical phase and each vaccine
   • Transitional POS at a phase and a platform level
   • Integrated monthly manufacturing productivity for each platform
   • Vaccine demand for each goal

• Cycle times were significantly compressed due to development urgency.¹⁶For lead candidates,^15,16 clinical trial stages are even more compressed and overlapped because of availability of additional funding in order to maximize the speed of development. The result is that much development work is done in parallel and at risk.
• Probabilities of success (POS)

POS is one of most important drivers of vaccine portfolio productivity. The first release of the model did not differentiate phased POS across vaccine platforms due to absence of data. This assumption was in line with POS data published in several sources at that time.^17-21 In these papers and reports, integrated POS for all vaccines^18,19 and vaccines for infectious diseases²⁰ were calculated based on statistical processing of historical data obtained from previous clinical trials. However, the lack of data needed to calculate POS at a platform level for COVID-19 vaccines portfolio required another approach.

Elicitation of POS for vaccine candidates across technology platforms

The three major approaches used to derive POS are summarized in Table 1:

Table 1. Advantages and disadvantages of different techniques for POS elicitation. Source: Shnaydman, 2020

For accurate simulation of COVID-19 vaccines portfolio, POS elicitation needs to be derived for each platform and each phase of a clinical trial. As the analysis in Table 1 indicates, currently available techniques may not be effective. For example, experts interviewing requires a large number of experts (16⁸) in order to make results statistically significant. Also, COVID-19 vaccines development portfolio is changing so rapidly that making another round of relevant interviews with the same large team of experts could be problematic.

Therefore, a new approach was proposed due to lack of historical data: POS elicitation from the development risk prospective. The approach is based on the assumption that the higher the POS, the lower development and regulatory risk, and vice versa. In other words, how can one convert development and regulatory risk into POS?

It looks very beneficial to evaluate development risk based on qualitative pairwise comparison of different risks related to global categories such as safety and efficacy at a platform level. Then, platforms are ranked according to risks related to each category, and POS derived according to ranked platforms using Analytic Hierarchy Process (AHP)—a multiple criteria decision making tool that has been widely used by in many applications and verticals,^22-24 such as planning, resource allocation, and risk management.

An example of structured risk analysis for COVID-19 vaccine development using AHP is presented in Figure 2.

The highest level of hierarchy, “Goal - Risk assessment across technology platforms and phases” is comprised of risk categories, such as efficacy and safety. Each category includes several most significant risks. Risk assessment includes two steps. The first step is a pairwise comparison of risk categories at a lexicographic scale from one to nine by an expert or a group of experts to determine weight of each category according to the goal. Pairwise comparison of risk categories generates their weights. For example, a vaccine efficacy of 0.4 and safety of 0.6; the sum should be equal to 1. Then, at the second step, pairwise comparison of risks within each risk category for each platform is conducted.

AHP software calculates ranking of each vaccine platform. In addition, experts estimated the minimum and maximum POS for each platform and phase, and the POS for each platform and phase was calculated.​

Risk assessment in vaccine development is subjective. In order to avoid bias, a group of experts is preferred to make robust assessments.^22-24 Also, AHP software is able to report inconsistencies in experts’ judgements, which interviewing lacks. Sensitivity to POS variations for the COVID portfolio simulator was tested.⁴

Figure 2. Risk hierarchical structure for POS calculation in vaccine development​

Source: Shnaydman, 2020

Manufacturing productivity and vaccine demand

Several manufacturers have announced their plans to produce about one billion COVID-19 vaccine doses by the end of 2020 and about eight to nine billion doses by the end of 2021.^3,25 Not all vaccines are expected to be successful, so production figures should be adjusted for POS.

The model incorporates an integrated network of manufacturing facilities for each platform, and is characterized by its productivity. For example, 25 facilities participate in manufacturing of COVID-19 vaccines.^26-28

Each platform includes multiple vaccine development programs. Vaccines within a platform have a lot of similarities, but they are still different. Therefore, the risk profile of each vaccine is unique, but the failure or success of a vaccine may have an impact on other candidates within a platform. Vaccine success may reduce risk for other vaccines within a platform and, therefore, increase their POS. Vaccine failure will increase risk for other vaccines, and therefore will reduce their POS.

It is assumed in the model that after each successful clinical trial, POS for other trials within a phase increases by x %, and decreases by y % if a clinical trial fails. This algorithm aligns with current practices of portfolio analysis for interdependent drugs.^14,17 Parameters x % and y % are derived based on an expert knowledge.³ POS may be increased up to maximum value of POS (POSmax) for a platform/phase, and will be decreased to the minimum value (POSmin) for a platform/phase. Incorporation of more complicated algorithms need to be supported by data availability and evidence from industry experts. POSmaxand POSminwere defined in section b “Elicitation of probabilities of success”.

Robust evidence was not found about vaccines interdependence between platforms.

There is a rule that permits scaling up of Operation Warp Speed (OWS) logistics in case of insufficient portfolio R&D productivity and diversification outside of preselected lead candidates. The rule can be applied to a limited number of qualified candidates due to constrained resources. It will allow overlapping clinical trial phases, and speed up the development and manufacturing process, making “at risk” production for a broader range of development programs. The rule could be applied to selected Phase II candidates, including replacement of failing lead programs. Therefore, a second round of financing may not be significant as for the original cohort²⁵ assuming an already developed manufacturing and logistical ecosystem. Authors⁸ limited the number of Phase III clinical trials to six. It seems contradictory to the industry practice where all Phase III trials are usually sped up.

Implementation of the proposed rule needs to be done gradually depending on portfolio risk profile, available resources, quality of candidates and policies.

Modeling results

1. Analysis of baseline scenario.
   1. Vaccine approvals within 18 months

Four scenarios were simulated – approvals within (1) 0-6 months; (2) 0-9 months; (3) 0-12 months, and (4) 0-18 months. Simulation results are presented in Figure 3.

Figure 3 legend:The graph shows maximum number of vaccine approvals (vertical axis) versus the likelihood of approvals (LOA) (horizontal axis). For example, in the 0 to 18-month period there is an almost 100% chance of at least one approval, approximately an 80% chance of more than two approvals, and approximately a 55% chance of more than three approvals. Source:Shnaydman, 2020

Modeling results indicate:

1. For the planning period 0-6 months, LOA of one approval in low (about 15%);
2. For the planning period 0-9 months, likelihood of zero approvals is about 10%, one approval is about 35%, two approvals is 30%, three approvals is about 21%, and four approvals is less than 5%. LOA for at least one approval is about 90%, LOA of two vaccines approval is 55%, etc.
3. More favorable results are for planning periods 0-12 months and 0-18 months. There is a good chance that up to three-four vaccines will be approved in 0-18 month period.

In further calculations, planning period (0-18) months will be used to analyze portfolio productivity and risk for both R&D/clinical trials and manufacturing.

In the baseline scenario, the model forecasts that only vaccines from five platforms out of eight will be approved at the end of 2021 due to the longer duration of clinical trials and the chance of additional funding for remaining platforms. Number of approvals vs. LOA for each platform is presented on Figure 5. Platforms corresponding with approved vaccines are:

1. DNA based
2. Inactivated virus
3. Non-replicating viral vector
4. Protein subunit
5. RNA-based vaccine

Detailed analysis of each platform is presented in many sources, such as [8].

Figure 4 legend:The graph shows maximum number of vaccine approvals (vertical axis) versus the cumulative LOA (horizontal axis) for the 0-18 month period across technology platforms. Results indicate that only vaccines from five platforms (1, 2, 4, 5, and 7) will be approved in 0-18 month period. Highest chances of approval has platform #2 (inactivated virus). Candidates from the WARP SPEED program [26] are among approved.Source:Shnaydman, 2020

Simulation results presented in Figure 5 indicate that upgrades of manufacturing plants related to five platforms listed above should be prioritized. At the same time,potential manufacturing capacity could be underutilized.3How can utilization of available manufacturing capacity be maximized and production risk be minimized?

Interdependence between vaccines within a platform.

Figure 5 analyzes sensitivity of the portfolio productivity to different interdependency rules.

Figure 5 legend:Sensitivity of portfolio productivity to interdependence rules. The graph shows number of vaccine approvals (vertical axis) versus the likelihood of approvals (LOA) (horizontal axis) for different degree of interdependence between vaccines within a platform. For example, reduction of POS by 30% in case of a trial failure, reduces portfolio productivity comparing to the baseline scenario. Source:Shnaydman, 2020

Figure 6 illustrates the POS dynamic for a phase/platform. POS is fluctuating depending on the outcome of previous trials within a platform.

Figure 6 legend:The graph shows fluctuations of POS (vertical axis) versus the number of clinical trials (horizontal axis) for a platform/phase due to vaccines dependency. In the presented scenario, first vaccine failed and POS reduced from 0.52 to 0.42. Then three successful trials raised POS to its maximum value 0.67.Source:Shnaydman, 2020

R&D approval time and manufacturing time across platforms

Simulation results indicate thataverage completion of R&D cycle is around 5-8 months for five prioritized platforms. The R&D cycle time for vaccines in platforms “Live attenuated virus” (#3), “Replicating viral vector” (#6), and “Virus like particles” (#8) exceeds 18 months. Therefore, the manufacturing process will continue for about a year for five platforms to reach a short-term goal of two billion doses. Could short and long-term vaccine production goals be met? What would be the associated risk.

Vaccine manufacturers announced cumulative capacity that could produce up to one billion doses in 2020, and up to approximately nine billion in 2021,²⁵ but in reality, due to high attrition rates, some candidates may fail and corresponding capacity plans may be altered. Therefore, the model will address several questions, such as:

1. Would R&D productivity of vaccines pipeline be sufficient to meet world-wide vaccination demand in the shortest time period?
2. Would manufacturing capacity be sufficient to produce enough vaccine doses worldwide?
3. Would corresponding production risk of not meeting the goals be tolerable?

Production risk is defined as probability of not meeting vaccine demand. The higher demand, the higher the risk of meeting the goal for both end of 2021 (two billion doses) and vaccination of the world (11.2 billion doses).

Short-term goal – two billion doses produced by the end of 2021

Figure 7 legend:The graph shows how risky manufacturing production is for different goals—two billion and four billion doses within the 0-18 month time interval. There is approximately a 50% chance that the goal of two billion doses will be met, and only 18% probability, approximately, for four billion doses (82% risk). X axis is risk, Y axis is the vaccine demand level, in millions of doses. Source:Shnaydman, 2020

Figure 7 indicates that the risk of not meeting short-term production goal of two billion doses is relatively high (about 50%). It will be much higher for the world vaccination. How can the production risk be mitigated and the portfolio risk profile be improved in order to meet vaccination goals?

Figure 8 presents the dynamics of average vaccine production.

Figure 8 legend.The graph shows average cumulative production dynamics for COVID vaccines, 0-18 months. On average 2.2 billion doses could be produced at the end of 2021. X axis is time in months, Y axis is the cumulative vaccine production, in millions of doses. Source:Shnaydman, 2020

Average dynamic production of 2.2 billion doses means that the risk of meeting the short-term goal is about 50%. How can this risk be reduced?

Risk mitigation

Several potential portfolio mitigation strategies were analyzed. They included: (1) Increased portfolio size; (2) Compressed cycle times; (3) POS increase by 10%. Here we expand the number of risk mitigation strategies to guarantee world vaccination in the shortest time and with minimum risk.

Increasing manufacturing productivity

Simulation results presented on Figure 9 indicate that an increase in manufacturing capacity corresponds with only a marginal increase in vaccine production.

Figure 9 legend:The graph shows that increasing manufacturing productivity by up to 20% does not significantly increase production. For example, at the end of 2021, cumulative production will increase from 2.2 to 2.5 billion doses, if manufacturing capacity increases by 20%. X axis is time in months, Y axis is cumulative vaccine production, in millions of doses. Source:Shnaydman, 2020

Therefore, it appears that vaccine manufacturing productivity is not a system bottleneck. Is there a more effective solution? Deeper analysis indicates that R&D portfolio productivity could be a bottleneck. How can R&D productivity be increased?

Scaling of OWS

Companies benefiting from the OWS compressed and overlap clinical trials cycle times are based on significant US government funding. These companies became leaders in the vaccine race, and, if their vaccines are successful, they may produce enough doses to meet worldwide demand. At the same time, the risk of not meeting production goals is still significant (see Figure 8).

How can the vaccine portfolio production risk profile be improved?

Figure 10 legend:The graph shows that 30 collaborative agreements/sponsorships based on governmental and international funding significantly increase LOA of approvals comparing with baseline scenario. Median approvals for collaborative scenario = 9 versus a baseline scenario = 3. Also, vaccines from seven platforms are contributed in a “collaborative” scenario versus vaccines from five platforms in a baseline scenario, due to compression of clinical trials cycle times and at risk execution of Phase III clinical trials. X axis is LOA, Y axis is number of approvals. Source:Shnaydman, 2020

Mitigation scenarios related to various portfolio risk profiles are presented in Figure 11.

Figure 11 legend:The graph shows that mitigation strategy corresponding to increasing number of deals (for example, R&D subsidizing, buy-outs, in-licensing) can significantly reduce production risk when compare to just mitigation strategy related to increasing manufacturing capacity only. X axis is production risk, Y axis is cumulative vaccine production, in millions of doses. Figure 12 indicates that in “collaborative” scenario with 30 deals (for 200+ portfolio candidates) the risk of reaching two billion doses is close to zero; the risk will be reduced from 90% to about 35% for the goal of producing four billion doses. Source: Shnaydman, 2020

Figure 12 shows the dynamics of the production of multiple vaccines.

Figure 12 legend:The graph shows that partnering deals increase vaccines average production volume from 2.4B to 4.5B by the end of 2021 if “partnering” mitigation strategy would be implemented (month 18). Source:Shnaydman, 2020

Production variation increases between months 20 and 40 among the scenarios due to delayed Phase III entrance.

The likelihood of worldwide vaccination in 36 months is high as shown in Figure 14 if global policy is focused on portfolio diversification and financing of the most promising candidates beyond original OWS.

Figure 13. The graph shows that prioritization of vaccine candidates for scaled up OWS can significantly increase likelihood of worldwide vaccination. World vaccination could be accomplished in 36 months with low risk (in case of smooth distribution). For the baseline scenario, likelihood of worldwide vaccination even in 60 months could be only 40%. Source:Shnaydman, 2020

Assessment of economic impact

In order to prove effectiveness of proposed COVID vaccines portfolio diversification strategy, the timing of the production of 11.2 billion doses was estimated by crude extrapolation of curves 1-4 in Figure 13.

In order to do that let’s make several assumptions:

1. World GDP = US$ 87.7 trillion in 2019.²⁹
2. Historical world GDP annual growth is about 3.5%.³⁰
3. World vaccination will start almost immediately after approvals of vaccines (beginning of 2021). Some vaccination delays are possible, but not significant.
4. World economic growth will be on track if about 70% of the world population will be vaccinated.
5. Four scenarios were analyzed. They are marked on Figure 13 as 1 – baseline; 2 – 10 deals; 3 – 20 deals and 4 - 30 deals and increased manufacturing capacity by 20% (most favorable scenario).

Figure 14 illustrates assessment technique.

Figure 14 legend.The diagram illustrates calculation of the World GDP loss for less favorable scenarios than Scenario #4. In 2020, world GDP contracted by about 5% [. In the best scenario, it is assumed that world economy will come back to pre-COVID-19 state in 36 months (world vaccination), and GDP loss=0 comparing to the baseline scenario – 75 months (extrapolated) where GDP loss would be maximal. Source:Shnaydman, 2020

According to the assessment, in order to minimize GDP loss, additional investments should be made to speed up Phase III trials and maximize portfolio productivity. The graph on Figure 15 shows the potential impact of global vaccine development strategies on lost world GDP.

Figure 15 legend. The graph shows impact of vaccine development strategies on potential world GDP loss for different vaccines’ “weight” in world recovery. For example, even if the world vaccination will contribute 20% to the GDP recovery, GDP loss for Scenarios 1-3 will range from ~0.9T to 2.1T comparing to the “best case” scenario. For 40% world vaccination weight global GDP loss could reach $4.2T (if the “best” scenario is not materialized), for US it could be about $1.05T over three years. X axis – time in months, Y axes – Potential world GDP loss. Source:Shnaydman, 2020

ACT Accelerator³¹ estimates that $35 billion is required to meet demand in all COVID-19 related products. If several billion dollars will be invested in OWS scale up, return on investment would be significant.

Conclusion

The paper presents a methodology and modeling results for COVID-19 vaccines portfolio forecasting, including R&D output (rate of approvals and their likelihood at a platform level) and manufacturing production output. Methods of analysis include Monte-Carlo portfolio simulation and a technique for elicitation of probabilities of success (POS) at a platform level. Data was extracted from multiple sources.

Simulation results for a baseline scenario indicate that time to produce enough vaccines for world vaccination is significant despite prioritization of a handful portfolio candidates as a part of Operation Warp Speed (OWS). Limited manufacturing capacity and insufficient portfolio diversification contribute to that.

In order to minimize the time and risk of world vaccination, scaling up of OWS could be very beneficial including additional financing in both Phase III clinical trials and manufacturing for additional vaccines development programs. It would lead to a reduction in global production time for world vaccination from 75 months for baseline scenario to 36 months, reducing potential global GDP loss up to $4.2 trillion (US - $1 trillion) when compared to the baseline scenario.

Vladimir Shnaydman, PhD, is the President of ORBee Consulting

Acknowledgement

The author is very thankful to Michael Quinn.

References

1. Milken Institute. https://milkeninstitute.org/sites/default/files/2020-04/Covid19%20Tracker-4-1-WEB.pdf​
2. www.clinicaltrials.gov.
3. G. Agrawal, M. Conway, J. Heller, A. Sabow, G. Tolub. On pins and needles: Will COVID-19 vaccines ‘save the world’? July 29, 2020, https://www.mckinsey.com/industries/pharmaceuticals-and-medical-products/our-insights/on-pins-and-needles-will-covid-19-vaccines-save-the-world​
4. V. Shnaydman, J. Scannell. COVID-19 Vaccines: Tool to Predict and Manage Global Portfolio Productivity and Risk. Applied Clinical Trials, April 23, 2020. https://www.appliedclinicaltrialsonline.com/view/covid-19-vaccines-tool-predict-and-manage-global-portfolio-productivity-and-risk.
5. J. Scannell, J., V. Shnaydman (2020). How to pick a real winner in the Covid-19 vaccine race | Financial Times. https://www.ft.com/content/3bb6f5d4-8e14-11ea-af59-5283fc4c0cb0​
6. H.A. Levine. Project Portfolio Management. 2005, 538pp
7. Barry R. Bloom, Glen J. Nowak, Walter Orenstein. “When Will WeHave a Vaccine?” — Understanding Questions and Answers about Covid-19. NEJM.https://www.nejm.org/doi/full/10.1056/NEJMp2025331?query=C19&cid=DM98874_NEJM_Registered_Users_and_InActive&bid=262656486​
8. A. McDonnell et al. COVID-19 Vaccine Predictions: Using Mathematical Modelling and Expert Opinions to Estimate Timelines and Probabilities of Success of COVID-19 Vaccines. https://www.cgdev.org/publication/covid-19-vaccine-predictions.
9. Covid-19 Vaccine Delivery Will Present Tough Challenge to Cargo Airlines. https://www.wsj.com/articles/covid-19-vaccine-delivery-will present-tough-challenge-to-cargo-airlines 11601890203?mod=searchresults&page=1&pos=1​
10. Kalipso Chalkidou, Hannah Kettler, Ganesh Ramakrishnan, Rachel Silverman, and Adrian Towse.Leave No One Behind: Using a Benefit-Based Advance Market Commitment to Incentivize Development and Global Supply of COVID-19 Vaccines, May 2020. https://www.cgdev.org/sites/default/files/Chalkidou-Kettler-Benefit-Based-COVID-Vaccine.pdf​
11. Kalipso Chalkidou, Hannah Kettler, Ganesh Ramakrishnan, Rachel Silverman, and Adrian Towse. Development and Global Supply of COVID-19 Vaccines, May 2020 https://www.thinkglobalhealth.org/article/new-way-drive-covid-19-vaccine-development​
12. Kalipso Chalkidou,Hannah Kettler, Ganesh Ramakrishnan, Adrian Towse, Rachel Silvermanand Amanda GlassmanIn the Race to Develop a Vaccine For COVID-19, Is a Pull for R&D Essential or Optional? June 8, 2020. https://www.cgdev.org/blog/race-develop-vaccine-covid-19-pull-rd-essential-or-optional.
13. Tung Thanh Le,Jakob P. Cramer,Robert Chen &Stephen Mayhew Evolution of the COVID-19 vaccine development landscape, NRDD, Sept. 7, 2020. https://www.nature.com/articles/d41573-020-00151-8​
14. V. Shnaydman. Industry Drug Development Portfolio Forecasting: Productivity, Risk, Innovation, Sustainability, Commercial Biotechnology Feb. 2020, vol.25, issue 2, pp.15-25
15. Moncef Slaoui, Matthew Hepburn.Developing Safe and Effective Covid Vaccines - Operation Warp Speed’s Strategy and Approach. NEJM https://www.nejm.org/doi/full/10.1056/NEJMp2027405.
16. Nicole Lurie, Melanie Saville, Richard Hatchett, and Jane Halton. Developing Covid-19 Vaccines at Pandemic Speed. NEJM, May 2020 https://www.nejm.org/doi/full/10.1056/NEJMp2005630​
17. Hay M, Thomas D, Craighead J, Economides C. & Rosenthal J. (2014) Clinical development success rates for investigational drugs. Nature biotechnology, vol. 32, Number 1, January, pp.40-51
18. Clinical Development Success Rates, 2006-2015. Bio.org, BioMedtracker, Ampilion, 2016.
19. Dimitrios Gouglas, Tung Thanh Le, Klara Henderson, Aristidis Kaloudis, Trygve Danielsen, Nicholas Caspersen, Hammersland, James M Robinson, Penny M Heaton, John-Arne Røttingen. Estimating the cost of vaccine development against epidemic infectious diseases: a cost minimization study. www.thelancet.com/Vol 6 December 2018, pp. 1386-1396
20. Lo, A. W., Siah, K. W., & Wong, C. H. (2020). Estimating Probabilities of Success of Vaccine and Other Anti-Infective Therapeutic Development Programs. Harvard Data Science Review. https://doi.org/10.1162/99608f92.e0c150e8​
21. J. A. DiMasiJ. M. ReichertL. FeldmanA. Malins.Clinical Approval Success Rates for Investigational Cancer Drugs. Clinical Pharmacology & Therapeutics(2013); Volume 94, issue 3, 329–335
22. Saaty, T. L. 1977. A Scaling Method for Priorities in Hierarchical Structures, Journal of Mathematical Psychology 15: 234–281. doi:10.1016/0022-2496 (77) 90033-5
23. Saaty, T. L. 1980. The Analytical Hierarchy Process. New York: McGraw-Hill.
24. Saaty, T. L.; Vargas, L. G.; Dellmann, K. 2003. The allocation of intangible resources: the analytic hierarchy process and linear programming, Socio-Economic Planning Sciences 37(3): 169–184. doi:10.1016/S0038-0121(02)00039-3
25. Jonathan Gardner, Ned Pagliarulo, Ben Fidler. The coronavirus vaccine frontrunners are advancing quickly. Here's where they stand. Last updated September 30, 2020 • Published June 9, 2020 https://www.biopharmadive.com/news/coronavirus-vaccine-pipeline-types/579122/​
26. Steve Usdin.Warp Speed’s Slaoui expects two COVID-19 vaccines this year. BioCentury. Sept. 26, 2020. https://www.biocentury.com/article/630710/warp-speed-s-slaoui-expects-two-covid-19-vaccines-this-year-rebuts-sen-warren​
27. Moncef Slaoui, Ph.D., Matthew Hepburn, M.D. Developing Safe and Effective Covid Vaccines - Operation Warp Speed’s Strategy and Approach NEJM August 26, 2020.
28. Sarun Charumilind, Ezra Greenberg,Jessica Lamb, and Shubham Singhal.COVID-19: Saving thousands of lives and trillions in livelihoods. August 17, 2020. https://www.mckinsey.com/industries/healthcare-systems-and-services/our-insights/covid-19-saving-thousands-of-lives-and-trillions-in-livelihoods​
29. WORLD ECONOMIC OUTLOOK REPORTS. World Economic Outlook Update, June 2020. A Crisis like No Other, An Uncertain Recovery. IMF 2020https://www.imf.org/en/Publications/WEO/Issues/2020/06/24/WEOUpdateJune2020​
30. World GDP Growth Rate 1961-2020. https://www.macrotrends.net/countries/WLD/world/gdp-growth-rate. 
31. An economic investment case & financing requirements. ACT accelerator. September 2020 – December 2021. 30p. https://www.who.int/publications/i/item/an-economic-investment-case-financing-requirements​