Phase III Trial Failures: Costly, But Preventable

Thomas Henry Huxley (1825-1895) once stated, “The great tragedy of science-the slaying of a beautiful hypothesis by an ugly fact.”  Indeed, in drug development not all ground-breaking scientific ideas translate into miracle cures. Biopharmaceutical companies around the globe strive to develop novel therapies to fulfill unmet medical needs, yet these efforts all too often die in development. Despite the numerous scientific, technological, and operational advances in R&D that would be expected to increase the efficiency and success of drug development, a significant number of clinical trials still fail to produce new, effective, and safe medicines.

Approximately 70% of Phase II trials are unsuccessful.^1,2 Although this percentage might seem high, failure of early-phase trials is expected to some extent, as these trials are “exploratory,” “proof of mechanism,” and “proof of concept” trials in patients.³ What is unexpected, however, is the percentage of “confirmatory” Phase III trials that fail-about 50%.^1,2 Theoretically, if early-phase trials provide the necessary criteria for moving a drug program to Phase III testing, relatively few Phase III trials should fail; but that is not the case. 

In a recent PAREXEL analysis,⁴ we collected and evaluated data on 38 Phase III failures from mid-2012 through 2015 from a variety of publicly available sources (It is important to note that this list may not be exhaustive and may not include all of the Phase III trials that failed in this time period). As indicated in Table 1 below (click to enlarge), these trials failed to meet primary or secondary efficacy endpoints; Phase III trials that failed due solely to safety issues are not included. These 38 failed trials collectively enrolled nearly 150,000 patients.

Peeling the onion: What are the drivers behind these Phase III failures?

Several groups have sought to analyze data from failed clinical trials to uncover the underlying trends and potential drivers of Phase III failures. Selected trends are discussed below. 

1. A study by the Tufts Center for the Study of Drug Development (CSDD)⁵ evaluating clinical trials from 2000 to 2009 found that the three
 most common reasons that drugs or trials fail in Phase III of development are:

• Efficacy (or rather lack thereof) - i.e., failure to meet the primary efficacy endpoint 

• Safety (or lack thereof) - i.e., unexpected adverse or serious adverse events 

• Commercial / financial - i.e., failure to demonstrate value compared to existing therapy

​

2. New molecular entities (NME) versus all drugs. A study by BioMedTracker (BMT) and the Biotechnology Industry Organization (BIO) evaluated R&D projects involving more than 9,500 different drug

and biological products from 2004 to 2014.¹ In each phase analyzed, the failure rates of trials involving NMEs were higher than trials involving all drugs, and the difference was greatest between Phase III development and submission of regulatory applications (See Figure 2). Among the Phase III trials analyzed, only 61% of NME trials succeeded in moving to the application phase (39% failure rate), whereas 67% of all drug trials moved to the application phase (33% failure rate). The high failure rate of Phase II trials reported in that analysis (62% and 67%, respectively) is not unexpected for exploratory trials.

3. Failure rates differ by therapeutic area. The BMT/BIO study also found that clinical trial failure rates differ by therapeutic indication.¹ Among the Phase III trials evaluated, oncology and cardiovascular trials had the highest

failure rates (data not shown). Comparing failures for oncology and non-oncology trials specifically, failure rates for these indications differed more significantly at later stages of development (See Figure 3 at right). The oncology trials failed more often than non-oncology trials during the Phase II-to-Phase III transition and, in particular, from Phase III testing to regulatory submission (48% failure rate for oncology trials vs. 29% for non-oncology trials). The higher failure rate for oncology trials might be due to the inclusion of survival endpoints and the need to show efficacy by an improvement in overall survival.⁶

4.Failure rates differ by type of molecule. A study by the Tufts CSDD found that the probability of success for clinical trials of small molecules is lower than for trials of large molecules (see Figure 4 below).⁷ Among the Phase III trials

analyzed, only 61% of studies involving small molecules progressed from Phase III testing to the regulatory application phase (39% failure rate). Trials of large molecules, however, were more successful in moving to the application phase, with a success rate of 74% (26% failure rate). The study also found that within the large-molecule subtypes, failure rates for recombinant proteins and monoclonal antibodies were similar overall, but recombinant proteins failed more often than monoclonal antibodies in the transition from Phase III testing to regulatory review (34% failure rate for recombinant proteins vs. 13% for monoclonal antibodies; data not shown).

The European Center for Pharmaceutical Medicine (ECPM) organized a seminar in September 2014 titled “Why Clinical Trials Fail?” Based on presentations delivered by various experts at the seminar, we have grouped the drivers behind Phase III failures identified by these experts into six categories: basic science, clinical study design, dose selection, data collection and analysis, operational execution, and other. Table 2 below provides examples of specific events in each category that could lead to Phase III failure.

Impact of Phase III failures: Human and financial 

The effects of Phase III failures are significant, with the most poignant being the impact on human lives and financial cost. First and foremost is the negative impact on the thousands of patients who enroll in clinical studies hoping to find a viable treatment option. As mentioned previously, PAREXEL’s analysis revealed that in a relatively narrow time frame (2012-2015), nearly 150,000 patients had enrolled in Phase III trials that eventually failed. Most of these patients had cardiopulmonary diseases, diabetes mellitus, or cancer.⁴ Patients who participate in clinical trials often have difficult-to-treat or later-stage diseases with few, if any, standard treatment options. For many of these patients, time may be running out, and a clinical trial is the last option for a potential cure or prolongation of life. Thus, the impact of failed drug trials can be widespread and may worsen patient prognosis, quality of life, and emotional well-being.

Not surprisingly, failures of large Phase III trials also result in a material financial burden to biopharmaceutical companies and are a contributor to poor capital productivity. Considering the substantial financial investment in R&D, few new drugs are being approved by regulators. One study found that for every billion U.S. dollars spent on R&D, the number of new drugs approved has decreased by approximately 50% every nine years since 1950 (inflation-adjusted).⁸ And, the financial consequences of failed drug development often go beyond R&D. An examination of the most impactful Phase III failures of 2014 found that these failures resulted in the termination of drug programs and employees as well as financial losses for investors, as shown in Table 3 below (note that only some of these failures are also listed in Table 1). With slower growth in R&D spending,⁹ it is important that biopharmaceutical companies use the limited funds and resources efficiently and effectively to get drugs to market and provide new medicines that meet unmet medical needs.

Selected strategies to reduce the risk of late-stage failure

Although no simple solution exists yet for preventing Phase III failures to confirm the efficacy of a new drug or biologic, several approaches have been and are currently being developed to reduce the risk of such failure. Given our unique perspective in working with hundreds of companies across thousands of clinical trials and compounds, we and numerous colleagues at PAREXEL are exposed to these approaches on a daily basis and briefly highlight some of these in the next section.  These strategies can be implemented during the entire development process, in specific phases of development, and/or during clinical trial design. 

Applying more rigor to the overall development process

Over the past few years, most drug development companies have established and adopted more disciplined protocol, progress and portfolio review frameworks. In 2011, AstraZeneca aimed to overhaul its R&D process to improve the health

of the organization and increase the chance of success of its Phase III trials. By evaluating its small-molecule drug projects over a five-year period (2005–2010), AstraZeneca identified the factors associated with project success and developed a framework that now drives its development process. The 5R framework guides R&D teams in identifying the right target, the right tissue, the right safety, the right patients, and the right commercial potential (See Figure 5). Having the right culture-i.e., a culture where the facts and data are confronted with brutal honesty, and courageous kill-early decisions are encouraged where appropriate-is an important, additional dimension to this framework.¹⁰ Applying more rigor and discipline to the development process holds potential for weeding out likely failures earlier in the process, thereby reducing Phase III failure rates.

Adequate Phase II testing

Many Phase III trials fail because of a fundamental lack of understanding of the mechanisms of action of NMEs. To address this issue and improve the chance of NMEs being successful during Phase II testing, Pfizer performed an analysis of

44 Phase II programs during a four-year period (2005-2009) to identify factors associated with success.¹¹ The result of that analysis led to the “Three Pillars of Survival” framework that helps the R&D teams determine three key elements that increase the likelihood of an NME surviving Phase II testing and moving on to Phase III testing: Pillar 1 involves exposure of the drug to the target site of action; Pillar 2 involves binding of the drug to the target; and Pillar 3 involves expression of pharmacology (see Figure 6). Rushing to get to Phase III without adequate understanding of these three critical components is risky and more often leads to unpleasant surprises in Phase III.  Not only does this proposed approach in Phase II make intuitive sense, but we also believe that it holds potential for reducing late stage failures.

Optimized Phase III Clinical Trial Design

Flaws in clinical trial design are a major driver of Phase III failures. Several strategies have been developed for optimizing trial design. The next section aims to highlight a few of them:

• More rigorous protocol review and optimization is an approach that many companies are employing in varied degrees ranging from stage gate reviews and sign-offs by various internal committees to live, in-practice simulations of protocols to identify and resolve potential glitches proactively. Typically, a number of trade-off decisions need to be made in the compilation of most Phase III protocols-e.g., will the protocol include a specific secondary objective that has commercial value but potentially prolongs the duration of the study? Identifying and quantifying the impact of these trade-offs is helpful in designing better protocols. Likewise, in the right capable hands, the vast amount of data available in public sources (e.g., Clinicaltrials.gov) can often be harnessed to determine what worked well and what did not.

• Modeling and simulation have contributed to regulatory approval and labeling decisions in recent years and are currently being employed as a potential solution for mitigating late-stage failure risk.¹² In a review of 198 applications submitted to the FDA between 2000 and 2008, modeling and simulation was found to provide pivotal or supportive insights into effectiveness and safety that contributed to the approval of 126 applications.¹³ Additionally, it provided information on dosage, administration, and safety that supported 133 labeling decisions. The FDA encourages sponsors to incorporate quantitative modeling and trial simulation in the drug development plan and to seek regulatory guidance on these strategies through participation in an end-of-phase 2A (EOP2A) meeting.¹⁴ The EOP2A meeting provides the sponsor with an opportunity to discuss modeling and simulation plans and receive feedback from the FDA on how to optimally implement these strategies to quantify the exposure-response relationships and select appropriate doses for Phase III trials. Modelling and simulation is currently being used more broadly than just for the selection of an optimal dose. Modeling and simulation of various clinical trial designs is one such example that is being used in an effort to design optimal Phase III studies.

• Adaptive designs are gaining momentum as a way to reduce failure risk. This is, in large part, due to the fact that these adaptive designs provide an opportunity to assess interim data and to sense-check some of the initial uncertainties or assumptions that were made at the outset of the trial.^15,16 More importantly, these provide an opportunity to change course during a trial and to correct these incorrect assumptions in a prospectively planned manner that does not jeopardize statistical validity and the operational integrity of the trial.¹⁷

• Biomarkers are being used increasingly to assess efficacy in a rapid and more objective manner.¹⁸ A particular challenge associated with the use of biomarkers in clinical trials is the requirement to validate these biomarkers as relevant disease-modifying endpoints and correlate changes in these endpoints to clinically significant changes in disease progression.¹⁹ Likewise, enrichment strategies often involving genotyping are being used increasingly to optimize the proposed study population to those individuals most likely to respond to the treatment under investigation.²⁰

De-risking study execution

A number of approaches are currently being used to mitigate study execution risks. These include:

• Leveraging data from a variety of sources (e.g., electronic health records [EHR], prior site performance, central labs, etc.) to ensure that the selection of patients, sites, and countries are optimal. The impact of EHRs on clinical trials is carefully tracked by numerous interested parties, and it is not difficult to imagine the potentially transformative impact that these might have on clinical trials. At the same time, many industry insiders hold a somewhat skeptical view-especially about the accuracy of data in these systems.

• Ongoing surveillance of the quality of data being collected during a Phase III trial and tying this to a properly planned risk-based monitoring protocol allows companies to set certain quality triggers that would result in increased monitoring when and where needed. This is helpful in reducing execution risk, as errors at the site level are often discovered too late in the process-i.e., once the study is clinically completed and the database is locked. Often, at this time, very little can be done to salvage the situation and improve the quality and potential for success in the trial. 

Completeness and clarity of submissions and interaction with regulatory agencies

Many drugs fail to get approved because the information submitted to the regulatory agency is insufficient to allow for a determination on safety and efficacy, not because the drug is actually unsafe or ineffective. A review of marketing applications for NMEs submitted to FDA from 2000 to 2012 revealed that about 50% of those reviewed failed to obtain approval during the first-cycle.² However, nearly 50% of these failed applications were eventually approved on resubmission after applicants addressed the FDA’s concerns related to safety, manufacturing, and labeling. Delayed approvals and non-approvals may be reduced or prevented by seeking advice from regulators during the drug development process. The FDA offers sponsors and applicants the opportunity to schedule formal meetings at critical points in the development and regulatory process to discuss the plan and identify areas of concern that may need adjustment.²¹

Other bold/promising/theoretical/philosophical strategies that could reduce Phase III failures

Other areas that are currently being explored and could hold potential in reducing Phase III failure risk include: 

• Replacement of the current gold standard, the randomized controlled trial, with real-world evidence

• Wearable devices that collect real-time data²²

• Adaptive licensing

• Next-generation sequencing and improved understanding of the genetic basis of disease

• Basket/master protocols

Although Phase III failures cannot be eliminated, the risk of these failures can be reduced. The strategies discussed in this article may provide sponsors with some of the tools aimed at minimizing the risk of failure in their drug development plans. 

Conclusion

We believe that the current failure rate in Phase III studies is unacceptably high, and that industry is keen on reducing this failure risk, although some in industry may believe that failure is the price to be paid for innovation. As a first step, it is important to understand the reasons and root causes driving these failures. Our research identified recently failed Phase III studies that have enrolled nearly 150,000 patients.  Based on data from our analysis and others, we have listed the main reasons why Phase III trials fail. In addition, and given our unique perspective in working with hundreds of sponsors across thousands of trials, we have highlighted some of the approaches that pharmaceutical companies are implementing in an effort to reduce these costly late-stage failures. Along with our colleagues in the pharmaceutical industry, we are optimistic about the potential of some or all of these approaches to improve the Phase III success rate.

Alberto Grignolo, PhD, is Corporate VP, Global Strategy, PAREXEL; Sy Pretorius, MD, is Chief Scientific Officer, PAREXEL

Acknowledgements: Dawn Giles from PAREXEL International and Jennifer Swanson from JS Medical Communications, LLC, contributed to the development of this article by providing assistance with research, writing, and editing.

References

1. Stopke E, Burns J. New drug and biologic R&D success rates, 2004-2014. PAREXEL’s Bio/Pharmaceutical R&D Statistical Sourcebook 2015/2016.

2. Sacks LV, Shamsuddin HH, Yasinskaya YI, et al. Scientific and regulatory reasons for delay and denial of FDA approval of initial applications for new drugs, 2000-2012. JAMA. 2014;311(4):378-84.

3. ICH Expert Working Group. General Considerations for Clinical Trials E8. 1997. Available at: http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/ Efficacy/E8/Step4/E8_Guideline.pdf. 

4. PAREXEL. Independent analysis of Phase III trial failures 2012–2015. Unpublished data.

5. Tufts CSDD. Causes of clinical failures vary widely by therapeutic class, phase of study. Tufts CSDD study assessed compounds entering clinical testing in 2000-09. Tufts CSDD Impact Report. 2013;15(5).

6. Khozin S, Liu K, Jarow JP, Pazdur R. Regulatory watch: Why do oncology drugs fail to gain US regulatory approval? Nat Rev Drug Discov. 2015;14(7):450-1.

7. Tufts CSDD. Comparative success rates for self-originated/licensed-in and small/large molecule drugs: a 2010 analysis. PAREXEL’s Bio/Pharmaceutical R&D Statistical Sourcebook 2010/2011.

8. Scannell JW, Blanckley A, Boldon H, Warrington B. Diagnosing the decline in pharmaceutical R&D efficiency. Nat Rev Drug Discov. 2012;11(3):191-200.

9. EvaluatePharma. World preview 2018: embracing the patent cliff. http://www.evaluategroup.com/public/EvaluatePharma-World-Preview-2018-Embracing-the-Patent-Cliff.aspx. 

10. Cook D, Brown D, Alexander R, et al. Lessons learned from the fate of AstraZeneca’s drug pipeline: a five-dimensional framework. Nat Rev Drug Discov. 2014;13(6):419-31.

11. Morgan P, Van Der Graaf PH, Arrowsmith J, et al. Can the flow of medicines be improved? Fundamental pharmacokinetic and pharmacological principles toward improving Phase II survival. Drug Discov Today. 2012;17(9-10):419-24.

12. Stone JA, Banfield C, Pfister M, et al. Model-based drug development survey finds pharmacometrics impacting decision making in the pharmaceutical industry. J Clin Pharmacol. 2010;50(9 Suppl):20S-30S.

13. Lee JY, Garnett CE, Gobburu JV, et al. Impact of pharmacometric analyses on new drug approval and labelling decisions: a review of 198 submissions between 2000 and 2008. Clin Pharmacokinet. 2011;50(10):627-35.

14. FDA. Guidance for Industry End-of-Phase 2A Meetings. 2009. Available at: http://www.fda.gov/downloads/DrugsGuidanceComplianceRegulatory Information/Guidances/ucm079690.pdf. 

15. Krams M, Dragalin V. Considerations and optimization of adaptive trial design in clinical development programs in adaptive trials in: He W, Pinheiro J, Kuznetsova OM (Eds.). Practical Considerations for Adaptive Trial Design and Implementation, Statistics for Biology and Health. Springer Science+Business Media. New York, New York. 2014.

16. Gallo P, DeMets D, LaVange L. Considerations for interim analyses in adaptive trials, and perspectives on the use of DMCs in: He W, Pinheiro J, Kuznetsova OM (Eds.). Practical Considerations for Adaptive Trial Design and Implementation, Statistics for Biology and Health. Springer Science+Business Media. New York, New York. 2014.

17. Dragalin, V. Adaptive designs: terminology and classification. Drug Information Journal. 2016;40:425-435.

18. Marrer E, Dieterle F. Promises of biomarkers in drug development--a reality check. Chem Biol Drug Des. 2007;69(6):381-94.

19. Kelloff GJ, Bast RC Jr, Coffey DS, et al. Biomarkers, surrogate end points, and the acceleration of drug development for cancer prevention and treatment: an update prologue. Clin Cancer Res. 2004;10(11):3881-4.

20. Kelloff GJ, Sigman CC. Cancer biomarkers: selecting the right drug for the right patient. Nat Rev Drug Discov. 2012;11(3):201-14.

21. FDA. Formal Meetings Between the FDA and Sponsors or Applicants of PDUFA Products Guidance for Industry. 2015. http://www.fda.gov/downloads/Drugs/GuidanceCompliance%20Regulatory Information/Guidances/UCM437431

22. Kumar S, Nilsen WJ, Abernethy A, et al. Mobile health technology evaluation: the mHealth evidence workshop. Am J Prev Med. 2013;45(2):228-36.