Employing the Power of Bayesian Methods to Expedite Learning

Developed in the 18th century by the Reverend Thomas Bayes, the essence of the Bayesian method is to update our beliefs about hypotheses as data accumulates. This approach allows every new piece of data to serve as evidence to update a hypothesis. Bayes’ rule then consists of a “prior” either based on evidence already collected, or scientific findings in the case of early-phase trials (i.e., a prior can be both informative and non-informative). A rule then explains how to update these priors in order to make sense of newly collected evidence. A “posterior” is then the result of the prior being updated in light of this new evidence.

Touted for their ability to create flexible trial designs and accelerate learning while doing so, Bayesian methods have played a key role in transforming clinical research in therapeutic areas from oncology and rare diseases, where they protect patients by efficient modeling, to medical device trials, where the gains in effectiveness can be quickly calculated using Bayes’ rule. Bayesian methods are also popularly used for interim analyses, where Bayesian predictive power calculates the probability of reaching a specific trial outcome based on the data at hand. They can also be used in a variety of ways for performing meta-analyses, for integrating historical data into a trial, and for combining insights from multiple data sources. They have also begun to play an important part in infectious disease trials for epidemics and pandemics like Ebola and COVID-19, where the demand for fast learning and flexible designs comes together.

This article reviews the main Bayesian methods used in clinical trial design. After a short history of Bayesian and Frequentist methods, we delve into Bayesian methods for early-phase and late-phase designs. We highlight the fact that Bayesian methods can be useful for trials that have both Bayesian and Frequentist designs, and that ultimately the decision to use Bayesian methods depends on the specific context of the trial in question. Sound trial design requires understanding the specific research question, and having a sense of the program-level strategy as necessary to drive scientifically rigorous conclusions under resource constraints. There is no firm rule on when to use Bayesian and Frequentist methods, but knowledge of both can result in informed decision-making about a specific trial’s design in order to generate sound statistical results under resource- and time-constraints.

Brief history of the status and uses of the p-value

Bayesian statistics have a much older history than Frequentist methods, but their use within the clinical development setting has taken time to establish. Frequentist methods became the norm for drug development in the 1960s and ’70s, mostly because it was easier to undertake the calculations and computations necessary for Frequentist statistics without the use of fast or powerful computers. Even today, the number of simulation and modeling tools necessary to perform Bayesian computations requires statisticians to be well- resourced technologically.

As, historically, Frequentist methods have been easier to use, this has resulted in the widespread use of Frequentist concepts like p-values and type 1 error rates as industry norms for measuring statistical rigor. Unlike Bayesian probabilities, which aim to tell us the probability of a statement or hypothesis given the data, the Frequentist p-value tells us the likelihood of seeing the data before us if a certain hypothesis were not true. A low p-value then means it would be highly coincidental if the hypothesis were false while the data that appears to confirm the hypothesis is true. The type 1 error rate then conveys the probability that we rejected a null hypothesis when it is true (false positive).

Frequentist designs can often require higher sample sizes than Bayesian methods, and are considered by some to be less flexible and less intuitive. Others point out that the decision to use Frequentist or Bayesian designs is a matter of context. Both methods have particular strengths, in varying situations. Therefore, this paper considers both Bayesian methods used for Bayesian trials, as well as examples of how Bayesian methods can strengthen insights of trials that are Frequentist.

Early-phase Bayesian designs

Bayesian designs are frequently used in early-phase trials due to their flexibility and efficiency. For example, as participants in what are typically first-in-human trials, there is very little known about the response of a drug within the human body, especially regarding toxicity. Therefore, patients and sponsors benefit from a trial design that enables frequent interim looks, opportunities for adjusting doses, and stopping for futility when required.

Popular Bayesian dose-finding approaches like the continual reassessment method (CRM), the Bayesian logistic regression method (BLRM), and the modified toxicity probability interval (mTPI) employ Bayesian methods, modeling, and decision making to achieve two goals: proper patient allocation to safe and efficacious doses during the trial and efficient estimation of the maximum tolerated dose (MTD) at the end of the study. Parametric Bayesian models, with or without a dose-response curve, provide the foundation for statistical inference in these designs, which increase the probability of finding the MTD as well as assigning patients to better doses.

After dose-finding, Bayesian designs can be applied in Phase Ib expansion cohort trials and Phase II trials with one or more treatment arms. Recent developments in oncology drug development have provided great opportunities for Bayesian designs, such as MUCE, to improve the Phase Ib multiple expansion trials.¹ Phase II Bayesian designs using hierarchical models allows basket trials to efficiently assess the efficacy of a therapeutics in multiple disease indications.² Other usage of Bayesian designs includes the Bayesian continuous monitoring in single-arm Phase II trials, as well as Bayesian dose-ranging designs for non-oncology drug development.^3,4 A recent review can be found in Tsimberidou, Müller and Ji (2020).⁵

Bayesian methods in early-phase Frequentist designs

While several Bayesian methods exist for early-phase trials, Bayesian tools can also be used to prepare or augment trials that make use of Frequentist designs. When designing a Frequentist trial, the use of historical data has become much more common as a starting point for developing hypotheses, estimands, and designs. Historical data may also be useful when a concurrent control is not feasible for ethical or practical reasons. A variety of approaches to using historical data exist, through which data from an external source can be incorporated into the analysis of the current trial and act as a guide to update the current data.

The initial step in such approaches involves the identification of relevant data from previous or ongoing studies. These data must be selected carefully, and one must determine whether they are exchangeable with data from the current trial. They are rarely exchangeable at the patient level, although exchangeability at the aggregate data level may be more realistic. Common methods for the incorporation of historical information include commensurate priors, power priors, meta-analytic predictive (MAP) priors, and others.

The latter MAP approach derives a prior distribution from a random-effect meta-analysis of historical data via Markov Chain Monte Carlo (MCMC) algorithms. The prior distribution is represented by a parametric mixture whose components are determined by the use of an expectation-maximization algorithm. One can also increase robustness of the parametric MAP by adding a weakly-informative prior component to the mixture derived from the previous steps. The degree of how informative the MAP prior is can be assessed with its effective sample size, which provides a rough guide of how much the sample size can be reduced when using the respective frequentist power calculation as a reference.

At the program-level or portfolio-level, Bayesian model-based meta-analyses can also play a role to determine which of many candidate therapies ought to proceed to the next stage of clinical development. They can also anticipate what data will be needed for later stages of trial design, making them invaluable for seamless studies or strong Phase III trials.

A further use of Bayesian methods involves the calculation of benefit-risk profiles, usually conducted after safety studies are complete. When making such a calculation, the ethical standard should take into consideration every piece of data available at our disposal, from historical data from other trials, to the effects of the current therapy on various endpoints. Bayesian methods enable trial sponsors to collate these various forms of data into one benefit-risk calculation.

Late-phase Bayesian designs

Recent publications in medical literature have illustrated several advantages of Bayesian methods for designing and analyzing confirmatory trials. The recently published case study redesigning the Phase III OSCAR (High Frequency OSCillation in Acute Respiratory Distress Syndrome (ARDS)) trial using Bayesian methods showed that the Bayesian sequential design could shorten the trial duration by 15 to 40 weeks and recruited 231 to 336 fewer patients.⁶ Authors of another trial illustrated that by using Bayesian designs, the trial could have declared adrenaline superior for 30-day survival with 1,500 fewer patients.⁷

The advantages of Bayesian analysis have also been shown by post-hoc Bayesian analysis of two recent high-profile trials, EOLIA and ANDROMEDA-SHOCK. Both studies demonstrate a large clinically important mortality difference but failed to reach p-value ≤ 0.5.8 These analyses show that the Bayesian analysis helps inform interpretation of the study beyond Frequentist’s binary “positive or negative” delineation.^8,9,10

Bayesian methods help optimize trials at several levels. Optimizing interim monitoring by enabling more efficient and informative decision-making as early as possible is one of the key advantages. Within the Frequentist framework, interim analyses usually evaluate the treatment benefit based on completed patients. However, a more relevant question to answer at interim analysis would be whether a trial is likely to reach a definitive conclusion by the end of the study. Futility is inherently a prediction problem and, therefore, best addressed by predictive probability than by p-value.¹¹

Bayesian methods also offer advantages for efficacy monitoring. The predictive probabilities formalize the decision to stop accrual in a manner consistent with the decision process of the data monitoring committee (DMC). Another important advantage of Bayesian methods is the ability to incorporate partial data into the estimation of predictive probabilities, especially for trials requiring long follow-up periods.¹¹ Although partial information would not be used for regulatory submission purposes, the use of incomplete information can help to de-risk interim futility or efficacy.

Conclusion

Here we have highlighted the need for more Bayesian methods employed in pharmaceutical development, both for design and analysis. The array of Bayesian methods is vast, and with computing power now more easily accessible to a wide range of statisticians, these methods are perpetuating the trial design space. From early-phase trials to late-phase development, utilizing Bayesian tools can expedite and/or de-risk trials, even when used to augment a Frequentist framework. They are critical for learning quickly and employ flexible strategies for sound scientific outcomes.

There are statisticians who insist that Bayesian methods are always an improvement on Frequentist methods, and vice versa. At a practical level, though, a knowledgeable quantitative strategist should be able to pick and choose methods from both these paradigms that serve the needs of a specific sponsor.

Therefore, it is always important to partner with a team that has strong working knowledge of both paradigms, along with regulatory knowledge of how to combine these methods acceptably.

References

1. Berry D, Müller P, Grieve A et al. Adaptive Bayesian Designs for Dose-Ranging Drug Trials. Case Studies in Bayesian Statistics Volume V. 2002;162:99-181. doi:10.1007/978-1-4613-0035-9_2
2. Berry S, Broglio K, Groshen S, Berry D. Bayesian hierarchical modeling of patient subpopulations: Efficient designs of Phase II oncology clinical trials. Clinical Trials: Journal of the Society for Clinical Trials. 2013;10(5):720-734. doi:10.1177/1740774513497539
3. Goligher E, Tomlinson G, Hajage D et al. Extracorporeal Membrane Oxygenation for Severe Acute Respiratory Distress Syndrome and Posterior Probability of Mortality Benefit in a Post Hoc Bayesian Analysis of a Randomized Clinical Trial. JAMA. 2018;320(21):2251. doi:10.1001/jama.2018.14276
4. Harhay M, Casey J, Clement M et al. Contemporary strategies to improve clinical trial design for critical care research: insights from the First Critical Care Clinical Trialists Workshop. Intensive Care Med. 2020;46(5):930-942. doi:10.1007/s00134-020-05934-6
5. Ji Y, Tsimberidou A, Müller P. Innovative trial design in precision oncology [published online Oct. 3, 2020]. Semin Cancer Biol.
6. Lee J, Liu D. A predictive probability design for Phase II cancer clinical trials. Clinical Trials: Journal of the Society for Clinical Trials. 2008;5(2):93-106. doi:10.1177/1740774508089279
7. Lyu J, Zhou T, Yuan S, Guo W, Ji Y. MUCE: Bayesian Hierarchical Modeling for the Design and Analysis of Phase 1b Multiple Expansion Cohort Trials. arXiv.org. https://arxiv.org/abs/2006.07785. Published 2020. Accessed Oct. 21, 2020.
8. Ryan E, Brock K, Gates S, Slade D. Do we need to adjust for interim analyses in a Bayesian adaptive trial design? BMC Med Res Methodol. 2020;20(1). doi:10.1186/s12874-020-01042-7
9. Ryan E, Bruce J, Metcalfe A et al. Using Bayesian adaptive designs to improve Phase III trials: a respiratory care example. BMC Med Res Methodol. 2019;19(1). doi:10.1186/s12874-019-0739-3
10. Saville B, Connor J, Ayers G, Alvarez J. The utility of Bayesian predictive probabilities for interim monitoring of clinical trials. Clinical Trials: Journal of the Society for Clinical Trials. 2014;11(4):485-493. doi:10.1177/1740774514531352
11. Zampieri F, Damiani L, Bakker J et al. Effects of a Resuscitation Strategy Targeting Peripheral Perfusion Status versus Serum Lactate Levels among Patients with Septic Shock. A Bayesian Reanalysis of the ANDROMEDA-SHOCK Trial. Am J Respir Crit Care Med. 2020;201(4):423-429. doi:10.1164/rccm.201905-0968oc

Yannis Jemiai, Chief Scientific Officer, Cytel, Natalia Muhlemann, Vice President of Strategic Consulting, Cytel, Rajat Mukherjee, Research Fellow, Cytel, Pantelis Vlachos, Principal Statistician, Cytel, Apurva Bhingare, Strategic Consulting Analyst, Cytel, Yuan Ji, Professor, University of Chicago