Operation Adaptive

It's well recognized that lengthy and expensive clinical trials, high failure rates, and increased patient safety concerns collectively make the economics of drug development more challenging than ever before. Part of the remedy is likely to be a heterogeneous group of protocol designs, collectively termed "adaptive clinical trials." The common thread between all adaptive trial designs is the use of accumulating data to make modifications while the trial is still underway, increasing the likelihood of selecting the right dose (or endpoint) for the right patient population earlier in a development program.

PHOTOGRAPHY: GETTY IMAGES

Adaptive trials are adaptive by design, and they require a wholesale change in the way drug development has traditionally been carried out.^¹ The entire development of a therapeutic candidate, not just the next trial phase, needs to be mapped out in advance. More, and earlier, operational planning is a prerequisite to conducting adaptive trials, which may require a major culture shift at some companies.

Just as the technology enabling the adaptive trials themselves has come of age, so too will organizations need to move beyond traditional tools and approaches for the operational planning, forecasting, and budgeting of adaptive trials. The planning spreadsheet, a tool commonly used to estimate budget needs for clinical trials, does not provide the flexibility or granularity to quickly and accurately model and compare the many potential scenarios for an adaptive study. New, more sophisticated approaches are needed.

Fundamentals

The rationale for conducting adaptive trials is that they can increase the probability of treatment success, identify ineffective drugs sooner, and require fewer patients, or fewer patients at less informative doses—all without compromising the trials' validity and integrity. They utilize learn-and-confirm concepts, whereby results get substantiated in an ongoing manner rather than waiting for a phase to complete. Examples of adaptations that might be made midstudy are changes in eligibility criteria, sample size, randomization, treatment regimens, and even the primary endpoint.^²

Adaptive trials require the use of prospective statistical simulation to optimize the trial design, and the criteria for making adaptations (modifications) during the study must be defined prior to the study start.

In order to control for potential bias once an adaptive study is underway, an independent Data Monitoring Committee (DMC) will usually be established to review planned or pending trial adaptations, including verifying their face validity and making recommendations concerning further conduct of the trial, in addition to its traditional duties.

A separate independent body, with expertise in statistics, also needs to be assembled to prepare data, conduct interim analyses of unblinded data, and prepare briefing documents for discussion with the DMC. The current randomization algorithm "in play" must be known only to unblinded personnel, and any sponsor involvement on the DMC requires careful consideration due to the increased potential for bias.

Adaptive trials will often be more expensive than similarly sized conventional studies, but the long-term savings over the course of a development program can be substantial. Savings can come from avoidance of future studies that are no longer necessary, reduced lag time between phases, and the need for fewer subjects. Ultimately, an adaptive trial may result in a product getting registered sooner.

Importantly, an adaptive clinical trial is not always necessary, and may not be viable. In some cases, enough is known about the patient population and the investigational drug to make an adaptive design unnecessary. In others, outcomes that can trigger an adaptive response won't be observable early enough in treatment in order to "adapt."³

From adaptive trials that have been successfully completed to date, a general guideline has become apparent: The recruitment duration needs to be at least four times longer than the observational period in order to preserve the opportunity to make adaptations and prevent the need to pull back on enrollment. If the observable endpoint is too far out, it's not worth trying to do an adaptive trial unless a biomarker or other predictor of the outcome is available that shortens the time frame.

Key Operational Planning Pointers

The importance of planning

Upfront planning and strong project management are critical to the effective execution of an adaptive trial because, relative to a traditional trial, data needs to be more quickly captured and processed. New players need to get pulled into the planning process, and input from the functional areas usually involved, such as biometrics, clinical research, data management, drug supply, and clinical operations, is required earlier.

A substantial amount of time and effort must be devoted to performing simulations of patient response and preparing the simulation report that summarizes the study operational characteristics, in addition to the complex logistics involved in setting up the appropriate IT infrastructure (e.g., EDC, IVRS, algorithm host), which is critical to ensuring that data are available and adaptations made as efficiently as possible.

Additional upfront time is also needed for coordinating randomization and drug inventory. Creative dosing and packaging strategies are often necessary to ensure that the available drug supply is sufficient for the study. For example, for studies involving a large range of multiple doses, supplies may be used more efficiently by adding pills in lieu of creating more pill strengths.⁴

Planning should begin at the drug development plan stage, long before the study actually occurs. Regulatory authorities are receptive to adaptive trial designs, but must be consulted early in the development of a product to ensure that the methods utilized are appropriate.

Figure 1. High-level comparison of timelines for 12 different potential study scenarios.

For sponsors, obtaining internal buy-in may be the tougher task. Adaptive studies require significantly more effort in the planning stage, and will generally cost more than a similarly sized conventional study. In addition, successful implementation also requires specialized knowledge that many companies do not yet have in-house, which means that outside consultants and vendors must be identified and hired. Finally, although the need for innovative approaches to increase drug development efficiency is recognized by all, companies have successfully obtained marketing approval for drugs for decades using conventional study designs.

Until official regulatory guidance is available, and the companies at the forefront of using this approach begin to reap its benefits, many sponsors may choose to stick with the tried and true methods of drug development, in spite of the advantages that adaptive designs may offer.

Forecasting and budgeting

For conventional studies, single point estimates are usually developed for the budget, headcount, and timeline—for example, the study will cost $15.2 million, require "X" number of personnel, and the final report will be signed off on December 15, 2011. However, adaptive trials require a flexible, adaptive planning approach to forecasting and budgeting. Because of their very nature, single point estimates that represent the most likely budget, headcount, and timeline must always be associated with a range of possibilities when prepared and submitted for review and approval. Of primary importance for planning, then, is the ability to easily model and compare multiple operational scenarios based on the predefined adaptation "checkpoints" built into the trial design, as well as the numerous potential branches resulting from the adaptations made at those checkpoints.

Figure 2. CRF generation by month for study scenarios 1 and 11.

In order to adequately plan, clinical operations will need to simulate various scenarios to fully characterize the potential outcomes of the study in terms of budget, headcount, and timing. This is similar in principle to the simulations of patient responses conducted by biostatistics and the simulation of potential scenarios for drug needs across study sites performed by the drug supply department, although the tools used are different.

The scenarios evaluated by clinical operations aren't developed in a vacuum; they require close collaboration with the rest of the project team. Based on modeling of patient responses, a minimum and maximum number of potential patients to be enrolled can be identified, as well as potential enrollment periods and treatment periods. It isn't feasible (or necessary) to evaluate every potential scenario for the study, but the scenarios considered should cover the range of possibilities, and particular attention should be paid to their impact on drug supply requirements.

This kind of planning represents a paradigm shift for many in clinical operations. As stated previously, the most commonly used planning tool, the planning spreadsheet, does not provide the flexibility needed to easily model and compare many different adaptive scenarios based on differing planning assumptions. Planning spreadsheets usually provide high-level cost estimates based on benchmark data from previously conducted trials (this technique is also called top-down benchmarking). However, because adaptive trials differ significantly from one trial to another, benchmark data are often not available and, if available, are prone to error due to undetected differences in underlying assumptions between studies. These errors apply to traditional studies as well, however, the impact is even more pronounced with adaptive studies.

In addition, most benchmark data are based on outsourced studies and do not take into account internal resource requirements at all, which means that estimates for internal headcount needs for activities that aren't outsourced can't be generated.

Figure 3. CRA resource needs by month for study scenarios 1 and 11.

Activity-based costing (ABC) is a planning methodology widely used in other industries with highly complex projects, such as construction and manufacturing, and which is gaining currency in drug development due to its benefits in planning adaptive, as well as traditional, trials. Although a full discussion of this technique is outside the scope of this article, a brief overview helps shed light on its applicability to the operational planning of adaptive trials.

ABC essentials

ABC provides estimates for the required budget and headcount based on the work breakdown structure (WBS) for the project, in other words, all of the work to be performed on the study.⁵ The work to be performed, the types of resources who will perform it (and when), and the cost of those resources are determined, and the budget and resource estimates are built from the ground up.

Creating the WBS is the most time-consuming part of the process, and requires a thorough understanding of the work to be performed; however, once it is created, this WBS can be modified for other studies and used again. Software systems also exist that have embedded clinical development WBS and can automate much of the above effort.

The advantages to ABC are significant:

• Planning starts with the study assumptions and is built from the ground up. The undetected inconsistencies in underlying assumptions that plague the top-down benchmarking process do not occur with this technique.
• Accurate costs for outsourced work as well as internal resource needs are generated. Having the task-level detail that ABC provides helps ensure appropriate internal staffing, in addition to helping ensure that CRO partners have allocated adequate resources.
• Creating multiple adaptive planning scenarios and reforecasting based on the findings at the adaptive "checkpoints" once the trial is underway, are straightforward and can be performed quickly and accurately by modifying the underlying assumptions.

A hypothetical adaptive study

Following is a look at a 10-arm, placebo and positive-controlled Phase II dose-finding study of four doses of investigational drug 1, and four doses of investigational drug 2, in patients with Condition X.

Minimum number of subjects: 250

Maximum number of subjects 500

Anticipated enrollment rate: two patients/site/month

Treatment Period: six months

Prior to the study, the project manager met with the statistician and medical director to determine which scenarios should be evaluated. A set of 12 were selected, which were considered to more than adequately cover the range of potential outcomes. Budgets, resource needs, and timelines were developed using the ABC method. Because this study could have been stopped abruptly, it was important to look at scenarios where patients were allowed to complete the treatment period (full treatment period) as well as scenarios where the study was stopped abruptly (see Figure 1).

There are no differences in the duration of study start-up activities between the 12 scenarios. While there are relatively small differences in the duration of close-out activities due to the decreasing number of patients/CRF pages, because of the large differences in the active treatment part of the study the timing of the close-out activities shifts markedly to the left, which will have a significant impact on when resources are needed for these activities.

Budget and resource estimates for the 12 scenarios were prepared. Two scenarios were selected to illustrate how CRA resource needs are determined using the ABC method (see Figure 3). Scenario 1 assumed that all 500 planned patients were enrolled and the study was completed. Scenario 11 assumed that the study was stopped abruptly for safety reasons after enrolling 300 patients, and subjects currently in treatment did not complete the treatment period.

Efficient monitoring is based on data availability at the study sites, which is based on the treatment schedule and patient enrollment rates. Figure 2 shows the CRF generation curve for each of these two scenarios.

This simple example highlights both the importance of simulating a range of outcomes for an adaptive trial to understand the budget and resource implications, as well as the flexibility and granularity of the ABC planning method.

Forecasts and budgets

In addition to the initial planning process, once the trial is underway, reforecasting using actual trial data may be necessary. Reforecasting for studies that were planned at the operational task level—in other words, using an activity-based rather than a "top-down" approach—can be performed quickly by adjusting the underlying cost drivers.

For example, if one or more treatment arms will be dropped and the study sample size decreased (i.e., patients that would have been randomized to the dropped arms are not enrolled at all, rather than allocated to different treatment arms), the effect on the study budget and timelines will be significant. Or, to borrow a common problem in all types of studies, there can be differences between the actual and planned enrollment rates, which can have a significant impact on timelines, budget, and resource needs.

With the level of visibility offered by the ABC method, gaining the full benefit of an adaptive trial is possible. In addition, resources (either headcount or budget) that can be redeployed to other projects can be identified faster should an adaptive trial finish early.

Even ballpark budgets and forecasts for adaptive trials should be as comprehensive as possible. The planning method should guide users to include all of the usual internal and external costs, such as project management, monitoring, and investigator grants, as well as those specific to adaptive trials, which include

• External statistical consulting
• Information Technology
• Data integration and multiple analyses
• Data Monitoring Committee
• Additional drug supplies

Finally, for adaptive studies it is critical that budget planning happens at the portfolio as well as the individual trial level so that potential cost efficiencies and time savings can be fully evaluated within the drug development plan as well as across development plans and products. Potential savings, when recognized, can be reallocated internally or to other development programs.

A high-value proposition

An adaptive design can improve the overall effectiveness of an individual clinical trial by potentially providing better data for efficacy and safety (often by optimizing dose allocation), and by moving decision-making further upstream, allowing early termination of a study and/or development program for futility. Precious patient resources get preserved as well, as treatment is focused on patients who benefit from the intervention delivered at the optimal dose at the optimal time, and patient exposure to suboptimal doses or side effects is minimized. An additional effect of this is that per-patient drug costs can be substantially lower across a study.

The major attraction of adaptive trials is the higher information value for the research investment; not only can overall development costs be substantially lower due to the avoidance of additional studies, overall development timelines may be compressed as well. Good therapies may get to regulatory submission sooner, and development of compounds lacking efficacy and/or associated with unacceptable safety concerns can be stopped earlier. But to realize these benefits, a great deal of preparation must be done in collaboration with a broader group of stakeholders than previously necessary.

Planning for adaptive should be comprehensive, flexible, and incorporate the same type of simulation modeling and scenario planning that steers crafting of the protocol.

References

1. J. Quinlan, "Where Are We Now With Adaptive Designs?" Adaptive Trial Design Conference, Washington, DC, 2008.

2. R.J. Temple, "Adaptive Trial Design," Adaptive Trial Design Conference, Washington, DC, 2008.

3. T. Parke, "Adaptive Clinical Trials in the Real World," presentation to the Massachusetts Biotechnology Council, Cambridge, MA, 2008.

4. J. Quinlan and M. Krams, "Implementing Adaptive Designs: Logistical and Operational Considerations," Drug Information Journal, 40, 437-444 (2006).

5. G. Haugan, Effective Work Breakdown Structures, Project Management Essentials Library Series (Management Concepts, Vienna, VA, 2002).

Molly Blake-Michaels is director, clinical services for ClearTrial, 900 Oakmont Lane, Westmont, IL 60559, email: mmichaels@cleartrial.com