Blinded by Science with Adaptive Designs

There has been a lot of recent discussion about adaptive designs with the release of a draft guidance from European regulators, and similar guidance is expected from the U.S. FDA.

Adaptive or flexible designs involve design modifications based on the results of interim analyses. Although the use of interim analysis is not new, the option to make design modifications within a formal framework that is accepted by regulators is. Such designs can speed up drug development without lowering regulatory standards, but to execute them well requires good planning and the ability to maintain control of the processes involved.

In this article we describe how the use of technologies such as interactive voice response (IVR) systems allows sponsors to take full advantage of some of the available opportunities for design changes while maintaining necessary standards.

Gaining ground

In Europe, a new CHMP (Committee for Medicinal Products for Human Use) guideline for consultation was issued on the use of adaptive clinical trials in March 2006.¹ It describes the conditions under which adaptive designs can be used and the necessary criteria for running them, and draws attention to the areas where extra care is required. The guideline demonstrates the increasing acceptance of adaptive designs.

In the United States, the FDA is also seeing the increased use of adaptive designs. And in July 2006, it announced plans to promote adaptive clinical trials and issue five new guidances.²

Adaptive designs are reviewed in a recent paper by Philips and Keene³ and in a November 2006 White Paper⁴ from the Pharmaceutical Research and Manufacturers of America Working Group on Adaptive Designs. Both the guidelines and reviews emphasize the important areas of maintaining the blind and, where applicable, keeping the knowledge of the adaption away from investigators.

Later we describe how the investigator and associated staff might be affected by an intervention, why this might be serious, and how you can guard against the problem while still gaining the advantages of reduced time and increased cost–benefit.

Adaptive designs

Historically, clinical trials were planned, executed, and analyzed in three distinct steps. The practicalities of managing the complexities of a trial and the drive to reduce decision times and costs have forced statisticians to examine the process for greater flexibility. Without adversely affecting requirements for the unbiased and independent selection of patients and the random allocation of treatments, adaptive methods allow changes to the original design. Importantly, these changes are beneficial to the decision process and costs.

Early work in this area included two-stage trials and sequential alterations to the sample size, but recently a wider framework for adaptive designs has been developed and is becoming widely accepted. Possible adaptive designs now include:

• Dropping or adding treatment arms (or doses)

• Sample size re-estimation

• Changes to the treatment allocation ratio

• Change in primary endpoints

• Change in objectives (e.g., switching from noninferiority to superiority trials)

• Change in statistical methods

• Adjustment to the measurement time points

• Changes to the interim analysis schedules

For example, a sponsor company might commence a study with several dose options then, with the approval of an independent data monitoring committee (DMC), change the allocation as it becomes clear an arm is unlikely to be suitable. At the end of the trial, more patients are exposed to the drug and dose level of interest than to those that will not be taken forward. Thus, patients and drug supplies are used efficiently.

The framework allows the traditional Phase II dose finding trial and subsequent Phase III confirmatory trial to be combined into a single "seamless" trial (See Figure 1). Generally, adaptive designs offer the ability to make rapid decisions and to react to changes external to the trial. However, approaches like this cannot be employed without caution. Care has to be taken to ensure that plans include the right statistical methods, that the delivery and packaging of supplies can cope with the changes, and that any changes do not influence the trial results. This latter point is expanded in the European guidelines:

Figure 1: The Adaptive Design: A Single, Seamless Trial

"If the staff involved in the recruitment, selection, and execution of the trial become aware of a change, their behavior, whether conscious or not, can influence the results."

As you will read below, technology approaches such as IVR systems used to control treatment allocation and pack dispensing help address the challenge of keeping study site staff unaware of design changes when they are implemented.

Possible compromises

Readers will be well aware of the dangers inherent in breaking the blind (assessments can no longer be accepted as unbiased and randomization schemes may be compromised for future patients). A related fear with adaptive designs is that different patients might be recruited before and after the adaption if investigators become aware of it (e.g., recruiters might avoid patients considered more at risk until they are confident that inefficacious trial regimens have been dropped).

The procedures for running a well-controlled and well-executed trial are already established, but unless care is taken, the added complexity of dropping treatment arms or altering allocation ratios can risk impairing the blind or alerting the site staff to the existence of the changes—which may influence their future patient recruitment and selection decisions.

Consider, for example, a study involving four treatments using a simple randomization list. Without central control of medication pack allocation, supply kits are prelabelled with patient numbers and sites allocate the next available patient number to newly randomized patients. The contents of kits are dictated by a predefined randomization list, so a study including four treatments (A, B, C, and D) might be supplied in such a way that each subsequent group of four patients at a site contains one in each treatment group—the order of which is randomized. In this scenario, if a treatment arm is dropped mid-study, it would immediately become apparent to the investigators, as they would be required to skip certain future patient numbers in their allocations. It may also reveal the block size, increasing the possibility that they guess the treatment groups the current patients belong to. The importance of keeping the block size hidden from investigators is specifically mentioned in regulatory guidance.⁵

Any hint that there is "something going on" has the potential to change the required unbiased selection of patients, which in turn can affect the validity and usability of the study.

The CHMP guidelines are quite specific: After an intervention, analysis plans should include tests to determine whether the outcomes before and after are the same. If arms are stopped, "all attempts should be taken to maintain the blind and restrict knowledge that recruitment to the arm has been prematurely stopped." So, whether the blind is knowingly broken or not, we need to be confident that there is not a significant effect from the alteration on the validity of the trial.

Overcoming challenges

By employing a technology solution with a central distribution method—such as by applying an IVR system to manage trial supplies and randomization—the knowledge of any change to treatments can be kept from investigators. Thus, there is no potential for bias and no reason to expect any observed differences between the before and after results (something the regulators will be specifically looking at). There are two aspects to the protection provided by this approach.

The first is separation of the randomization and dispensing steps. By using pack labels that are not linked to patient numbers in any way, knowledge of alterations in treatment allocation can be restricted. By using a pack list with scrambled (double randomized⁶ ) numbers (see Figure 2), the supplies allocated by the investigator are completely disconnected, so there is no hint of the randomization block size and, importantly, any information about changes in treatment cannot be detected by site staff. By leaving gaps in the pack list numbering system, it is even possible to randomize a new treatment midstudy without the investigator knowing.

Figure 2: Benefits of Double Randomization

There is another feature of IVR solutions that facilitates the execution of adaptive trials: the automated management of the supply chain and maintenance of site stock levels. Even if the investigator cannot deduce anything from the altered numbering in an IVR trial, he or she might deduce something is up if there is a sudden surge in supply deliveries to the site—which could happen if site inventories are adjusted to cope with the altered allocation. But IVR systems can be configured to maintain low stock levels, with regular resupplies as needed. So the investigator is unlikely to deduce anything related to the change. The adjustment of the stock levels to account for the changed situation is automatically handled by the IVR system.

Smooth implementation

The first three adaptive designs listed on the previous page are the most common, and it is possible to preprogram and validate them on an IVR system so that the sponsor can activate the change with a single telephone call, while masking the occurrence from frontline staff. A further aid in implementation is the speed with which IVR or its integration with an EDC system makes key data available for decision making.

Dropping treatment groups. This is easily done by a call to the system, which triggers it to mark the relevant entries in the randomization list as not available for allocation. The need for more supplies of the nondropped treatments is controlled by the system's automated supply chain management.

Re-estimating sample size. The number of patients that the system will automatically close entry to is updated following sample size re-estimation. Supplies can continue to be distributed as needed up until the sample size changes.

Altering the allocation ratio. With IVR the switch can be made instantly via a call. The new allocation is taken from a replacement randomization list, which can be prepared in advance but is only activated in an automated manner at the time of the switch. Similarly, it is very easy to alter the allocation ratio in studies using dynamic randomization allocation techniques to maintain the balance across the study.

Making data available to the DMC. In adaptive trials it is important to rapidly get the key data available to the DMC for decision making. In some cases (as in the case study in the sidebar), key endpoint data may be patient reported and come directly from the IVR system in addition to information about treatment group allocations. In other cases, the key endpoint data may come from an EDC solution with the data integrated with the IVR system, or may be captured on paper CRFs with the key endpoint data sent by mail or fax or collected via an additional IVR/IWR (interactive Web response) event.

A Case Study in Adaptive Clinical Trial Design

Conclusion

With the publishing of the latest CHMP guidelines and a more active response from FDA, it is now acceptable to use the established methodology for adaptive designs to run and manage very efficient studies.

To open the door to all the benefits of adaptive design, the trial needs to be executed carefully to ensure that the right information is available at the right time in the right format for the right people. It is important that the data are collected, collated, and interpreted, and that the results are communicated to the DMC as efficiently as possible.

An IVR system is a proficient way to gain the advantages of adaptive design. Through the IVR system or its integration with EDC, interim results can be rapidly compiled for the DMC. Then, any resulting study design changes can be followed up promptly, with any necessary steps related to treatment allocation and supplies performed accurately and speedily and without the site staff becoming aware of the change or the treatment allocation becoming visible. By using IVR and the double randomization method, not only can the blind be maintained, it can be shown to be maintained.

Thus, by using an IVR system fully, the DMC can make timely recommendations and the research staff can readily take action in the knowledge that the changes are well managed and the results will conform to regulatory requirements.

References

1. Committee for Medicinal Products for Human Use, "Reflection Paper on Methodological Issues in Confirmatory Clinical Trials with Flexible Design and Analysis Plan," March 2006, http:// www.emea.eu.int/pdfs/human/ewp/245902en.pdf.

2. "FDA Pushes Adaptive Trials with Special Teams, Five New Guidances," Clinical Trials Advisor, 11 (14)1–2 (2006).

3. A.J. Phillips and O. Keene, "Adaptive Designs for Pivotal Trials: Discussion Points from the PSI Adaptive Design Expert Group," Pharmaceutical Statistics, 5, 61–66 (2006).

4. PhRMA Working Group on Adaptive Designs, "White Paper," Drug Information Journal, 40 (4) 421–484 (2006).

5. International Conference on Harmonisation, E-9 Document, "Guidance on Statistical Principles for Clinical Trials," Federal Register 63 (179) 49583–49598 (1998), http://www.fda.gov/cder/guidance/91698.pdf.

6. M. Lang, R. Wood, and D.J. McEntegart, "Protecting the Blind," Good Clinical Practice Journal, 12 (11), 10–13 (2005).

7. M.K. Smith, I. Jones, M.F. Morris, A.P. Grieve, and K. Tan, "Implementation of a Bayesian Adaptive Design in a Proof of Concept Study," Pharmaceutical Statistics, 5, 39–50 (2006).

Damian McEntegart,* MSc, FIS, is head of statistics & product support at ClinPhone PLC, Lady Bay House, Meadow Grove, Nottingham NG2 3HF, United Kingdom, +44 845 3659900, email:dmcenteg@clinphone.com Graham Nicholls, MSc, is product manager, randomization & supply chain management, and Bill Byrom, PhD, is product strategy director at ClinPhone PLC.

*To whom all correspondence should be addressed.