Stopping a Trial Early for Treatment Benefit: Pros & Cons

There are compelling reasons to stop a trial early because of harm or futility. Both subjects and future patients will be protected from harmful or useless medical interventions. At first glance, early trial stopping for benefit seems to provide also undoubted advantages: required sample sizes may be lower, financial and management resources will be saved, and the results will be available earlier.

Compared to trial stopping due to harm or futility, the social implications of early trial stopping for benefit are quite different: rapid drug approvals and exciting publications (mainly in the Lancet and the New England Journal of Medicine, which attract high-level attention by the medical community) may lead to accelerated and widespread use of a new therapeutic option.^1,2

Do prematurely stopped trials for benefit always provide a sound basis for future treatment decisions? Firstly, the overall database of a trial will be reduced by preventing more patients from being enrolled or by shortening the median follow-up period. This may dilute conclusions on the safety of a new intervention, as related risks sometimes show up only in larger patient populations, after long-term use,³ or even first in daily clinical practice.⁴ Secondly, a relevant proportion of enthusiastic, highly cited clinical research trials were later followed by clinical trials with moderate or even contradictory results.⁵ The risk of overestimating efficacy is further increased if conclusions are based on interim results—because the play of chance becomes more important. The following article demonstrates some pitfalls with early trial stopping for benefit.

When it was useful not to stop early

The UK Medical Research Council Acute Myeloid Leukemia Trial (AML12) compared five courses of chemotherapy with four.⁶ Two interim analyses (on 340 and 480 patients, respectively) showed large and highly significant survival benefits in favor of the additional treatment course. The Data Monitoring Committee (DMC), however, considered the findings too large to be clinically plausible and recommended that the trial continue. At the end of the trial, the analysis on 1078 patients provided a "null result." The continuation of the trial therefore prevented future young AML patients from receiving an ineffective and toxic 5th course of chemotherapy. "Be skeptical about unexpected large apparent treatment effects"—so stated the publication title of the final results.⁶

Another randomized trial evaluated the efficacy and safety of tifacogin (a recombinant tissue factor pathway inhibitor) in patients with severe sepsis. The second performed interim analysis in 722 patients demonstrated a 10% absolute survival benefit for tifacogin over placebo with respect to the 28-day mortality rate. Like the previous example, the DMC recommended to continue the trial as originally planned. When a total of 1754 patients was analyzed, treatment with tifacogin had no effect at all on mortality.⁷ Because the trial was not stopped, future patients with severe sepsis were prevented from taking a useless treatment with a higher bleeding risk.

The CHARM program,⁸ consisting of three separate randomized trials comparing the A-II blocker candesartan with placebo in chronic heart failure patients, provides a third example. The prespecified stopping criterion for the whole program was met by the 4th interim analysis, when candesartan reduced all-cause mortality by 24% relative to placebo. Following the DMC decision, the three trials continued. In the final published result, the difference in mortality had shrunk to 9%.⁸ This example again illustrates the importance of continuing a trial to its scheduled completion unless there is proof beyond a reasonable doubt.⁹ Not stopping the CHARM trials early avoided any exaggerated claim of a survival benefit.

Stopping Trials Early: Lessons Learned

In all three examples, interim analysis was at a "random high" of benefit, but by the collection of further data a "regression to the truth" occurred.¹⁰

When it was probably not useful to stop early

In contrast to the previous examples, the following trials were actually stopped early for treatment benefit.

Bisoprolol reduces peri-operative mortality and myocardial infarction (MI) in patients undergoing major vascular surgery. This was the conclusion based on a total of 112 patients on either betablocker or standard therapy after the trial was stopped early.¹¹ Only two patients in the bisoprolol group died compared to 18 patients in the standard therapy group who either died or had a MI—leading to a relative risk reduction of more than 90%. Is this "too good to be true," as these data were inconsistent with results of previous trials? The meta-analysis of V.M. Montori et al. on a set of randomized trials stopped early for benefit¹ could also show that small trials with only few events yielded, in general, greater treatment effects than large trials with a high number of events.

Southwestern Oncology Group (SWOG) Trial 9701 compared a three-month paclitaxel treatment versus a prolonged maintenance 12-month paclitaxel treatment in patients with advanced ovarian cancer in first clinical remission.¹² When an interim analysis revealed a seven-month progression-free survival (PFS) advantage in favor of the 12-month treatment, the trial was stopped early. In the final analysis, however, long-term treatment had no survival benefit but induced grade 2 and 3 neuropathy in a quarter of the patients. Nevertheless, the authors concluded the PFS advantage to justify a 12-month maintenance paclitaxel treatment.¹² The early trial termination, however, makes it difficult to appropriately interpret the data, as the three-month treatment might indeed be equivalent—with less toxicity—since patients still may have prolonged survival through the use of second-line paclitaxel at time of relapse.¹³

In a third clinical trial, adjuvant treatment with intra-arterial radio-labelled lipiodol (a fatty acid ethyl ester) was assessed in patients after curative resection of hepatocellular carcinoma.¹⁴ After 30 patients had been enrolled, an interim analysis showed a 48% reduction in cancer recurrence as compared to untreated control patients. Until the decision was drawn to stop the trial early, 13 more patients were randomized. In order to increase the follow-up time of the recently enrolled patients the data analysis was postponed. In the final analysis (based on 43 patients), the absolute difference in the recurrence rates shrank to 30%. What would have happened if the trial had finished the planned recruitment of 120 patients? A comment to this trial stated¹⁰: "It is a fundamental problem that most cancer clinical trials are far too small. Even if the study had reached its intended sample size, it would have lacked statistical power to detect realistic treatment benefits."

To stop or not to stop—The ethical dilemma

Decisions by a DMC to stop a trial early have potential implications for three groups of patients: patients who have already been enrolled in a trial, prospective patients to be enrolled, and the current and future population of patients who may be treated or not treated with a drug based on the trial results. The obligations to minimize the risk and maximize the benefit in each patient group occasionally conflict with each other.¹⁵ The ethical dilemma is to protect patients in a trial while also protecting society from ineffective and unsafe treatments or, in other words, to save the social value of a trial.^2,15

The Blood Pressure Lowering Arm of the Anglo-Scandinavian Cardiac Outcomes Trial (ASCOT-BPLA¹⁶) is an example of this dilemma. Although the DMC recommended in October 2003 to possibly terminate the trial early (as atenolol treatment was associated with more deaths than amlodipine treatment at an early interim analysis), the investigators decided to continue the trial—the treatment differences could have been related to differences in high blood pressure management and might have evened out over time. However, when a further interim analysis in December 2004 showed a relative 24% higher mortality with atenolol, the trial was immediately stopped. "Would you go to court and say you carried this for another two years knowing that the death rate was higher in one side than in the other?" one of the investigators (who initially was strongly in favor of continuing the trial) later said in an interview.¹⁷

In addition to the clinical considerations previously mentioned, there is also a critical challenge with respect to ethics: finding the appropriate balance between maximizing the scientific validity and social value of a trial and the obligation to protect the trial subjects.¹⁵

Conclusions

All major clinical trials should have an independent DMC.¹⁸ The main risk with early stopping for benefit is overestimating treatment effects. If a clinical trial is going to be stopped early for benefit following a scheduled interim analysis, stringent statistical stopping boundaries for benefit are necessary. The actual final decision to stop a trial early based on the available preliminary data, however, is a clinical and ethical decision. Without doubt: Trial participants need to be protected. But it is also an ethical demand to protect future patients from ineffective and unsafe treatments. Therefore, a trial should remain conclusive about the true treatment effects—which means that stopping a trial early for benefit is not always the best alternative.

Peter Kleist, MD, FFPM, Swiss specialist in pharmaceutical medicine, is medical director with PFC Pharma Focus AG, Chriesbaumstrasse 2, CH-8604 Volketswil, Switzerland, email: peter.kleist@pfc.ch.

Editorial material for this article was presented at the ACT European Summit 2006.

References

1. V.M. Montori, P.J. Devereaux, N.K.J. Adhikari, K.E.A. Burns, C.H. Eggert, M. Briel et al., "Randomized Trials Stopped Early for Benefit. A Systematic Review," Journal of the American Medical Association, 294, 2203–2209 (2005).

2. S.J. Pocock, "When (Not) to Stop a Clinical Trial for Benefit, "Journal of the American Medical Association, 294, 2228–2230 (2005).

3. R.S. Bresalier, R.S. Sandler, H. Quan, J.A. Bolognese, B. Oxenius, K. Horgan et al., "Cardiovascular Events Associated with Rofecoxib in a Colorectal Adenoma Chemoprevention Trial," New England Journal of Medicine, 352, 1092–1102 (2005).

4. D.N. Juurlink, M.M. Mamdani, D.S. Lee, A. Kopp, P.C. Austin, A. Laupacis, D.A. Redelmeier, "Rates of Hyperkalemia after Publication of the Randomized Aldactone Evaluation Study," New England Journal of Medicine, 351, 543–551 (2004).

5. J.P.A. Ioannidis, "Contradicted and Initially Stronger Effects in Highly Cited Clinical Research," Journal of the American Medical Association, 294, 218–228 (2005).

6. K. Wheatley and D. Clayton, "Be Skeptical About Unexpected Large Apparent Treatment Effects: The Case of MRC AML12 Randomization," Controlled Clinical Trials, 24, 66–70 (2003).

7. E. Abraham, K. Reinhart, S. Opal, I. Demeyer, C. Doig, A.L. Rodriguez et al., "Efficacy and Safety of Tifacogin (Recombinant Tissue Factor Pathway Inhibitor) in Severe Sepsis," Journal of the American Medical Association, 290, 238–247 (2003).

8. M.A. Pfeffer, K. Swedberg, C.B. Granger, P. Held, J.J.V. McMurray, E.L. Michelson et al., "Effects of Candesartan on Mortality and Morbidity in Patients with Chronic Heart Failure: The CHARM-Overall Programme," Lancet, 362, 759–766 (2003).

9. S. Pocock, D. Wang, L. Wilhelmsen, C.H. Hennekens, "The Data Monitoring Experience in the Candesartan in Heart Failure Assessment of Reduction in Mortality and Morbidity (CHARM) program," American Heart Journal, 149, 939–943 (2005).

10. S. Pocock and I. White, "Trials Stopped Early: Too Good to Be True?" Lancet, 353, 943–944 (1999).

11. D. Poldermans, E. Boersma, J.J. Bax, I.R. Thomson, L.L.M. van de Ven, J.D. Blankensteijn et al., "The Effect of Bisoprolol on Perioperative Mortality and Myocardial Infarction in High-Risk Patients Undergoing Vascular Surgery," New England Journal of Medicine, 341, 1789–1794 (1999).

12. M. Markman, P.Y. Liu, S. Wilczynski, B. Monk, L.J. Copeland, R.D. Alvarez et al., "Phase III Randomized Trial of 12 versus 3 Months of Maintenance Paclitaxel in Patients with Advanced Ovarian Cancer After Complete Response to Platinum and Paclitaxel-based Chemotherapy: A Southwestern Oncology Group and Gynecologic Oncology Group Trial," Journal of Clinical Oncology, 21, 2460–2465 (2003).

13. S.A. Cannistra, "The Ethics of Early Stopping Rules: Who Is Protecting Whom?" Journal of Clinical Oncology, 21, 1542–1545 (2004).

14. W.Y. Lau, T.W.T. Leung, S.K.W. Ho, M. Chan, D. Machin, J. Lau et al., "Adjuvant Intra-arterial Lipiodol-iodine-131 for Resectable Hepatocellular Carcinoma: A Prospective Randomised Trial," Lancet, 353, 797–801 (1999).

15. A.S. Slutsky and J.V. Lavery, "Data Safety and Monitoring Boards," New England Journal of Medicine, 350, 1143–1147 (2004).

16. B. Dahlöf, P.S. Sever, N.R. Poulter, H. Wedel, D. Gareth Beevers, M. Caulfield et al. for the ASCOT investigators, "Prevention of Cardiovascular Events with an Antihypertensive Regimen of Amlodipine Adding Perindopril as Required Versus Atenolol Adding Bendroflumethazide as Required, in the Anglo-Scandinavian Cardiac Outcomes Trial-Blood Pressure Lowering Arm (ASCOT-BPLA): A Multicentre Randomised Controlled Trial," Lancet, 366, 895–906 (2005).

17. N. Osterweil, "Teaching Brief: The Dilemma of Halting Clinical Trials Early," Medpage Today, 1 November 2005, http://www.medpagetoday.com/HematologyOncology/ClinicalTrials/tb/2047 (accessed 30 October 2006).

18. A. Grant, "Stopping Clinical Trials Early. Data Monitoring Committees May Have Important Role," British Medical Journal, 329, 525–526 (2004).