Review Highlights Gaps in Publicly Funded Non-Inferiority Clinical Trials, Call for Better Design and Reporting Practices

Credit: Thurstan Hinrichsen/peopleimages.com | stock.adobe.com

A review of 114 publicly funded non-inferiority (NI) trials in the UK show inconsistent adherence to best practices, with high levels of non-inferiority margin (NIM) reporting, but insufficient statistical justification and limited use of both intention-to-treat (ITT) and per protocol analysis populations. The authors of the study, published by Trials, said their findings indicate the need to improve the design and reporting from NI trials to improve clinical relevance.¹

“Publicly funded NI trials are increasing in popularity and therefore ensuring the guidelines (created with a focus on industry NI trials) are being suitably followed is important to ensure best practice,” the study authors wrote. “The NI trials in this review had high levels of reporting of the NIM with over half justifying this chosen value; however, the justification was more commonly based on clinical importance and statistical considerations (i.e. confirming superiority over placebo) were often absent.”¹

Investigators note that NI trials, which seek to determine whether a novel therapy is no worse than an existing treatment, have become more popular in the UK and within the National Health Service, specifically. The design allows for a comparison of treatments without the use of a placebo to focus on additional benefits such as safety, cost-effectiveness, and convenience. However, the design of NI trials creates complexity, specifically around the NIM, which determines how much less effective the novel therapy can be while still being deemed acceptable for use in treatment.

The NIM defines the maximum acceptable difference between treatments that would be considered non-inferior. As such, the value chosen for the NIM is crucial to defining the success of the trial, according to the investigators.

“There are guidelines available which advise on selecting the most appropriate NIM. A recent survey, however, found that these guidelines were followed by only 44% of respondents suggesting uptake of this advice in practice is not high,” the investigators wrote. “Additionally, previous reviews of NI trials suggest that although the NIM used was well reported, it was not well justified with under half of the NI trials providing this justification. By not justifying a margin it reduces the ability for readers to assess its appropriateness.”¹

The authors noted that despite current guidelines recommending that the basis of NIM be on statistical evidence and clinical relevance, prior research demonstrates poor adherence to these guidelines, with many trials falling short of justifying their chosen margin. Other challenges they identified include the selection of analysis methods, significance levels, and sample sizes, as most of the current recommendations are based on industry-funded trials, which limits the understanding for how publicly funded trials can address these issues.

The authors of a prior analysis published by The New England Journal of Medicine stated that appropriately designed and executed NIs can help identify innovative treatment alternatives with clinical value, but improvements are needed.²

“Standards for the design and reporting of superiority trials have been widely disseminated, but adherence to these standards is not universal,” they wrote. “Furthermore, unique challenges continue to emerge for noninferiority trials as their uses become both more common and more diverse.”²

The investigators of that analysis recommend that NI trials provide an explicit justification of the NIM based on a measured or anticipated benefit for the novel therapy; that caution is used in considering composite end points that may include discordant benefits and risks; and they urge the use of a sensitivity analysis of missing data in the planning and analysis of NI trials.²

To conduct their analysis, the authors of the current study used the International Standard Randomised Controlled Trial Number web registry and they searched the National Institute for Health and Care Research’s Funding and Awards Library and Journals Library using the term ‘non-inferiority’ and logical synonyms. To qualify for inclusion in their analysis, a trial needed to be UK publicly funded, randomized, and controlled.

Through January 2022, investigators found 477 potential trials, which was paired down to 114 NI trials for inclusion. For most of the trials included, NIM was defined with a median of 8% (IQR: 3%–10%) for risk differences (n = 58) and 0.35 (IQR: 0.26–0.43) standardized mean difference for continuous outcomes (n = 30).

The justifications for the margin chosen by investigators (n = 62) were more commonly based on clinical relevance (49/62) and less so on statistical considerations (13/62). The current study authors found that the most common primary analysis population was solely on an ITT basis (49/114). They noted that treatment superiority was well described but was not always included as an outcome—only being powered for in approximately one-third of cases.

“By not powering for the superiority outcome, it implies that the success of the trial (and therefore the evaluation of the new treatment) is not dependent on a secondary outcome being declared superior as well as the primary NI outcome being declared non-inferior,” the study authors concluded. “Although this will be true for some treatments which are deemed to be more convenient for the patient, benefits such as improved safety (the most commonly used justification for completing a non-inferiority trial) should be confirmed within the trial to demonstrate the new treatment is a worthwhile alternative for use in practice.”¹

References

1. Totton, N., Julious, S., Walters, S. et al. A review of UK publicly funded non-inferiority trials: is the design more inferior than it should be?. Trials 25, 809 (2024). https://doi.org/10.1186/s13063-024-08651-3

2. Mauri L., and D’Agostino R. Challenges in the Design and Interpretation of Noninferiority Trials. N Engl J Med 2017;377:1357-1367. DOI: 10.1056/NEJMra1510063. Vol. 377 No. 14.