Harnessing AI in European Clinical Trials: Regulation in Progress

It might be considered banal to observe that the accelerating pace of technological evolution is posing ever more intractable problems for developers of healthcareproducts—and for those that regulate them. But the most cursory glance over the last decade offers abundant evidence that alongside the growth in opportunity has come a heavy—and anything but banal—responsibility for developing new skills, new disciplines, and new duties. And it isn't going to get any easier. So rapidly has the clinical trials community had to come to terms with the concepts of 'omics, biomarkers, precision medicine, or nanotechnology that it is easy to forget how novel these concepts were until just a few years ago. As recently as 2020, the European Medicines Agency (EMA) was still referring to big data, novel manufacturing, novel clinical trials design, and the revolution in synthetic biology as "emerging science and technological innovations."

Now, as major pharmaceutical legislative reforms and new rules on sharing of healthdata further complicate the current preoccupations with mastering real-world evidence or complex clinical trials, out of the blue comes the brave (or frightening) new world of AI. And there's no getting away from it—as EMA is one of the first to insist. It has now launched a consultation to obtain stakeholder feedback. So summer break or not, it's time to refine thinking on yet another—and potentially more far-reaching—challenge for drug developers in what EMA calls "this fast-evolving field."

According to the reflection paper the agency has drafted on "the use of artificial intelligence in the medicinal product lifecycle,"¹its principal concerns are to avoid risks to patients or to the integrity of clinical study results, and at the same time to promote AI trustworthiness, particularly by scrupulously avoiding the incorporation of bias into applications. It wants to define scientific principles relevant for regulators when AI or machine learning (ML) are applied to support development and use of medicines.

Some of the principles have, apparently, already been established by EMA. The paper says it is the responsibility of the marketing authorization applicant to ensure that all algorithms, models, datasets, and data processing pipelines used "are fit for purpose and are in line with ethical, technical, scientific, and regulatory standards as described in GxP standards and current EMA scientific guidelines." And in a lengthy section devoted to clinical trials, it stresses that GCP should apply both to the use of AI/ML in a model generated for clinical trial purposes, in a clinical trial setting, including in trials involving devices or in vitro diagnostics. When AI/ML models are used for transformation or analysis of data, they are considered a part of the statistical analysis and should follow applicable guidelines and include analysis of the impact on downstream statistical inference. In late-stage clinical development, a detailed description of a prespecified data curation pipeline and a fully frozen set of models used for inference will be necessary.

In late-stage pivotal trials, all risks related to overfitting and data leakage must be carefully mitigated, says the paper, and prior to model deployment, performance should be tested with prospectively generated data acquired in a setting or population representative of the intended context of use. Incremental learning approaches are not accepted, and any modification of the model during the trial requires a regulatory interaction to amend the statistical analysis plan. Prior to the opening of any dataset used for hypothesis testing, the data pre-processing pipeline and all models should be locked and documented in a traceable manner in the statistical analysis plan. And once opened, any non-prespecified modifications to data processing or models implies that analysis results are considered post hoc and, hence, not suited for confirmatory evidence generation.

The draft guidance for early-phase trials covers treatment assignment or dosing, requiringstatistically robust planning of subsequent trials. And in a section focused on precision medicine, it advises that AI/ML can be used to individualize treatment in relation to factors such as disease characteristics, patient genotype, wide-band biomarker panels, and clinical parameters, and could include patient selection, dosing, de novo design of product variants, and selection from a pre-manufactured library of variants. It is possible that an AI/ML application is referenced in the summary of product characteristics to aid decisions on indication and posology, and this is, the agency insists, "a matter for medicines regulation," because this is tantamount to a high-risk use. "Special care" is needed in defining what constitutes a change in posology, and this requires a regulatory evaluation before implementation.

After all these strictures (and many more besides in the draft paper), it will come as a relief and a reassurance that the conclusion clearly states that the "field of AI and ML shows great promise for enhancing all phases of the medicinal product lifecycle." But anyone wanting to nuance the approach that EMA is taking has until the Dec. 31 deadline to get their views in.

Reference

1. https://www.ema.europa.eu/en/documents/scientific-guideline/draft-reflection-paper-use-artificial-intelligence-ai-medicinal-product-lifecycle_en.pdf