Pandemic Highlights Urgency for New Methodologies for Evidence Generation
Applied Clinical Trials
The need for statisticians, mathematicians, and computer and data scientists to collaborate on modern methods for “substantial evidence” in drug development is critical.
Even at the risk of being accused of heresy, we think that moving beyond traditional statistics is overdue. And here is why: Sir Roland Fisher, who has been described as “a genius who almost single-handedly created the foundations for modern statistical science” was born in 1890. Randomized controlled trials, which many consider to be the gold standard of clinical research, were developed in the 1940s.
This was all way before anyone had the slightest idea about big data, machine learning, neural networks, deep learning, artificial intelligence, etc. Those old methods were created for relatively small and simple datasets and before we really understood the complexity of biological systems, where interrelated and interdependent parameters always play together to generate a certain physiological output.
But these methods still form the basis for modern medical research. We believe that a drug “works” if the difference of the means of the effect size of a variable in a large treatment group compared to a large control group is statistically significant. But, unfortunately, the “mean patient” rarely exists. Therefore, individual patients in the real world react often very differently to a specific drug than what has been predicted by the “mean” of a clinical trial. We must acknowledge that these century-old scientific methods have significant limitations, which, in our view, hamper the progress of modern medicine. A mean derived from n=1000 patients has little meaning for personalized medicine where n=1.
The need for new, better ways for substantial evidence generation has become painfully obvious in the current COVID-19 pandemic. While there are many investigational drugs against the coronavirus, thousands of patients are still dying because these drugs are not approved for broader use due to the lack of traditional clinical evidence. This evidence is currently derived from randomized controlled trials that take months to years to complete.
We think nothing illustrates the failure of the old methodologies more than the fact that a large number of people lose their lives because the evidence generation takes simply too long to save them.
We urgently need statisticians, mathematicians, and computer and data scientists to come together and develop, with the tools of modern digital sciences, new 21st century methodologies for “substantial evidence” generation in a global health crisis. The signs of how powerful and game-changing these new methodologies can be are already here:
• AI is beginning to surpass human radiologists’ in its ability to diagnose disease.1
• AI and advanced machine learning methods are starting to show promise in their ability to accelerate the discovery of novel therapeutics. Many biopharmaceutical companies and AI startups are betting that with enough data, these methods will work so well that they will help to accelerate the discovery of new therapies for the novel corona virus, 2019-nCoV.2
• Causal AI methods might be uniquely positioned to discover underlying causes of disease and clinical response to treatment on an individual level, making personalized medicine real. This approach leverages the richness of multimodal patient data from genomic, molecular, imaging across cells and tissues, deep and digital phenotyping, labs, and clinical measure across many individuals to train models that predict causal drivers of disease and response to treatment.
Several results have been published demonstrating ability to find causal molecular drivers that emerge as a result of using AI to learning complex networks, underlying the disease with the goal of using these insights to better target treatments to patients in clinical trials and eventually at the point of care.3, 4
We envision a future where these new tools and methods, developed under solid mathematical grounding, will enable us to go beyond the restrictions of traditional statistics, which are limited by sample sizes, the inadequacy of p-values as the metric for statistical significance,5 and the limitations of multiple hypothesis testing.
Looking at the big picture, they will reduce the time and cost of drug development substantially, and, more importantly, they will help to save the lives of patients desperately waiting for new, effective treatments.
Ülo Palm, MD, PhD, Senior Vice President, Digital Sciences, Allergan; Iya Khalil, PhD, Co-Founder and Chief Commercial Officer, GNS Healthcare
