Update on Clinical Performance Metrics

Clinical performance metrics has been a topic of interest in the industry, as databases from legacy systems have aggregated and become more structured over the past several years. With newly integrated data structures, the biopharmaceutical industry is now able to implement novel approaches to address clinical trial issues. In this article, we will review insights and updates from ExL’s 15th Clinical Performance Metrics & Analytics Summit.

Predictive Modeling in Protocol Risk and Site Risk

Alex Hsieh, Director of Predictive Analytics at Pfizer, discussed how Pfizer implemented machine learning and AI to predict future quality risk and events at the final approved stage. Hsieh used the analogy of weather, that everyone can relate to, to link clinical trial risk prediction to weather forecasting; essentially, predicting risk classes of an upcoming clinical trial using the empirical data, upgrading or downgrading the study risk based on the new data, and updating the risk predictive model by incorporating the latest studies. Hsieh indicated that, by using 15-20 model inputs from more than 700 studies, can generate an output that predicts the risk class of a study with an 80% accuracy rating. The other predictive model is capable of predicting which sites will likely have critical and/or major audit findings and that more information about the condition of those sites should be evaluated to mitigate the potential risk.

Digitizing SOPs and SOP Deviations

Nancy Dynes, Quality Metrics Consultant at Eli Lilly, explained how Lilly is focusing on improving quality in clinical development by carefully aligning SOPs and SOP deviations across systems. Dynes indicated that before this integration, their clinical development system contained a variety of SOPs that were organized loosely by

function, as documents were housed in a differently structured but validated electronic system. Lilly’s process for tightly integrating their deviations with the SOPs involved efficiently aggregating and understanding SOP deviation patterns, as previous reporting was challenging to produce and was labor intensive. Lilly implemented a new document management system to enhance structured linkages to individual SOPs; specifically, the deviations are linked specific processes in the SOP documents and the SOP infrastructure is integrated to the deviation system via output metadata. Accordingly, with this new system, quality personnel can view descriptive fields for each SOP in the deviation system, and have an ability to better assess, report, and visualize SOP conformance analytics. This new system enables quality personnel to measure the number of deviations per topic and evaluate change over time.

Clinical Operations Metrics Implementation Strategies

Judy Hwee, Clinical Metrics & Reporting at Amgen, elaborated on the importance of executing robust strategies through enterprise transformation methodologies. Hwee indicated that the effects of data latency have exponential implications on time from business events to actions taken, as data, analysis, and decision latencies need to occur before action is taken. Accordingly, minimizing this latency through better information flows provides significant business value. Hwee explained how biopharmaceutical enterprises could drive sustainability for advancing analytics by implementing the three pillars of transformation. The three pillars of transformation include visualizations (i.e., deploying new reporting and visualization platforms, implementing user training, and having a

strong partnership with the information systems group for analytic application development), metric engines (i.e., establishing a single source of truth, having recursive metrics, and parameter flexibility), and creating a business glossary (i.e., establishing data governance models with a business focus, launching and deploying new data governance tools, and chartering centralized business rules and definitions). Hwee elaborated on a few vital roadmap principles for implementation which includes accessibility (i.e., offering systems access to all users), core concepts (i.e., maintaining core domain-focused reports), simplistic user experience views (i.e., visualization to support risk detection and performance management), and data dimensions (i.e., visualizations to support driving actions and decisions). Through this method, users can practice flexibility in their data analyses (i.e., drilling through and visualizing reports) and decision making, hence, improving enterprise scalability and efficiency.

Summary

In summary, database integration initiatives in the biopharmaceutical industry are now enabling clinical development departments to leverage that data for enhancing decision-making. It is likely that we will continue to see customized uses that leverage data integration for clinical trial improvement. ExL’s European Clinical Quality Oversight Forum is occurring Feb 25-26 in London.

Moe Alsumidaie, MBA, MSF is Chief Data Scientist at Annex Clinical, and Editorial Advisory Board member for and regular contributor to Applied Clinical Trials.