Unlocking the Potential of Risk-Based Quality Monitoring

Patrick Hughes

Risk-Based Quality Management (RBQM) and Risk-Based Monitoring (RBM) are shifting the clinical research landscape, dramatically accelerating the drug development pathway, driving data quality, and ensuring early risk detection.

Endorsed by both the FDA and the EMA, RBQM encourages the more effective use of remote monitoring practices, reducing site visits, and eradicating 100% source data verification (SDV).

The method has proved its worth during the pandemic, not least in the race to find a vaccine.

The full benefits of RBQM will be further unlocked as the practice is encoded in the current ICH E6 (R2) GCP mandates and will be further reinforced in the R3 update, released this year.

As such, we are now seeing an industry-wide paradigm shift towards this new way of working.

According to our recent webinar polls, out of 650 industry representatives, 56% are either already implementing RBQM or are about to start. A further 25% said it is already an integral part of running trials. So overall, 81% are either implementing or have already fully adopted RBQM.

The benefits of RBQM and RBM

Robust RBQM solutions allow teams to access information centrally, collaborate on investigations, and document any mitigations or remedial actions. Not only does this result in a clean dataset, but it also creates a reliable audit trial for regulatory scrutiny.

RBQM also facilitates greater adoption of sophisticated central monitoring (CM), which allows sponsors to interrogate clinical and operational data in real-time, increasing efficiency, improving patient safety, and reducing costs.¹

Throughout the pandemic, CM has presented a proven alternative to on-site monitoring.

A real-time risk-based approach to vaccine development

At the start of 2020, the scientific community was faced with an unimaginable task: developing, testing, and deploying effective COVID-19 vaccines at lightning speed without compromising safety.

Despite many challenges, Pfizer and BioNTech’s mRNA product received regulatory approval just 266 days after the World Health Organization’s pandemic declaration. As of January 15, 2022, 307 million doses of Pfizer’s BNT162b2 vaccine have been administered in the USA alone.²

The pivotal trial included more than 40,000 people across 150 global sites, and participants were from a broad range of demographics. To expedite development, the study encompassed Phases I, II, and III, and infection-control measures presented logistical issues in terms of site visits.

To support the oversight process and protect the integrity of the vast volumes of data being generated by the trial, which recruited at a rate of 5,000 people a week, Pfizer worked with CluePoints to deploy an agile, real-time RBM approach.

Led by a core clinical study team and clinical data scientists, the CM analytics focused on an evolving set of study-specific key risk indicators (KRIs), data quality assessments (DQAs), and quality tolerance limits (QTLs).

Data was analyzed daily, rather than the usual bi-weekly, monthly, or quarterly review, allowing data quality issues to be identified, and rectified in real-time. In addition, teams were expanded to cope with the additional workload, and global sites enabled 24/7 centralized monitoring.

This process enables multi-functional teams to identify the critical data, and relevant KRIs and QTLs, for use in the design of the CluePoints RBM database.

In the case of this particular COVID-19 study, two CluePoints databases were built to make the CM as efficient as possible and run all the different analyses in real-time.

Additional resources were also employed at the Central Monitoring and site level to ensure signals were both generated and investigated/actioned. As a result, the action was taken on between 50% and 55% of all signals, compared to the non-COVID portfolio average of 15%. This allowed sites to operationalize root cause analysis findings quickly before issues could impact data quality or integrity.

All risk signals were also presented to the core study team during weekly data surveillance meetings at the study level. This enabled a more holistic view and informed decisions on site visits, SDR, or SDV. In addition, the process was cyclical, with risks identified at the data surveillance meeting being fed back into the RACT, informing the evolution of KRIs.

The team developed a range of new reports and visualizations to report the CM findings and drive engagement with sites and monitors. These included a signal and action tracker, which has become standard across Pfizer’s CM programs.

Key learnings from Pfizer vaccine development

The example above represents a leap forward in how clinical trials are conducted. By employing an agile, RBQM approach, Pfizer increased efficiencies to such a level that it submitted drug applications in record time, with very high levels of quality. So, what can we learn from the use of RBM during Pfizer’s vaccine development?

First, we need to be flexible and agile with KRIs, updating throughout the life of the study to correspond with risk signal findings. For example, reviewing two weeks of data to see what was happening allowed the team to ensure KRIs were not diluting the view of risk signals and always focused on the most critical subset of data.

Secondly, study-specific QTLs enable a view of trends at the study level and within regions and sites.

Last, CM analytics can alert the team to sites with high risk–particularly important when on-site visits cannot occur.

Using unsupervised statistical monitoring to uncover fraud

Innovations in data analytics are not just helpful in detecting risks–they can also be used to see fraudulent results.

Working with partners at Boehringer Ingelheim, the International Drug Development Institute (IDDI), and the Interuniversity Institute for Biostatistics and Statistical Bioinformatics, CluePoints reanalyzed the database of a large randomized clinical trial, known to have been affected by fraud.

The results, published in September 2021³, showed unsupervised statistical monitoring could have uncovered fraud at a clinical trial site around a year earlier than traditional methods were able to do.

The Second European Stroke Prevention Study (ESPS2) was conducted in the early 1990s. Serious inconsistencies in the case report forms (CRFs) of one site, #2013, led the trial’s steering committee to question the reliability of the data. A for-cause analysis of quality control samples, and extensive additional analyses, including blood concentrations of the investigational drugs, proved the patients never received these drugs as mandated by the protocol.

The site’s data, comprising 438 patients, were excluded from the trial analyses, and after lengthy legal proceedings, the investigator was convicted.

The ESPS2 study database provided to CluePoints for reanalysis comprised all clinical data from the CRFs and laboratory results across all patients and sites.

The project was divided into two phases. The objective of the first phase was to confirm if DQA would identify the known fraudulent site (#2013), while the second phase aimed to find out how much earlier this site would have been detected. While the fraudulent site was known to the sponsor team, CluePoints researchers remained blinded until the analysis was complete.

Phase two revealed that site #2013 would have been detected as atypical when only 25% of its final subject data volume had accrued in May 1991.

By contrast, the ESPS2 study team first developed suspicions and conducted a detailed statistical assessment of the site’s data in June 1992. At this point, approximately 75% of site data had accrued. As a result, the fraud was not confirmed, and site data was excluded until June 1993.

Had CluePoints’ DQA been used in this study, the findings would have enabled the sponsor to confront the investigator at least a year before the traditional approach allowed.

This shows that an unsupervised approach to CM, using CluePoints statistical models, can effectively detect sites with fraud or other data anomalies in clinical trials. Increasing the use of such methods could enhance safety concerns and improve the reliability of study outcomes due to more accurate study data.

The R2 roadblock to adoption

With such clear benefits, it may seem surprising researchers have been slow to adopt innovations fully. This delay is in no small part due to a lack of clarity around Good Clinical Practice (GCP).

In 2016, ICH E6 (R2) updated its guidelines to support innovation and encourage the wide-scale use of RBM. However, it failed to spell out how sponsors and CROs should embrace this new way of working.

The resulting lack of clarity meant researchers were unsure on how to proceed, and many have ignored the possible advantages of RBM. Others have overcomplicated processes by collecting and monitoring all forms of data rather than just those that matter and, in doing so, defeated the very object of RBM.

ICH E6 (R2) has also acted as a roadblock to adaptive trial designs by asking teams to define all key parameters at the start and stifled advances in technology take-up. For example, it is now widely accepted that patient-centered practices reducing participant burden play a role in increasing recruitment and retention rates.⁴ Yet the guidelines are unclear on whether innovations such as eConsent are acceptable.

ICH E6 (R3)–An opportunity to unlock the full potential of RBQM

The true potential of RBQM could be unleashed with the publication of ICH E6 (R3).

ICH has said the updated guidelines aim to emphasize achieving quality by design (QbD) and ensure innovations in technology and design are facilitated and encouraged.

Of note to researchers hoping for more clarity around the use of RBQM are the focuses on QbD and risk-based approaches, both of which clear the way for more wide-spread adoption of modern research practices.

In addition, the guidelines appear to be sufficiently flexible to allow the industry to embrace advances in innovations as they arise.

In terms of RBQM, R3 offers a recipe for effective deployment, touching on how trial processes should be proportionate to the risks inherent in the trial and the importance of the information being collected. It also advises quality factors to be prioritized during trial design to identify those critical to the study.

Accelerating development while protecting data integrity

Medical science is evolving at an incredible rate. The tech sector supports this evolution by designing and launching innovations that allow researchers to increase their likelihood of success and streamline timelines while protecting safety.

By detecting issues earlier than traditional methods, allowing sponsors to deploy quick and decisive remedial action, and helping to avoid expensive and unnecessary delays to approval and market, RBQM can shorten the drug development timeline.

Moving from theory to practice during the pandemic has proven the approach can accelerate development while protecting data integrity.

The publication of R3 looks set to unshackle the potential of RBM further and set the scene for the drug development landscape for years to come.

Patrick Hughes, Co-Founder, and Chief Commercial Officer, CluePoints

References

1. Looking to the Future with RBQM Controlling Risk Across the Entire Study Lifecycle. (2020). https://globalforum.diaglobal.org/issue/september-2020/looking-to-the-future-with-rbqm/
2. https://www.statista.com/statistics/1198516/covid-19-vaccinations-administered-us-by-company/
3. https://doi.org/10.1007/s43441-021-00341-5
4. https://acrpnet.org/2018/01/16/recruitment-retention-patient-centric-approach-proven-accelerate-recruitment/