The Practical Implementation of Risk-Based Monitoring

Data quality matters most in determining study success and ensuring patient safety. However, patient numbers continue to rise, CRFs are becoming more complex with each new study and, with the use of sites around the globe, these elements significantly impact the ability to guarantee the quality and integrity of clinical trial data while containing costs. Until recently, the standard paradigm for clinical site monitoring was to do routine site visits every four to 10 weeks with 100% source data verification (SDV) in most cases. This process typically accounts for 30% of a sponsor's overall costs but results in less than 3% of any data changes post-monitoring visit. This antiquated and inefficient paradigm costs the industry hundreds of millions if not billions of dollars. As budgets become more difficult to manage and data issues grow by the day, regulatory authorities and most in our industry no longer believe that traditional "on-site monitoring" and "100% SDV" are the best solutions. Humans are less efficient at detecting erroneous data than an automated process and are the most expensive resource employed in any trial. Computer algorithms, by contrast, are relentless in their interrogation of the data and are much more cost-effective.

The FDA guidance for industry and the EMA reflection paper encourage the use of risk-based monitoring tools as well as increased efficiency in the form of reduced SDV. The documents bring to our attention the fact that sponsors should be focusing or "targeting" their on-site monitoring activities. One of the most common ways to do so is by remote monitoring. This means taking a look at the data off site and determining where the issues may be prevalent. They also refer to data management metrics and trending which includes things such as subjective key risk indicators (KRIs). Then, of course, there is central statistical monitoring. Sponsors need to look at their data in numerous ways in order to target on-site monitoring visits effectively. By doing so, the goal is not only to reduce 100% SDV levels but to de-risk their studies by improving data quality at significantly lower costs.

In contrast with KRIs, which only indicate the potential for risk, independent central statistical monitoring determines the expected values of each variable by assessing and examining the data from all sites in order to identify statistical outliers. This concept has been perfected by CluePoints over the last 10 years using the patent-pending process within the SMART™ engine. Complex and proven statistical algorithms drill down into individual patient data to detect issues that could put the study at risk and create a road block to successful submission. This approach requires minimal work for the study team in gaining objective information in order to optimize on-site monitoring by targeting centers at risk. In addition, sponsors who strategically outsource to CROs are finding increased efficiencies by utilizing central statistical monitoring as an oversight tool to regularly check the quality of their data.

Key risk indicators are applied because they focus on important known risk factors, for example, the proportion of AEs or SAEs collected. There is no question that the safety reporting in a clinical trial must be looked at very carefully. However, unlike other providers, CluePoints provides an independent, EDC-agnostic approach to central statistical monitoring that is based on the actual clinical data and not subjective indicators. The idea behind this is that all variables are indicative of quality—whether it is lab data, clinical data, baseline data, or treatment outcomes; everything goes into this analytical system and all data are deemed equally important. In a clinical trial, everything that is collected should be worth collecting, and therefore worth checking—central statistical monitoring ensures the ability to determine the quality and integrity of the data and ensure that monitoring efforts focus on errant sites quickly and efficiently.

CluePoints has been utilized in several dozen studies with sponsors such as GSK Vaccines and Gilead Sciences. Brian Nugent, Associate Director, Clinical Operations at Gilead has explained his experience with CluePoints and how he and his study teams have practically implemented a central statistical monitoring solution in three quick and easy steps—setup, analysis and report. During the setup stage, the sponsor's data manager and CluePoints discuss the structure of the datasets and their contents and the data is uploaded. During analysis, CluePoints select statistical tests and variables, browse and filter results, and generate outputs. The sponsor is provided with a detailed report that lists any outliers. Data discrepancies are clearly defined and the graphs and summary results are well displayed and informative.

In Gilead's first study with CluePoints, there were 120 centers and 660 tests conducted, generating 79,200 p-values. The tables of p-values were very detailed and CluePoints provided expert interpretive guidance to make this information easily digestible—both known and unknown risks and issues were detected in this study. "Rather than to fight the fire once the fire started, we really wanted to uncover the data; which we did, and we addressed a number of things," Nugent said. "We now plan to integrate KRIs and central statistical monitoring into our everyday study quality and risk management plans and of course we expect to have improved data quality and certainly at a lower cost."

Many companies already have KRIs in place and central statistical monitoring can further improve risk-based monitoring solutions by more efficiently detecting errors, sloppiness, tampering, and even fraud. When utilizing central statistical monitoring during a study, sponsors can take corrective action, such as that taken by Gilead, in a timely manner to prevent issues becoming deep-rooted and re-analyze data regularly as part of an iterative process. The sponsor or CRO can perform focused site monitoring visits, identify CRA and site retraining needs, or develop site tools to provide clarification on protocol and study processes. CluePoints can also be used as a final data quality check at the end of a study.

During a recent webinar, CluePoints surveyed over 300 industry colleagues from clinical operations, data management, and statistics regarding their experience with risk-based monitoring and the benefits of a centralized statistical monitoring approach (Figures 1 and 2).

Francois Torche, is the CEO of CluePoints, 185 Alewife Brook Parkway, Suite 410, Cambridge, MA, e-mail: [email protected].