Using Data Patterns to Proactively Improve Study Quality
Recent analysis showed 81% of clinical trial sites using RBQM statistical data monitoring experienced an improvement in quality.
Statistical data monitoring (SDM) plays a critical role in ensuring the quality and integrity of clinical trial data. By applying a range of statistical tests to patient data across study sites, SDM is designed to detect highly atypical data patterns that may indicate underlying systemic issues in trial conduct. These issues can range from data fraud and inaccurate recording to training deficiencies and equipment malfunctions or miscalibration. Through proactive identification of such anomalies, SDM helps maintain the reliability of study outcomes while safeguarding patient safety and compliance with regulatory standards.
Quantitative analyses of central monitoring's impact on clinical trial quality remain rare, with most studies focusing on specific datasets, simulated data, or retrospective analyses. In this analysis we explored the impact of SDM on improving quality metrics in clinical trials, comparing results to studies conducted without the use of central monitoring to assess its effectiveness.
The analysis, recently published in a peer-reviewed journal,1 is updated here to include the most recent data collected from the CluePoints Central Monitoring platform, now covering the time period between September 2015 and November 2024.It focuses on 2,044 atypical sites across 300 completed studies from a range of therapeutic areas and study phases. A site’s atypicality is assessed using the platform’s "Data Inconsistency Score" (DIS), calculated for each site from hundreds of p-values, where DIS scores above 1.3 indicate atypical behavior. To provide a baseline for comparison, we conducted the same analysis on 43 atypical sites from two studies that did not make use of central monitoring.
For each atypical site, we collected two DIS of interest:
- DISO—Site’s DIS score when it first exceeded 1.3 and at least one SDM-generated risk signal was investigated by the study team (“opening DIS”)
- DISC—Site’s DIS score when all risk signals at the site were closed (“closing DIS”) (Figure 1).
Figure 1. Illustration of site selection for the quality improvement analysis. Blue and Magenta DIS line: The line is magenta during the period of time that the study team is investigating and remediating issues at the site. The line is blue when no risk signals are being processed by the study team. Dotted Line: represents the 1.3 DIS threshold (representing a P-value of 0.05). Lines starting with a diamond and finishing with a circle: The diamond represents a signal being opened and the circle a signal being closed.
Source: CluePoints
The first objective was to determine the proportion of sites with improving quality, considering quality to have improved if DISC < DISO. Using this approach, we found that 81% of the sites demonstrated improvement in quality when SDM was applied, compared to only 56% of sites not using any central monitoring. This difference was statistically significant, highlighting the impact of SDM on improving site quality.
The second objective was to measure the size of quality improvement by examining the relative change between the DISO and the DISC. On average, sites using SDM showed a 43% improvement in their quality scores. This improvement was consistent across different therapeutic areas and study phases. In contrast, sites not using central monitoring demonstrated a much smaller average improvement of only 17%, further emphasizing the effectiveness of SDM in driving quality enhancements.
In Figure 2, we observe that the distribution of the DISO for sites using SDM and sites not using any central monitoring is quite similar. However, when examining the DISC, a stark contrast emerges. Sites using SDM show a much narrower distribution, with 61% of sites no longer considered atypical. In contrast, sites without central monitoring have a broader distribution, with 51% still deemed atypical at the end of the study. Furthermore, for many sites, the DISC was even higher than the DISO, indicating that issues left unaddressed early on tend to persist and even worsen in subsequent data snapshots. This highlights the value of SDM, as it enables the identification of emerging issues, allowing the study team to take corrective actions and adapt their approach, preventing the same problems from impacting future data collected during the trial.
Figure 2. Distribution of the DIS open and DIS close for sites using statistical data monitoring and not using central monitoring (Circle represents the median and the line represents the inter-quartile range, IQR)
Source: CluePoints
In conclusion, SDM has proven to be effective in improving quality in clinical trials. However, the mere availability of the tool is not sufficient. Success depends on having a study team that is not only ready to adopt SDM, but also committed to investigating potential issues, taking appropriate corrective actions, and ensuring proper follow-up. It is this proactive and engaged approach that maximizes the impact of SDM in enhancing trial quality.
Sylviane de Viron, data and knowledge manager, and Steve Young, chief scientific officer; both with CluePoints
Reference
1. de Viron, S., Trotta, L., Steijn, W. et al. Does Central Statistical Monitoring Improve Data Quality? An Analysis of 1,111 Sites in 159 Clinical Trials. Ther Innov Regul Sci 58, 483–494 (2024). https://doi.org/10.1007/s43441-024-00613-w
