Importance of Data Entry Timeliness

But MCC survey indicates poor oversight undermines ability to monitor study quality

With the adoption of the addendum to GCP, ICH E6 (R2), there are some significant developments that are needed by the industry in relation to identification and management of risk and defining critical data points. As stated in section 5.0 Quality Management, “The sponsor should implement a system to manage quality throughout all stages of the trial process. Sponsors should focus on trial activities essential to ensuring human subject protection and the reliability of trial results.” Section 5.0.1 Critical Process and Data Identification starts “During protocol development, the sponsor should identify those processes and data that are critical to ensure human subject protection and the reliability of trial results.”¹

The industry group Metrics Champion Consortium (MCC) and its member organizations wanted to explore the timeliness of data entry for critical data points by sites. Until data are entered, there can be neither centralized monitoring nor the meaningful remote analysis that is needed for risk-based monitoring.² Without the data being entered in a timely way, signals of potential safety or quality concerns or the need for increased site monitoring will take longer to emerge – leading to increased risks for subjects and for the trial. MCC and its members have defined clinical trial performance metrics for use by MCC member organizations.³ MCC Clinical Trial Performance Metric # 18 (v1.2) measures the cycle time of data entry - date patient visit complete to date all data entered into the Electronic Data Capture (EDC) system. The metric definition (e.g. definition of “data entry complete”) and the performance target are under review in light of the ICH E6 (R2) Addendum. Following discussions with member organizations, MCC conducted a cross-industry survey to understand better the expectations and actual performance for this metric as well as how the metric is, in practice, defined and metrics results used by organizations.⁴ This article explores some of the key results.

Survey and results

The 28-question, web-based survey was conducted June through August 2016. There were 35 respondents primarily from sponsors and Contract Research Organizations (CROs) both large and small. 26% of respondents identified themselves as “Data Manager”, 23% as “Functional Area Manager” and 17% as “Clinical Operations / Monitoring Manager.” The results should be taken as indicative if not necessarily fully representative of practices across the industry.

The industry considers timeliness of data entry into EDC (Electronic Data Capture) by sites to be very important. All respondents measure data entry cycle times and acknowledge the importance of timely data entry following subject visit. 89% of those surveyed consider this “Very important” with the remainder finding it “Somewhat important”. The one “Other” response was because the respondent considered that the cycle times are “somewhat important until DB Lock…when it becomes very important”. [Figure 1]

Figure 1:Responses to the question “How important is it for your organization to have sites enter subject data into the EDC as soon as possible following a subject visit?” (Note that percentages do not add to 100 due to rounding)Organizations encourage timely data entry to identify safety signals early and increase data quality. Respondents were asked to select “as many that applied” from a list of reasons for encouraging sites to enter subject data into EDC in a timely manner [Figure 2]. The most common reasons noted are to identify safety signals early and to increase data quality. 94% of respondents acknowledged the benefit of identifying safety signals early and 91% acknowledged the value in increasing data quality.

Given the move in recent years to risk-based monitoring,^2,5 it is somewhat surprising that barely half (54%) of respondents list modulating monitoring intensity as a reason to encourage sites to enter subject data into EDC in a timely manner. This is a further use of the metric we would expect to see increase in coming years.

Figure 2:Reasons organizations encourage sites to enter subject data into EDC in a timely mannerIndustry lacks agreement on how to define the “data entry complete” time point. Survey results reveal that organizations most often define data entry complete in one of two ways:

• Data entry “complete” when a single data point for the visit has been entered into EDC (43%)

• Data entry “complete” when all data for a complete visit have been entered into EDC (31%)

A smaller proportion of respondents defined data entry “complete” when all critical data have been entered into the EDC. This definition appears to match most closely with the aim of assessing risk (safety and data quality) in a trial. Indeed, the focus on critical data is explicitly stated in ICH E6 (R2).

Site data entry cycle time expectations for “single data” time point group not significantly different from “all data” entered group. Many organizations provided the maximum number of days they expected sites to enter subject visit data. Comparing the cycle time expectations for organizations that defined data entry complete as all data entered to those who defined data entry complete as when a single data point is entered revealed no significant difference in expectations between the two groups. Whilst we expected to see shorter cycle time expectations with organizations that define data entry complete when a single data point is entered, the results did not support this supposition. Additionally, the survey revealed surprising results when comparing expectations with actual cycle times.

Some organizations cite challenges with measuring data entry cycle time related to EDC system constraints. Some of the free text comments provided by respondents point to frustrations with measuring this cycle time metric – “This metric is one of the most complicated to calculate,” “This is a very difficult metric to measure in a meaningful way but is very important.” Several respondents mentioned that different EDC systems define the cycle time in different ways. Of the three EDC vendors that participated in the survey, two defined data entry “complete” when the first data point is entered and one defined it as all critical data are entered.

Discussion

The industry agrees that it is very important for sites to enter data into the EDC as soon as possible following a subject visit. Nearly all respondents cite identifying safety signals early and increasing data quality as important reasons to encourage sites to enter subject data into EDC in a timely manner. Given the growing prevalence of risk-based monitoring,^2,5 it is surprising that barely half (54%) of respondents cited modulating monitoring intensity as a reason to encourage sites to enter subject data into EDC in a timely manner. Timely data entry is an important component of risk-based monitoring for several reasons. First, slow site data entry can impact the credibility and usefulness of centralized monitoring data analytic reports. Much of the data needed to run the analytics is input by the sites following a subject visit. Thus, when sites delay entering subject visit data, the data reflected in the reports is out of date. This raises the question of whether safety and/or data quality issues are being identified or detected in a timely manner resulting in concerns about whether risks are being managed as effectively as they could be. Additionally, site data entry cycle time performance is used as a site key risk indicator in risk-based monitoring programs. When sites are slow to complete data entry, there is concern that the PI and site staff don't view the study as a priority study and/or the site has insufficient staff to complete data entry within the expected time line. In either case, the site monitor needs to investigate what is happening at the site and determine whether additional intervention is required.

Organizations vary significantly in how they calculate site data entry cycle time metrics. Specifically, there is no standardized definition of the “data entry complete” end point – some define data entry complete when one data element is entered, others define data entry complete when all visit data is entered and a small portion define data entry complete when critical data is entered. The lack of a common definition of “data entry complete” could have significant implications for the industry as organizations aggregate site data entry performance data to identify appropriate sites for new studies and explore the correlation between site data entry cycle times and overall site performance and data quality measures. Comparing results of the three metric versions described above is akin to comparing apples, oranges and pears – they are in essence three different metrics! Site data entry can be delayed when sites are waiting for results of tests run during the subject visit. Is it fair to compare “one data entered” cycle times to “all data entered” cycle time results? Additionally, if statistical analyses show no correlation between data entry cycle times and data quality and/or overall site performance does that mean that there is no relationship or are differing definitions of data entry complete impacting the analyses.

Call to action

The industry has an opportunity to standardize the definition of this important cycle time metric. ICH E6 (R2) section 5.0 states that organizations should identify critical processes and critical data as a beginning step of risk-based quality management.¹ As organizations implement risk-based quality management systems to focus on monitoring critical data, they should update their site data entry cycle time metric definition to align with critical data entry complete.

The industry should work together across the value chain (sponsors, CROs, investigator sites, EDC vendors) to:

• Standardize the definition of the “data entry complete” end-point across the industry

• Implement the definition in EDC systems

• Based on the needs identified (early detection of safety and data quality risks) determine an industry expectation/target for the cycle time that can also be implemented in EDC systems

• Conduct a study to determine whether reduced cycle times correlates to increased data quality and/or overall study quality

• Develop best practice for how to help sites comply with the expectation – likely to include ensuring sites are aware, providing feedback on performance, clear consequences for good/poor performance

Written by Linda B. Sullivan, MBA and Keith W. Dorricott, MBB

References

1. International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH). ICH HARMONISED GUIDELINE - INTEGRATED ADDENDUM TO ICH E6(R1): GUIDELINE FOR GOOD CLINICAL PRACTICE E6(R2). Current Step 4 version dated 9 November 2016. http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/ Efficacy/E6/E6_R2__Step_4.pdf Accessed 15 February 2017.
2. US Food and Drug Administration. Guidance for Industry: Oversight of Clinical Investigations - A Risk-Based Approach to Monitoring 0910-0733; Aug 2013 http://www.fda.gov/downloads/Drugs/Guidances/UCM269919.pdf Accessed 15 February 2017.
3. Metrics Champion Consortium. http://metricschampion.org/performance-metrics/ Accessed 15 February 2017.
4. MCC Industry Practices Insight Report - Clinical Trial Site Data Entry Study: An examination of how organizations measure site data entry cycle times, performance expectations compared to actual results and site management strategies, Metrics Champion Consortium, Apr 2017. http://metricschampion.org/publications-2/
5. European Medicines Agency. Reflection paper on risk based quality management in clinical trials, EMA/269011/2013; Nov 2013. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_ guideline/2013/11/WC500155491.pdf Accessed 15 February 2017.