MCC Metric of the Month Blog: A Risk-Based Monitoring Metric

MCC has just published an executive summary of its Risk Based Monitoring Usage Survey (check out our web site www.metricschampion.org for a free copy).  This month, let’s look at a metric that’s very important in a Risk-Based Monitoring (RBM) environment: the cycle time from patient visit complete to eCRF data entered.

Why this metric is important:  Ideally, a site should enter a patient’s protocol visit data into the EDC system (or onto a paper CRF) immediately after the patient visit.  However, this often doesn’t happen.  The longer the gap between visit and data entry, the higher the probability of errors.  Late data entry is also problematic in RBM environments, because off-site/remote monitors need timely EDC data to identify potential site problems. So this metric is a good trigger for increased monitoring when using RBM, and provides an indication of other potential site quality problems.

Definition:  The cycle time is calculated as the time – in calendar days – from the date of patient visit to the date when an eCRF page is completed.  A median cycle time for each site should then be calculated over a month or quarter. 

How to calculate this metric:  The formula varies slightly depending upon whether EDC or paper eCRFs are used.  For RBM purposes, an EDC system must be used to assure timely data, so we’ll assume EDC here. But the same concepts apply to paper based CRFs (where the cycle time is calculated to the date the CRF page is entered into the data management system).

• First, calculate the cycle time for each eCRF page as (date of eCRF page entry completed – date of patient visit).

• Once all of the page cycle times have been calculated for a given site, find the median value for each month or quarter for that site.

• Initially, it may be important to calculate trends on a monthly basis.  As sites come down the “protocol learning curve”, calculation can be extended to a quarterly review.

• Medians for countries, regions and entire protocols can also be calculated.

Seven (7) days is a reasonable target for this metric, although 0-1 days is strongly preferred.

Example:  In the graph below, we’ve plotted the median cycle times for a 100-site study for a given month.  Most sites are at or below our target value of 7 days, and many are in the “preferred” zone of 4 days or less.  Fourteen sites are above our 7-day target value and 6 of these sites are in the “problem” zone of at least 10 days.  These “problem” zone sites would likely be targeted for immediate fact-finding and the 14 sites exceeding the 7-day target might receive increased monitoring in an RBM environment.  (note that these zone and target values are examples only and not specific recommendations.)

What you need in order to measure this:  You need the following three things for each patient visit at the end of each month or quarter:

• the completion date for each eCRF page

• the patient visit date related to each eCRF, and

• the site information (site number, country, region, protocol, etc.) for each patient visit.

What makes performance on this metric hard to achieve:  Performance can be hard to achieve on this metric due to complexity of the protocol and CRF, difficulty in using the EDC system, or individual site problems.

Things that you can do to improve performance:  Once you are tracking this metric, the appropriate improvements are dependent on the metric trends that you observe. 

• If the value of the metric is high for most or all sites, the metric is likely pointing to a systemic problem, such as the complexity of the CRF or difficulty in using the EDC system. CRF changes or EDC training should improve performance. 

• On the other hand, if the value of the metric is high for only some sites, the metric is likely pointing to problems at specific sites.  In an RBM environment, this should trigger increased monitoring.  In a non-RBM environment, this should trigger additional fact-finding by monitors and possibly remedial training of site personnel.

Companion metrics:  Other metrics that you should consider in tandem with this metric include: (1) the MCC Site Quality metric and it’s related tracking tool, (2) site query response times and (3) other RBM metrics such as those developed by TransCelerate BioPharma. 

Dave Zuckerman, CEO, Metrics Champion Consortium, [email protected]

Linda Sullivan, COO, Metrics Champion Consortium, 

[email protected]