Quality Remote Monitoring: The Tools of the Game

A critical aspect of risk-based monitoring (RBM) is rapid access to a site’s clinical data. In 2013, industry median values from 2009-2012 Phase II and III clinical trials (see Figure 1 below) showed that the median time from electronic case report (eCRF) entry to data manager query opened was 59 to 89 days. This is even more extraordinary when one considers that the median time from subject visit to query close (all queries including automatically generated queries) ranged from 30 to 36 days.¹

These findings emphasize that direct data entry, either into the electronic data capture (EDC) or eSource systems, provides significant value in overseeing study conduct quality. Mitchel et al.² reported on their experience implementing direct data entry (DDE) and RBM in a clinical trial of 18 investigative sites in the U.S. and Canada studying 180 research subjects. In that trial, 92% of the data was entered within one day of the subject visit and 98% within eight days. Data review was also faster with 50% of the data reviewed within 13 hours of data entry. Source data verification (SDV) was completed at the site for approximately 20% of the data within the EDC. There were changes on 0.8% of the pages, with the majority in three areas: concomitant medications, medical history, and clinical laboratory results.

The evidence above, coupled with the finding that SDV was not an adequate approach to ensure trial quality,³ illustrates the importance of technology and process changes that should be implemented to enhance remote trial oversight as envisioned by the FDA,⁴ European Medicines Agency (EMA),⁵ and International Conference on Harmonization (ICH) guidance⁶ documents on RBM and quality management. The following technology solutions can provide significant benefits to implementing RBM and remote trial management.

Technology solutions

EDC and eSource

Direct data entry can be accomplished though web-based EDC solutions and tablets. It is imperative that sites have adequate Internet access to use tablets for direct data entry. Sites benefit from eliminating transcription of documents. Monitors and data managers also benefit from having immediate access to the data. Questions that document good clinical practice (GCP) compliance can be incorporated into the EDC or eSource. These fields (e.g., detailing timing for vital signs, informed consent processes) enable monitors to conduct source data review remotely. Many data managers may not be familiar with the additional questions the monitor will want to have documented, so cross-functional input into the EDC is needed during design. Tablet setup and testing ensures tablets work as needed by the site. The initiation visit should include site training on how to use the tablets to collect all data, including source data.

eSource tablets are specifically designed to collect data directly at the time of a subject visit (DDE), rather than having the data entered onto paper and transcribed into an EDC system. These systems also meet the FDA guidance and ICH GCP draft guidance (4.9.0) requiring the principles of ALCOA (Attributable, Legible, Contemporaneous, Original, and Accurate) for all data and records. Some focus on providing a “look and feel” like a paper chart. eSource tablets may not be as robust in data exporting and data query capabilities as EDC systems, which can also be used for direct data entry. In addition, allowing sites to enter data anywhere on the electronic chart requires that they be reviewed frequently to assure there are not entries that would indicate a safety or protocol compliance issue. Understanding what queries are possible and what queries require manual entry is important for the data and monitoring team. Often, the clinical team will assume a certain query will be generated by the data collection system, when that may not be the case.

“Site level” data is available from clinical trial management systems (CTMS). However, CTMS systems don’t include all the data needed to manage sites remotely. EDC systems can be set up to collect key site data. Risk assessments, issue management, investigational product receipt, training records, monitoring reports (with sign off), and site delegation logs can be provided from EDC. This provides greater flexibility and delivers key data for reporting and oversight of trial conduct.

IVR/IWR

Interactive voice response/interactive web response (IVR/IWR) outputs can be integrated into EDC or eSource systems using a web services interface. By integrating the data into the EDC, and having the data come directly from the IVR/IWR, fewer changes are needed during reconciliation because the data are not entered separately into two different systems, yielding a faster time to database lock. It also allows the sites to continue working in a single system rather than having to switch systems and transcribe data-which contributes to data errors.

When IVR/IWR is integrated, it is important for the project manager to plan time for integration testing and to assure both systems are ready simultaneously to facilitate integration testing during user acceptance testing.

SAE reporting

When electronic systems are used for data collection (EDC or eSource), no separate manual process is needed for faxing information to the company for manual entry into a separate database, which is the usual practice. A more efficient and effective use of resources is to have the serious adverse event (SAE) data entered into the EDC/eSource system by the site, then transferred to the safety database, if present. All questions about the episode should be generated as queries in the EDC system so it contains the most current and complete SAE data.

When data are integrated throughout the clinical trial, with the EDC containing the most current version of SAEs, there is less time needed for SAE reconciliation, resulting in a shorter time to database lock. In addition, there will be a common understanding of the SAE by all team members.

ePRO, eDiaries, and eConsent

The FDA released guidance on the use of patient reported outcomes in 2009.⁶ That guidance specified the need to have data collected from subjects at protocol-specified times. Since this cannot be documented using paper diaries (which are subject to “parking lot syndrome” or completion of all diaries immediately before a study visit), there has been a significant move to electronic systems. When ePRO data from subjects are integrated into the EDC, it is easier for sites to evaluate data and identify any errors with completion when the site sees the subject in the clinic than when the data are in separate databases. RBM requires the comprehensive review of subject data and confirmation that investigator assessments align with patient-reported outcomes.

eConsent/informed consent form (ICF) errors routinely rank as one of the top five findings during site audits. The complexity of the informed consent process (e.g., multiple consents required for the study and sub studies, such as genetic testing), the frequency of amendments, the number of different languages required, and the number of organizations managing the trial all increase the likelihood of informed consent errors.

eConsent systems are an excellent option for large trials that require the use of multiple languages because the translations can be incorporated directly into the eConsent system. This simplifies the process of informed consent review, particularly when there are amendments to the protocol. The systems that provide eConsent can also provide educational materials to help the research subject better understand the process. An audit trail confirms who signed the document and when. Ideally, on the same date the subject signs the informed consent, it is transferred to the EDC system.

This step eliminates the risk for transcription errors. Costs increase when the ICF requires multiple amendments. The benefit is that there is a greater likelihood that the correct version of the ICF has been provided for the site to use because they are loaded and delivered from a central system, which has been through a validation process.

An alternative to using an electronic system for reviewing ICF is to generate a certified copy of the signed ICF, upload it into the site’s eISF, and review it remotely shortly after the subject visit. This assures you have an executed ICF(s) for each research subject.

Medical coding is traditionally the domain of data management. It is usually performed infrequently during the trial. As part of the within-site and cross-site review, it is valuable to look at the AEs by system, organ, and class (SOC) as part of the review. Simply looking at the rate of AEs across subjects within a site and across sites, may not identify important safety issues-a key requirement of RBM.

eISF and eTMF

The electronic investigator site files (eISF) and electronic trial master files (eTMF) are very important additions needed to fully implement RBM. Many early adopters already have an eTMF, but companies gain great advantages to trial oversight by also incorporating an eISF.

A key component of trial conduct and GCP is the management and oversight of all documentation. The new ICH draft guidance on GCP provides two key requirements that are addressed with an eISF.

• Any copy used to replace an original must fulfill the requirements of a certified copy. 

• The investigator/institution should have control of all essential documents and records generated by the investigator/institution before, during, and after the trial.

  An eISF replaces the site’s regulatory binder. The eISF provides benefits as follows:

• It eliminates the need for having duplicate documents in both the investigator site file and the TMF. The files (eTMF and eISF) are managed together and following the trial, all relevant documents from the eISF (minus any documents with personal health information [PHI]) are merged into the eTMF for final archiving.

• Access control assures that documents containing PHI are seen only by the research site and monitor-not by other users who should not have access to this information.

• Certified copies are easily generated.

• Sites have access to all documents from anywhere; not just the site location.

• No storage space required during the trial and only minimal storage required after completion.

• All documents for each site (including informed consents) are complete and available for review.

• Review of informed consents and other key documents occur rapidly without requiring an onsite visit.

• Document or file audits are performed remotely in a fraction of the time with significant cost savings.

Key requirements for the eISF include assuring investigative sites have access to their entire file, granular access control to protect PHI, audit trails, version control, electronic signatures that meet the 21 CFR Part 11 requirements, and customized archiving so the site’s archive contains subject documents, but the sponsor organization archive does not.

Cost does not have to be an issue with adoption. A variety of solutions exist; some are very cost effective, allow users to set up the file structure and workflow, and can be launched in days. Other solutions have more customized, automated, workflow and take much longer to implement.

Data repositories/reports/data visualizations

Reporting and data visualizations are vital components in successfully implementing RBM. Barnes et al. published an in-depth discussion of the technology requirements for systems used to aggregate and analyze data and metrics on site and staff performance for RBM oversight.⁷ In their discussion, the complexity of managing multiple data sources and technology systems (e.g., EDC, labs, eInformed consent, IVR/IWR, CTMS, payments, eTMF) was highlighted and the need for systems that integrate and aggregate the data. They further provided user requirements for reporting and visualization, a critical component to RBM.

One additional, very important requirement for data repositories is needed-ad hoc reporting. While standard visualizations of key risk indicators with thresholds is important, there is a critical need to be able to develop oversight reports for protocol-specific risks identified during the risk assessment and categorization activity.

A crucial component of the monitor’s role in quality oversight is evaluating the research subject’s appropriateness for the study and safety oversight at the site level. Data visualization tools can enable faster identification of issues that would be difficult to see if a monitor had to “flip” across multiple pages within the CRF and across multiple systems.

For example, it would be important to see that the subject’s reported AE of anemia pre-dated treatment with the study drug. This knowledge allows the monitor to query the AE to see if the anemia worsened and to have the anemia reported as medical history. The graphical patient profiles can be built to interface with many different inputs (e.g., central ECG reading, ePRO vs. investigator reports, etc.). It is imperative that all data sources are represented in the reports, including metadata, not just the EDC data for analysis.

Many life sciences companies have a data repository for analysis of datasets, but the data used is often converted to study data tabulation model (SDTM) for analysis, which is problematic for real-time data review. SDTM conversion for submission of data to regulatory authorities involves data manipulation, conversion, and merging of data into subfolders. Much of this converted data, including metrics that are not included in analysis datasets, are needed for quality oversight reporting. Therefore, new reports may need to be generated using the real-time data sets and metrics.

For some companies with fewer trials, using a data repository with standard RBM and medical monitoring reporting included may provide the most expeditious solution rather than developing their own repository and reporting system. Development of internal data repositories and reporting tools requires significant resources for development and validation as well as managing the process of regular data imports from multiple systems. In addition, it is important to have a system that will enable ad hoc reporting to allow further evaluations if issues are identified. Having a system to develop and manage study-specific issue logs with input from multiple team members is another key requirement of technologies used for risk identification and analysis.

Conclusion

Many of the approaches that we recommend are not required for RBM, but all of the tools discussed make the review of critical data and processes better-which is a key component of RBM. While eSource is not required for RBM, it is a valuable tool because it eliminates the need for transcription, enables immediate review of data

(which is a component recommended in the RBM guidance), and aligns with the FDA guidance on eSource and ICH draft GCP guidance.

Remote review of informed consent forms is not required in RBM, but the review of critical documents for subject safety and GCP compliance is a critical component of quality management. Being able to perform that review remotely without waiting for onsite visits aligns with the FDA’s spirit of rapid, focused review. Our recommendation is that by implementing these tools, you enhance quality oversight and focus efforts on the areas of highest risk: protocol compliance, GCP compliance, investigational product management, and patient safety. 

Penelope Manasco, MD, is CEO of MANA RBM; email: Pmanasco@manarbm.com

References

1. Are your data cleaning cycle times out of control? Blog.mdsol.com. http://blog.mdsol.com/are-your-data-cleaning-cycle-times-out-of-control/

2. Mitchel JT, Cho T, Gittleman DA, Markowitz JMS, Kim YJ, Choi J, Hamrell MR, Carrara D, Nora SD. Time to Change the Clinical Trial Monitoring Paradigm. Applied Clinical Trials. Jan. 17, 2014

3. Sheetz N, Wilson B, Benedict J, Huffman E, Lawton A, Travers M, Nadolny P, Young S, Given K, Florin L. Evaluating Source Data Verification as a Quality Control Measure in Clinical Trials. Therapeutic Innovation and Regulatory Science.48: 6: 671-680. November 2014

4. Oversight of Clinical Investigations: A Risk-Based Approach to Monitoring.  U.S. Department of HHS, FDA, August 2013 OMB Control No. 0910-0733.

5. Reflection Paper on Risk-Based Quality Management in Clinical Trials. European Medicines Agency. 18 November 2013 EMA/269011/2013

6. Guidance for Industry: Patient Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. US Department of HHS, U.S. FDA, December 2009. http://www.fda.gov/downloads/Drugs/Guidances/UCM193282.pdf

7. Barnes S, Katta N, Sanford N, Staigers T, Verish T, Technology Considerations to Enable the Risk-Based Monitoring Methodology. Therapeutic Innovation and Regulatory Science, 2014, Vol 48(5) 536-545. August 2014