Does Your CRO Comply with ICH E6?

RBM Questions Every Sponsor Should Ask to Confirm Your CRO Complies with New GCP

The ICH E6 (R2) new guidelines for GCP became final in November 2016. This guidance fundamentally changed the requirements for GCP, which now must include a total quality approach. Your evaluation of vendors needs to confirm the implementation and dynamic management of the new GCP guidelines.

Risk Based Monitoring (RBM) has been implemented in a wide variety of ways. Some provide a significant benefit to quality oversight while others simply decrease the number of monitoring visits or decrease the amount of data that is source data verified and call that RBM. That could not be further from the spirit of the FDA Guidance or the new ICH GCP Guidance.

To help you understand the requirements and guide you in knowing what questions to ask, I have provided the following checklist and the type of answers you should expect.

1. Explain your approach for Quality Management (addendum 5.0)

What you want to hear: Your CRO should have a comprehensive quality management approach. One to capture and categorize risks with an oversight process that synthesizes review across all functions, focusing on critical data integrity, trial subject protection, and safety.  

The approach should be customized to the specific data collected in your trial. It should have specific approaches that define exactly how and when high-risk data will be reviewed. You should expect near real-time review of your critical data.  

There should be a process for issue management visible to all and that can demonstrate the process of identification of issues, root cause analysis of major issues, and a process for following issues to resolution.

Questions to ask:

•    What are your processes for Critical Process and Data Identification? (if CRO develops protocol-otherwise a sponsor activity).  

•    What are your processes for Risk Identification? This includes system risk identification and trial-specific risks. Who does the work, how are the risks captured, and how does this translate into oversight plans?

•    Risk Evaluation; For each risk, identify the likelihood of occurring, impact, and extent that error would be detectable.

• The Risk Assessment and Categorization Tool provided by TransCelerate provides a way to categorize the risks, but does not provide the critical information of how to translate the risks into an action plan. Therefore, if a CRO says they use the TransCelerate RACT tool, ask the next important question: Since the risk assessment guides the identification of your high- risk areas, explain how that data is translated into the data review plan. How are the two activities linked? There should be a process for assuring the high-risk areas are monitored quickly and completely.

•    Risk Control. Risks that can be reduced should be reduced.  

• How does the CRO incorporate risk mitigation activities for the risks identified in the Risk Identification? How does the CRO determine tolerance limits to identify systematic issues that affect subject safety or data integrity? Your CRO’s plans should reflect these activities.  

• Key Risk Indicators (KRI) are often used as indicators of site performance. These are lagging indicators (i.e., the issues have already occurred) and they usually do not reflect specific risks to your trial. While helpful, this should not be the ONLY aspect of risk oversight your CRO provides, nor should review of data be restricted only to those sites with KRIs that fall outside the required range.

• Key Risk Indicators should also be sensitive. If all indicators used for your trial are always “green,” are the triggers set finely enough?     

•    Risk Communication. How does the CRO communicate quality management activities to the site, team, and sponsor to facilitate risk review and continuous improvement?
•    Risk Review. How does the CRO facilitate Risk Review with updates to reflect evaluation of current risk management processes? How is the Risk Review performed with regard to the Risk Elements identified in Risk Identification? Is there a monthly meeting to review performance? Who attends the meeting? Are the minutes provided to the entire project team? Does performance include both the sites and the CRO? How does the CRO provide performance feedback to sites?
•    Risk Reporting. How does the CRO describe its quality management approach and summarize deviations from tolerance limits? How is the risk reporting accomplished? Who receives the reports? Does the data go to the sponsor or only a high-level report? How broad is the reporting? Do the sites get any type of performance report? Is a CRO performance included? Is the output of the Risk Reporting integrated into the clinical monitoring report?
•    Issue Management. How does the CRO manage issues? Is there a method to allow root cause analysis of major issues? Is there a way for the sponsor to see that issues are closed efficiently and that the appropriate follow-up, including confirmation that any remediation was successful? Is there a way to aggregate issues across technology, sites, and the study team? Does the sponsor have full access to all issues reported?

2. Explain your approach for rapid remote review of subject data and documents. How do you focus on the areas of greatest risk: Primary and secondary endpoints, safety assessments and maintaining human subject protection?

What you want to hear: Subject review occurs within x number of days. It is not predicated on having an onsite visit and is focused on assuring that subject data affecting safety and efficacy are reviewed promptly for missing or incorrect data.  

Remote or Central review is listed as an option in the FDA guidance. The key aspects for you to understand are:

•    Who does the review and how is that integrated with the on-site monitoring visits?

•    For subject data oversight:  

• What data do they review? If remote review only looks at subject CRF data alone, then monitors may potentially miss issues in other data sets such as labs, ePRO data, Investigational product management, and other data sources.

• How was the review guided by the Risk Assessment performed?

• If the review was remote, how was the document review incorporated into your process? If the CRO is not going out to the sites frequently, how can you be assured the informed consent forms (ICFs) are being reviewed? Informed consent errors are consistently one of the top five findings on investigator audits. You need to ensure the CRO has a plan to review the ICFs rapidly and completely.

• Is the review focused solely on Source Data Verification (SDV)? If so, SDV has been shown repeatedly to be an ineffective method of quality oversight. A warning sign for you should be if the CRO says they do only SDV of Primary and Secondary endpoints.

•    What is the approach for overall site performance review?

• Comprehensive review of site performance metrics across subjects including enrollment, deviations, early terminations, safety reporting, training, site turnover, investigational product management, and informed consent should be part of the CRO approach. Ask them to provide an example of how that review is accomplished and how the data are shared with the Sponsor and the sites.

• How is issue management incorporated into the review process (more to come in #3 below)?

• How can you, as a sponsor, access the performance metrics and evaluate performance of both the sites and the CRO?

• How are deviations evaluated? Is trend analysis done on the deviations and do you as the sponsor have access to the deviations and date they were identified?

• How are central monitoring activities reported? Ask for a copy of a central monitor report. (Addendum to 1.39 Monitoring Report)

• Ask to see an example of the Clinical Monitoring Plan that describes their methodology.

3. How are the computer systems validated (Addendum 1.60.1)?

• Does the CRO have SOPs for system set up, installation, and use? Some areas to focus on include validation, security, change control, data backup and recovery, and user training and access.  

• Are the responsibilities of the sponsor, Investigator and other parties clear?

• Is the integrity of the data assured, particularly when making changes?

You should have access to specifications, user acceptance testing results, and help desk reports. These help desk reports can be a valuable tool to identify whether the system(s) are working as designed. You should ask additional questions any time the system is modified during the trial (e.g., Have the exports been updated after a change is made?). One important recommendation (we consider it a requirement) is to have the data from all data sets able to be combined, so that it is easier to identify errors.

4. Explain your approach for managing site training (Addendum 4.2.6) and study staff. This is important for all user training on systems as well as study procedures.

The training plan ideally will include how replacement staff will be trained and documentation of the training materials used. 21 CFR Part 11 requires all users be trained to use all systems. This must be documented for each system.

5. Explain your use of source data. (Addendum 4.9.0) How does your CRO assure that the documents meet ALCOA principles?

• Attributable

• Legible

• Contemporaneous

• Original

• Accurate (and complete)

Changes should be traceable, should not obscure the original entry, and should be explained if necessary (e.g., via audit trail). The ALCOA principles and the FDA and EMA guidance on electronic source encourage direct entry of data into a technology system. This assures more rapid review of the data and allows errors to be identified earlier.  

Electronic Source Systems, Electronic Patient Reported Outcomes, and Electronic Data Capture using direct data entry should be the answers that you receive for this question.

If electronic systems are used, the person doing the work (Attributable) enters the data contemporaneously with the subject visit. An additional benefit is that errors previously attributed to an entire site can be traced to an individual user, aiding identification and correction of errors.

Having the paper source completed and scanned so the CRO can do remote SDV does not meet the spirit of ALCOA. First, the source may not be attributable and entry of the data is not contemporaneous or original, both requirements embraced by the new ICH E6 (R2).

6. Explain how you manage essential documents.

If site visits are not occurring, how are informed consents being reviewed? How can your CRO be sure documents are available and audit ready at each site?

7. How does your CRO support improved site performance?  

Conducting a clinical trial involves many players, but the most important is the clinical research site. The site staff are in direct contact with the research subjects. The PI has primary responsibility for the medical care of research subjects.

How does the CRO communicate performance information to the research site? We all expect to get performance feedback in our jobs-it is how we improve our performance. Does your CRO have a plan to provide comprehensive performance feedback to the site? If not, how can we expect the site to improve and how can the site identify staff that may not be performing as expected?  

8. How does your CRO support sponsor oversight? What access to data will you have?

Is there a way for the sponsor to evaluate monitoring reports to assure that all are completed as planned? Electronic reports are superior to having to review PDF copies of all monitoring reports. Ideally you can search or run reports on the frequency of issues, if not provided by your CRO.

You should expect reports from your CRO on whether monitoring visits are being performed as defined in plans. You should also expect the CRO to provide clear visibility into important aspects of sponsor oversight such as deviation evaluation.  

Access to data to perform its own inquiries into study conduct is one of the biggest issues Sponsors identify as a challenge working with a CRO. If your access comes only from spreadsheets that you must request, you may not know exactly what is happening and you might not be able to identify issues that occur, even if they are missed by your CRO.  

While many in the industry have quickly (and we believe prematurely) dismissed the guidance as not improving quality, a critical focus should include how your CROs implement the new requirements. By making these requirements part of the GCP requirements, it is the responsibility of everyone conducting clinical trials to have a plan.  

The tools needed to formulate and implement the plan do not have to cost an exorbitant amount. We have found that implementing the processes, technology, and training needed to adopt the new guidance has improved the conduct of our trials, without increasing costs. You should, however, expect changes in where your clinical trial budget line items are assigned. Look for more insight into that area in a future paper.

Penelope K. Manasco, MD is CEO of MANA RBM. She can be reached at pmanasco@manarbm.com