Competencies for the Changing Role of the Clinical Study Monitor: Implementing A Risk-Based Approach to Monitoring

Abstract:

The advent of Risk Based Monitoring will change the traditional Clinical Study Monitor’s (Monitor) role significantly. Source Document Verification (SDV) or transcription checking now consumes most of a current monitor’s time. The new Monitor’s role requires analytic, data interpretation and assessment skills, greater data-oriented communication capabilities, and the ability to learn and teach the basics of new technology to others. We developed these competencies through feedback from a group of clinical research professionals with extensive experience in clinical research studies. Developing competencies is the first critical step in designing and implementing training programs for the next generation of clinical study monitors.

Background:

In late 2013, the Food and Drug Administration (FDA) released final guidance on both “Oversight of Clinical Investigations –A Risk Based Approach to Monitoring”¹ and “Electronic Source Data in Clinical Investigations”². The European Medicines Agency (EMA) also recently released its reflection paper on risk based quality management of clinical trials.^3,4

The FDA and EMA identified significant limitations in the way clinical trial oversight is currently conducted. These limitations were clearly illustrated when Pfizer5, J&J⁶ and ICON⁷ received FDA 483 letters for significant lapses in oversight of trials despite the trials using 100% SDV of data at the site. In these examples, data errors were only identified when data were evaluated more holistically—across subjects and across sites.

The FDA and EMA recommended a more centralized approach to monitoring; one that included a broad variety of performance metrics to identify problems at the research site and eliminated or severely limited the amount of SDV performed.

Numerous papers have described the limitations of SDV, which is only a check of the data to identify transcription errors. Nahm et al. ⁵ reported that transcription errors occurred in less than 0.1% of data and that the design of sourcedocuments and data collection systems could enhance the quality of the transcription. Smith et al. found that SDV was expensive and identified random errors that made little impact on results and clinical conclusions of the trial.

Central monitoring using an external data source was found to be a more efficient approach for assessing the primary outcome of Overall Survival in their research study." ⁶

TransCelerate, a non profit organization founded by multiple Pharmaceutical companies to advance innovation in Research and Development, stated in its position paper on Risk Based Monitoring (RBM)⁷:
“SDV, commonly known as “transcription checking” is the process by which data within the case report from (CRF) or other data collection systems are compared to the original source of the information (and vice versa) to confirm that the datawere transcribed accurately. “ TransCelerate further noted that: “as a quality control measure it was the opinion of the RBM working group that SDV as purely a transcription checking exercise was not valuable.”

TransCelerate created a second term, Source Data Review (SDR), which focused on review of source documentation in the context of a comprehensive review of data quality. This review includes checking for protocol compliance, adequacy, appropriateness of subject selection, and compliance with regulations and standard operating procedures. TransCelerate further stated in its position paper, “SDR is not a comparison of source data against CRF data.”

These higher level skills and more dynamic approach to site performance assessment require both a change in skill sets for Monitors (and subsequent training) as well as incorporating changes in process. While it is easy to implement a plan to have onsite monitors simply verify data between a paper form and electronic data capture (EDC), the process for pulling data from multiple sources and making it available for sound decision-making is more complex and requires close interactions among project managers, monitors, data managers, and medical monitors.

The Monitor’s role in RBM is one component of an entire process of “Quality by Design” for a clinical trial. This process starts before the protocol is written and is incorporated into the protocol and a Quality Risk Management Plan.

A risk-based approach to study monitoring is dynamic and more readily facilitates continual improvement in trial conduct and oversight. For example, instead of simply identifying transcription errors, the monitor and the rest of the trial conduct team now focus its time and resources to evaluate site findings in the context of a single subject, all subjects at the site, across sites in a region, or across all sites in a study. Based on this review, the team will determine root cause(s) and what additional actions (e.g., training of clinical investigator and site staff, clarification of protocol requirements, design of CRFs and/or source documents, etc.) are necessary to ensure human subject protection and data quality across sites. The Monitor must also use specific performance criteria to determine whether the site has resolved identified, problem issues.

In addition to the RBM-adjusted monitoring competencies listed below, specific, critical data and processes associated with each protocol will be identified as areas for closer review. Examples of critical data and processes are listed below in the competencies, however, a cross-functional risk analysis of the study program and the specific protocol must be completed to identify high risk areas and methods for additional oversight of these domains. These critical data and process areas will be listed in the Monitoring plan and may be referenced in the Quality Risk Assessment and Management Plan.

Process to Develop Competencies:

We developed a set of Monitor Competencies and distributed them through Linked In and other personal networks to obtain feedback on the criteria. We compiled this feedback to create the list of competencies in this paper. We recognize variability may exist in how each organization develops the processes for the Monitoring role. In some organizations, monitor tasks may be split between a number of job roles, some not traditionally coming from clinical operations.

When Risk Based Monitoring approaches are adopted and organizations move to remote, centralized study oversight and management, the Monitor’s role is significantly more complex. The industry will have to acknowledge and reward this expanded expertise. Where more than one person performs these tasks or roles, duplication of effort needs to be avoided and excellent communication among the different clinical trial roles (monitor, data manager, medical monitor, project manager, document manager, regulatory, etc.) is critical.

To recognize the added expectations and new competencies required of the traditional monitoring role, we propose a new name “Site Manager” to replace the title of “Site Monitor”.

Monitoring Competencies Required for Site Managers

1. Apply the trial process and data risk assessment results to the qualification and selection of investigators and sites.
/a) Principal Investigators and sub-investigators have the appropriate knowledge, skills, and experience to perform the protocol. No Principal Investigator (PI) or sub-investigator has been debarred.
b) Staff has the appropriate training, processes, and time to conduct the study.
c) The study site has the appropriate patient population.
d) Facilities are sufficient to conduct the trial. This includes storage of investigational product, processing of samples, or other special, required assessments.
e) The IRB/EC meets all requirements for review of human subjects protection.

2. Evaluate clinical data from a research subject to identify whether:
a. Subject meets the inclusion and exclusion criteria for the research protocol. This ensures risk avoidance (exclusion criteria—subjects that should not be enrolled in the study) and risk reduction (inclusion criteria—assuring that all subjects are appropriate for the trial).
b. Subject’s data are logical and do not contradict each other. (e.g., if an adverse event is reported for a disease that would require a pharmaceutical treatment, that treatment is listed on a concomitant medication page).
c. Subject data indicates an adverse event (AE) or Serious adverse event (SAE) has occurred or data identifies an Unexpected or Unanticipated Event. Site Managers should be able to assure that the categorization is appropriate: SAE or Suspected Unexpected Serious Adverse Reaction (SUSAR), or Unanticipated Adverse Device Effect (UADE) and that the site performs all required assessments in the appropriate time frame. The Site Manager should be able to review reports to identify whether all appropriate data have been included (e.g., concomitant medications used to treat a SAE).
d. Subject has adequate human subject protection and protection of protected health information and it is appropriately documented. Assure, if applicable, subjects are able to obtain results from the research and associated counseling if appropriate (e.g. genetic research and results).
e. Safety Reporting requirements are met based on regulatory and Institutional Review Board (IRB) /International Ethics Committee (IEC) requirements (e.g., reporting of a Serious Adverse Event).
f. Laboratory data are complete, accurate, and reviewed to determine if there is an increased safety risk to the research subject based on subject’s laboratory values (e.g. large number of values outside the normal range, labs rated as clinically significant). This review should be done at the subject level, the site level, and the region level.

3. Evaluate data from multiple subjects to determine whether:
a) Site is performing and documenting the study appropriately according to the protocol.
b) Site is performing the study according to Good Clinical Practice and any other regulatory requirements (e.g., Health Insurance Portability and Accountability Act, (HIPAA)).
c) A Site (or user) is performing outside the norm based on specific domains for centralized risk based monitoring (see #4) and source data review.
d) There is evidence of fraud.
4. Perform centralized review and interpret data on site performance in comparison with other sites, which may include range checks, completeness of data, checks of unusual distribution of data within and between sites (such as too little variance), and identify the domains, if any, in which the site is having problems. Domains to be evaluated are as follows:
o Recruitment
o Screening Process
o Informed Consent (Special country requirements must be clear, for instance, audio video recording of informed consent has been mandatory in India since November 2013)
o Data Quality/Completeness/Timeliness
o Query Response
o Safety
o Deviations from Protocol
o Premature Discontinuations & relation to adverse events
o Key Endpoints (Primary and Secondary) and Critical Data and Process Assessments
o IRB/IEC Interactions
o Investigational Product Management
o Documentation
o Investigator oversight/review
o Change Management

5. Evaluate regulatory and administrative tasks, such as:
• Continuous IRB/IEC approval by reviewing electronic IRB/IEC correspondence
• Accuracy of investigational product assignment, distribution, and reconciliation
• Compliance to procedures essential to trial integrity (e.g., blinding procedures)
• Completeness of site documentation and archiving processes
• Investigator oversight of the clinical trial
• Management of changes in site staff, protocol, technology, etc.

6. Understand how clinical trial technology works, including what each system is able to do and what it cannot do. This includes understanding:
o Study Design and validation process including User Acceptance Testing
o Role-based access and privileges
o How to use Audit Trail information as part of centralized monitoring
o Electronic Signature
o Specific function of the technology for a trial to enable training and answering site questions
7. Understand the inter-dependency between Site Management and the role of the Clinical Data Manager. This includes:
• How and when centralized data review occurs for each group
• How data management output can support clinical site management
• Specifying and interpreting metrics reports and trend reports on subject-to-subject data comparisons and site-to-site performance comparisons
• How data needed for reporting is incorporated into the eCRF and data exports
• Communication flow between Site Management and Data Management

8. Develop consistent strategies for assessing and resolving problems. Identify root cause, develop and implement training and/or other remediation strategies that can be used in person or remotely to correct and prevent issues identified for a site or for the study. Determine criteria for issue resolution or escalation. Document process for remediation until resolution.
9. Maintain record of all monitoring activities (remote and on-site) including issue evaluation, root cause identification, remediation, and confirmation of effectiveness of remediation.

Discussion:

Adopting a Risk-Based monitoring approach to clinical trial conduct results in a significant change to the Monitor’s activities and the associated skill sets needed. The first step in developing training for the new monitoring roles is to define a set of core competencies. We submit these proposed competencies to the industry as a starting point in the evolution of the new monitor role. Further, to acknowledge these enhanced competencies, we suggest changing the title of Monitor to “Site Manager”.

While some people questioned the incorporation of Competency 6 (understanding clinical trial technology) in the Core Competencies, we believe it is critical. The adoption of centralized monitoring requires Site Managers to understand what the technology can and cannot do and how technology will affect their job. For instance, the monitor needs to know how edits are handled across different systems (EDC and eSource). Also, with increased adoption of electronic Source, which does not require transcription, the traditional monitor needs to understand that SDV is eliminated and consequently there is a new setof expectations about how they will support trial oversight.

With electronic Investigator Site Files (eISF), the Site Manager can now manage all regulatory documents and informed consents remotely. The Site Manager must understand regulatory requirements such as 21 CFR Part 11 and how it relates to the eISF, which are the purview of the monitors and clinical operations in most studies. The Site Manager is also the first line to answer site questions, so Monitors need to have a good understanding of how the EDC or eSource and eTMF and eISF work.

The next step in the process is to identify specific criteria to determine which areas the Monitors will need additional training. We are developing a self assessment tool to enable Monitors to assess their areas of additional training needs. Upon completion, we will make this self-assessment tool available to the industry.

Summary:

This paper presents a list of core competencies required for monitors to transition from simple SDV to the more analytic role needed for Risk Based Monitoring. This is a critical first step in the process of developing the new, more analytic roleof the Site Managers of the future.

References:

1. Oversight of Clinical Investigations: A Risk-Based Approach to Monitoring. U.S. Department of HHS, FDA, August 2013 OMB Control No. 0910-0733.
2. Reflection paper on risk based quality management in clinical trials. European Medicines Agency. 18 November 2013 EMA/269011/2013.
3. Electronic Source Data in Clinical Investigations. U.S. Department of Health and Human Services, Food and Drug Administration. Sept 2013.
4. Reflection paper on expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials. 09 June 2010. EMA/INS/GCP/454280/2010.
5. Ball, M.D, Leslie K. "Pfizer Warning Letter." Letter to Mr. Martin Mackay, Ph.D. 9 Apr. 2010.FDA U.S. Food and Drug Administration. U.S. Department of Health and Human Services. Web.

http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/ucm208 976.htm

.
6. Ball, M.D, Leslie K. "Johnson & Johnson Pharmaceutical Research & Development Warning Letter." Letter to Karen Grosser, Ph.D., M.B.A. 10 Aug. 2009. FDA U.S. Food and Drug Administration. U.S. Department of Health and Human Services. Web.

http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2009/uc m177398.htm

.

7. Ball, M.D, Leslie K. "ICON Clinical Research, Inc. WarningLetter."LettertoJohnW.Hubbard,Ph.D.27Nov.2009.FDAU.S.FoodandDrugAdministration.U.S.DepartmentofHealthandHumanServices.Web.

http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2009/ucm193156.htm

.

8. Nahm M, Pieper CF, Cunningham MM. (2008) Quantifying Data Qualityfor Clinical Trials Using Electronic Data Capture. PLoS ONE 3(8): e3049doi:10.1371/journal.pone.0003049.
9. Smith CT, Stocken DD, Dunn J, Cox T, Ghaneh P, Cunningham D,Neoptolemos JP. (2012) The Value of Source Data Verification in aCancer Clinical Trial. PLoS ONE 7(12):e51623.Dol:10.1371/journal.pone.0051623.
10. Risk-Based Monitoring Update-Volume 1. 27 Jan2014. TransCelerateBioPharma Inc.

http://www.transceleratebiopharmainc.com/wpcontent/uploads/2014/01/TransCelerate-RBM-Update-Volume-I-FINAL-27JAN2014.pdf

11. Position Paper: Risk Based Monitoring Methodology. 2013.

http://www.transceleratebiopharmainc.com/wpcontent/uploads/2013/10/TransCelerate-RBM-Position-Paper-FINAL-30MAY2013.pdf

By: Libby Cerullo¹, Catherine Radovich², Inder sen Gandi³, Beat Wilder⁴, Charlene Stubbs¹, Christine Riley-Wagenmann¹, Rosie McKellar⁵, and Penelope K. Manasco¹

¹MANA Consulting LLC, Denver CO 80202. Address inquiries to [email protected]² University of Michigan Medical School;³ Independent Medical Consultant, Hyderabad, India ; ⁴Wilder&Schiemann Ltd. Switzerland, ⁵ INC Research, Raleigh, NC.