A Structured Approach to Implementing a Risk-based Monitoring Model for Trial Conduct

Introduction

In today’s risk-aversive clinical trial environment, few clearly defined methodologies exist to aid sponsors and contract research organizations (CROs) in the identification, analysis and mitigation of monitoring risk factors for clinical trial conduct. Without a structured approach to model implementation, methodological variability may introduce process inefficiency, increased patient safety risk, increased operational expenditure and failure to improve overall trial quality and compliance. 

An effective risk mitigation model requires the development of a structured approach to risk identification, analysis and control, from trial inception to completion-one that is repeatable, reproducible, sustainable and adaptable in meeting regulatory demands and achieving quality outcomes (see Figure 1). While there are a variety of ways to structure a model, the focus should center on the core elements driving safety, compliance and quality. This article provides insight into the principles underlying trial conduct transformation and provides the reader with the tools and knowledge necessary to successfully implement the transition to risk-based monitoring (RBM).

Rationale for change: inadequacy of traditional 100% source document verification

As industry moves aggressively toward RBM, several initiatives are underway to adopt a structured method to implementing a more efficient, cost-effective approach to monitoring and quality trial conduct. Designed to protect the rights and welfare of study subjects, to promote validity and integrity of data generated, and to demonstrate product safety, these quality initiatives provide sponsors with the structure necessary to make a quality management system (QMS) work. Traditionally, industry resisted moving away from 100% source data verification (SDV), believing it was needed for all trials regardless of trial complexity or study design. Yet by itself, 100% SDV is not effective, and regulatory inspections continue to uncover noncompliance issues.¹ Further, TransCelerate conducted a retrospective review of SDV queries generated and determined SDV queries have a negligible effect on critical data points. (Transcelerate BioPharma, 2013)² It was clear that more effective methods of quality and monitoring oversight of clinical studies were needed.

With the initial release of the 2011 European Medicines Agency (EMA) reflection paper and the 2011 U. S. Food and Drug Administration (FDA) draft guidance document promoting a risk-based approach to study monitoring or management, early program adopters applied quality initiatives focused on identification, assessment and mitigation of trial-related risks.^3,4 It was quickly discovered, however, that before a risk-based approach was effectively implemented, a quality risk management system was needed for the model to work.^5,6

Quality risk management: the framework supporting RBM

The framework of a risk-based model is embedded in process improvement and efficiency methodologies described in International Conference on Harmonization (ICH) Quality Risk Management (Q9).⁷ Quality risk management is a pragmatic method that outlines a systematic approach for the assessment, communication, control and review of risks related to the quality of a drug product across its life cycle. For medical devices, a range of separate and additional standards exist to drive quality risk management, including ISO-14971 and ISO 14155:2011.^8,9 While these tools detail leading-practice requirements, they do not address risks associated with clinical decision-making nor define acceptable levels of trial conduct risk.⁵ That responsibility falls on both the investigator/site and sponsor.

Building the structure for RBM implementation

Constructing a sustainable risk management system relies on correctly diagnosing and designing a model that works for an organization’s risk-based approach to risk identification and mitigation. It defines the process of quality control-directed monitoring activities and recommends periodic review necessary to assure sustained improvement and control necessary to sustain the Quality by Design (QbD)/QMS model. In other words, QbD and QMS systems require continuous feedback to identify risks, initiate corrective actions and improve overall health of the system. To implement a quality risk management (QRM) program for a trial, sponsors must form a cross-functional team to drive the process. The team must have a basic understanding of the fundamentals of quality management principles and risk management in context with clinical trial conduct, as well as understand and define key risk indicators (KRI) and essential clinical trial key performance indicators (KPI). The team should operate under basic project management principles and be guided by a project charter and clearly defined risk management framework (see Figure 2).  Additionally, the team should identify a project or study to pilot¹⁰ and conduct a series of workshops using Failure Modes and Effects Analysis (FMEA) or PMI PMBOK Risk Methodology to identify the likelihood of occurrence, frequency, risk score and mitigation plan.

The goal of this approach is designed to decrease the overall burden of redundant activities while promoting a quality approach to clinical study conduct, enabling high-quality data delivery, increasing patient safety and effectively managing monitoring resources. Sponsors are testing various adaptive models in pursuit of a RBM model that will produce clear, consistent evidence of effectiveness. A leading practice approach combines 100% on-site source document verification of a limited set of critical data elements and the use of statistical sampling plans with remote or central monitoring review that follows the Plan-Do-Check-ACT (PDCA) methodology. The PDCA approach is an effective tool that can be used to help cross-functional teams demonstrate a commitment to quality, identify potential risks, analyze problems and implement solutions across the life cycle of a trial.^11,12 Using the PDCA tool, the team follows a four-part approach.

Plan

The risk assessment (RA)

Before beginning the actual exercise of risk identification, it is necessary to assemble a core team responsible for clinical operations, monitoring, data management, safety, biostatistics and, where applicable, product development. Study team representatives are essential to risk identification, analysis, mapping, assigning a weighted value and establishing mitigation strategies. Risk in clinical trial conduct arises from a number of areas, and the Risk Management Team must evaluate each KRI and KPI thoroughly and critically. Addressing risk potential requires the sponsor to consider root cause(s) and consequences posed by risk, as well as the likelihood the consequences will occur.

The seven common areas of risk arise from:

• Investigational product

• Protocol and trial-related procedures

• Study population or disease state

• Experience and performance of clinical investigator, study staff and site

• Regulatory status of the intellectual property (IP) filing

• Country-specific compliance requirements and infrastructure support

• Adequacy of planning for risk factors prior to protocol finalization and completion of the Clinical Investigational Plan by the sponsor

Mitigation of residual risks

Once the key risks metrics have been identified and mapped in the RA phase, the team defines, analyzes impact and establishes potential risk-mitigation strategies. Creating a mitigation plan requires the team to:

• Conduct a literature search and empirical analytical assessments for like or similar studies conducted on the same disease state/study population to identify similarities and likelihood of occurrence and uncover statistical trends

• Review risks associated with the investigational product during the development phase

• Identify the impact, likelihood and severity of the events associated with trial protocol and procedures, site management activities and performance indicators (generally ranging from very low to very high)

• Evaluate the financial, strategic, compliance and operational impact of events on study conduct

• Establish the criteria and effectiveness of interventional procedures and activities that will be used to define and manage mitigation strategies

Although each identified factor and corresponding consequence is important, the team should focus on risks having the greatest impact on data integrity and subject safety. Using the FMEA method and tool created during the RA enables the team to effectively manage mitigation strategies and track the impact of identified risks that affect patient safety and data integrity.  

Once risk identification and potential mitigations are determined, each risk must be further analyzed with the objective of gaining a deeper understanding of the risks, impact, probability of occurrence, weighted value, residual risk acceptability, and the effectiveness of the proposed mitigation response once a risk-benefit analysis is completed.¹³ The assessment should take into consideration the overall benefits and risks of the study related to investigational product, third-party vendors, disease state, study population, inclusion/exclusion criteria, study visits and procedures, as well as study endpoints and safety observations that must be critically evaluated for evidence supporting the study hypotheses. Upon completion of the aforementioned steps and the risk-benefit analysis in context with study risk, the protocol is finalized.  Other relevant functional study plans, such as monitoring, data management, communication, statistical risk analysis, escalation and safety should be drafted and approved in tandem with protocol finalization and approval.   

Individual study plans for monitoring, data management, communication, statistical risk analysis, escalation and safety must clearly outline timelines, requirements and response to risks identified in the benefit-risk determination document. The content should provide necessary clarity for staff involved in study execution to carry out roles and responsibilities for implementation. One such tool to assist in the identification of roles and responsibilities is a Responsible/Accountable/Consulted and Informed (RACI) chart.  The RACI exercise is an effective tool for providing clarity to individuals involved in study conduct, as well as providing an additional layer of compliance oversight. 

Do

The content built and decided upon in the “Plan” phase should provide the necessary clarity for staff involved in study execution to carry out roles and responsibilities for implementation. In other words, they should execute the “Plan” in the “Do” phase.  For example, the clinical research associate (CRA) begins monitoring according to the approved monitoring plan and completes both remote and on-site monitoring.  In the next step, the CRA reports applicable data via a trip report within a clinical trials management system (CTMS).  Management then receives trending dashboard reports with applicable KRIs and KPIs based on the trip reports. 

Implementation of the applicable plans such as the communication, risk mitigation and escalation plans should begin as the clinical trial planning begins.  Once the “Do” phase has begun (all PDCA phases may run in parallel with each other), data should be collected, analyzed in and mitigated as a result of identified issues.  The very nature of a QMS is to be adapt and improve overall clinical trial performance.  In other words, the clinical trial naturally adapts and changes overtime.  The Risk Management and study team should work together as the clinical trial will require iterative changes based on the PDCA information being received and analyzed. 

Check

Monitoring resource allocation and activities will be monitored through metric trend analysis (completed in the statistical risk analysis) and early performance variance by investigator and study team, and by the frequency and severity of KRIs and KPIs. For example, trending dashboard report analysis identifies a frequently occurring KRI of a pre-identified protocol deviation (compromising data integrity).  Early detection of the protocol deviation allows the CRA to report or notify the study team.  The study team is enabled to “Act” by beginning an investigation to determine the cause of the protocol deviations in real-time. 

In order for RBM to be successful, the roles and responsibilities of centralized¹⁴ versus on-site monitors must be clearly defined, along with the process of communication between the CRAs and the study team (see RACI and communication plan above). The combination of on-site, remote, centralized and/or triggered monitoring, while reducing costs, allows management and monitors to use dashboard KPIs and KRIs to drive the level of monitoring intensity. Ongoing trend analysis and metric performance measurements allow the Risk Management and Study team to compare variances against risk thresholds, identify outliers and initiate escalation response to mitigate risk.  By using metric measurements and tracking trends, the team can “define” the needs of the clinical trial while concurrently capturing and documenting the effectiveness of corrective actions. The effectiveness of mitigation strategies is enhanced through collaboration and communication between CRAs and the study team. A gap in risk mitigation can be avoided by creating a risk communication, escalation and response process.

As industry moves toward a risk-based model, the CRA skill set takes on increasing importance.  For example, the centralized CRA should have working knowledge of QRM, QMS, operational strategies and qualitative/quantitative analytics such as analysis of variance (ANOVA)¹⁵ to conduct data sorting and data trending for the entire study.  By contrast, the remote monitor should conduct site SDV remotely (several sites or one), communicate with the site and study team and report findings. The on-site CRA role should manage the day to day activities of study and site management. Clearly defining the roles and responsibilities in the RBM environment enables the study team to align CRA skills to each of the remote, centralized, or on-site monitoring roles.

Act

Risk mitigation strategies are a necessary part of the RBM approach and will guide the next steps to be taken in corrective actions, as well as the improvement process. Following the PDCA cycle presented earlier, the next step is to “act” upon findings, assess the effectiveness of the intervention and/or corrective actions, and continue to monitor progress across the life cycle of the study. Throughout the study, as data is gathered and analyzed, and root causes are determined, the process will undergo improvements and controls simultaneously to correct the process.

Embracing cultural change

Successful delivery and sustainability of the risk-based monitoring (RBM) delivery is rooted in the cultural transformation across sponsor, CRO and sites. This transformation begins with behavioral change supported through stakeholder buy-in, communication and training. The most common factor in process failures stems from people. Transforming the RBM culture and managing the organizational change requires understanding and addressing the human element. Organizational change helps integrate people’s roles and responsibilities into the new process. Engaging key stakeholders in the process of change and providing them with the appropriate tools enables successful implementation. This approach results in increased adoption, accelerated results and sustained benefits of the change.

With sponsorship and commitment of sponsors, CROs and sites, RBM program leaders align common goals and drive toward joint success. Establishing a cross-functional leadership team that is engaged in the change process with clearly defined roles and responsibilities will enable a successful transformation and adoption of the RBM model. Mapping issues to roles and responsibilities further enables effective solutions. Regular engagement of key stakeholders identifies where and when to involve people in the change process.  

While the goal of RBM may appear consistent for each stakeholder, the motivators differ across sponsors, CROs and sites. Sponsors pursue increased public trust in the evidence produced to justify fewer post-market registries and drive continuous improvement. CROs seek the ability to monitor exactly what matters and to produce a scientific methodology to monitor and to efficiently allocate resources. Finally, sites are motivated by increased access to data, therapeutic innovations for patients and opportunities for publication.

A key component in managing transformative change is communication. A clear communication strategy provides the road map for messaging the change, evaluating the response and initiating modifications to help meet stakeholder objectives. Providing all stakeholders with consistent, timely and accurate information, aligned with the programmatic timeline, avoids confusion and promotes comprehension and effective collaboration. The roles and responsibilities of remote versus in-field monitors must be clearly defined, along with the process of communication between the monitors and study team. The combination of on-site, remote and/or triggered monitoring, while reducing costs, allows monitors to use the dashboard performance and risk indicators to drive and optimize the level of monitoring intensity.

Conclusion

Building a sustainable quality system for clinical trial conduct relies on correctly diagnosing potential shortcomings of existing processes and designing a model that works for your specific organization. Be sure to further define processes for quality control monitoring and perform periodic reviews to continually improve the future-state system. Under this model, monitors simply cannot execute RBM without vital input from other study team members. Therefore, team member roles and responsibilities in trial risk management, as well as culture and behavioral changes, must be identified and mapped to the new/future model. Once the mapping is complete, new and revised processes for proactively identifying and mitigating trial risk can be implemented. It is imperative to gain stakeholder approval and support before undertaking change and to enable continued support during the entire sustainability process. If stakeholder approval and collaboration is not gained or is lost during the sustainability phase of a risk-based approach, the program is at high risk for failure.

Quality control monitoring and periodic review are intended to sustain the QMS approach, provide continuous feedback to correct errors, improve the overall health of the trial and – most importantly – protect subject safety.

References

1. Koreith K. The high cost and questionable impact of 100% SDV. CenterWatch Monthly. 2011. 18 (1): 15-17
2. (Transcelerate BioPharma, 2013)
3. European Medicines Agency. Reflection paper on risk-based quality management in clinical trials. 2011 August 4:1-31. EMA/INS/GCP/394194/2011
4. U.S. Food and Drug Administration. Guidance for Industry: Oversight of Clinical Investigations – A Risk-Based Approach to Monitoring. 2013 August 13. Available at www.fda.gov/downloads/Drugs/.../Guidances/UCM269919.pdf
5. Oleson K and Fay MF. Application and Key Concepts for Quality Risk Management in Medical Device Clinical Trials: The Monitor. 2013;12: 19-24
6. Fay MF and Oleson K. A Rationale for a Risk-Based Approach to Clinical Trial Conduct and Monitoring Practices: The Monitor. 2013; 12: 9-14
7. U.S. Food and Drug Administration. 2006. International Conference on Harmonization: Guidance on Q9 Quality Risk Management-Availability. Federal Register. 71(106): 32105–6
8. International Standards Organization. 2007. ISO 14971 Medical Devices – Application of Risk Management to Medical Devices. Edition 2. Available at www.iso.org/obp/ui/#iso.std:iso:14971:ed-2:v1:en
9. International Organization for Standardization. 2011. ISO 14155 Clinical Investigation of Medical Devices for Human Subjects – Good Clinical Practice. Available at http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber=45557
10. (Alsumidaie, 2014)
11. Juran JM. Juran on Quality by Design: The New Steps for Planning Quality Into Goods and Services. The Juran Institute Inc. New York: The Free Press, A Division of Simon and Schuster Inc.; 1992
12. International Standard Organization. 2008. ISO 9001 Quality Management Systems – Requirements. Available at http://www.iso.org/iso/catalogue_detail?csnumber=46486
13. Guidance for Industry and Food and Drug Administration Staff – Factors to Consider When Making Benefit-Risk Determinations in Medical Device Premarket Approvals and De Novo Classifications. Document issued on: March 28, 2012. Available at http://www.fda.gov/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm267829.htm
14. (Alsumidaie, The Emergence of the Centralized Monitor, 2013)
15. (Eberhart, Stevens, & Moran, 2013)