A Roadmap for Implementing Risk-Based Monitoring and Quality Management

At the 2018 DIA Annual Meeting, our DIA Clinical Data Management (CDM)/Risk-Based Monitoring (RBM) Community met with many people who were implementing or preparing to implement RBM and risk-based quality management (RBQM) approaches. Our attendees were looking for support and a forum to address their concerns. They knew that there were new guidelines and that the industry was anxious to get started with RBM activities, but they also wished to address the pitfalls one might expect. To address this need for a discussion of the changes taking place in our industry and to support each other in this endeavor, our group presented a segment of our roadmap beginning with a look at pre-study risk planning, vendor selection, and change management in RBM and RBQM.

ICH E6 (R2)¹ has focused upon the implementation of RBQM activities in clinical trials. Multiple stakeholders were looking for real experiences that can be incorporated into their risk management strategies. The FDA’s comment period on the draft guidance “A Risk-Based Approach to Monitoring of Clinical Investigations”² reminded us of the need to focus on how we were implementing the RBM guidance. The comment period also offered us an opportunity to communicate our suggestions for how to interface with and incorporate the needs of our communities and stakeholders. Another starting point for our roadmap was the case study from Cyntegrity.³ To assist in this overall endeavor, we have been working on an RBM Roadmap produced by the CDM and RBM DIA Communities. We envision a set of best practices.  

Risk-based monitoring – pre-study risk planning

ICH GCP E6 (R2) requires clinical trial sponsor companies to incorporate RBM in the risk management plan. 

Risk planning should start with the development of the target product profile (TPP)⁴ and then continue into the development of an individual protocol for a study. Some aspects of the TPP, such as once-daily dosing or a positive safety profile, must be taken into account when developing the risk management plan for a study.

Risk identification

Identifying critical data and processes

The most impactful operational risks to a prospective clinical trial will most often be associated with the most important processes. Yet, it is not only the critical data and processes, as highlighted by ICH E6 R2, but also the systems, (devices, sensors or software apps) that are used in clinical trials that contribute to risk.⁵

Thus, to begin with, the study team identifies critical data, tools, and processes. One example of critical data is the survival data of patients that withdrew their informed consent or those that are lost-to-follow up in an outcome trial. These inputs may come from data management (DM), biostatistics, or clinical operations (CO). Examples of critical processes include the timely reporting of serious adverse events to the sponsor and the adequate management of those by the sponsor. This process depends on multiple stakeholders, including DM, pharmacovigilance/drug safety, regulatory affairs (RA), medical affairs (MA), and CO. In addition, it may be worthwhile to check inspection findings from the FDA or European Medicines Agency (EMA) to see which have been areas of concern based on historic data. Keeping a closer eye on these findings may be very helpful in the future. As we will discuss later, each study team member plays a unique and crucial role in the risk-identification process. Thus, while RBM focuses on risk-based monitoring, the integration of a systemic risk management strategy for identifying and managing risks-from study design to the submission of the clinical study report-has resulted in a more evolved descriptor called risk-based quality management (RBQM).

Identifying risks

Once the critical data, processes, and systems used in a trial have been identified for the study, a core study team representing cross-functional expertise should conduct an exercise of risk identification. This includes identifying all of the potential risks that may impact patient safety, data quality, and/or the operational success of the study. Identified risks should be ranked and prioritized based on three dimensions: impact, likelihood, and detectability (ILD). The product of ILD helps derive the risk score or the risk probability number (RPN). This will help the team to focus their risk mitigation efforts on the greatest risks to the study.

Risk mitigation

Once the operational risks have been identified and characterized, the team develops a risk mitigation action plan to address the highest-priority risks. As previously mentioned, a high rate of lost-to-follow-up or discontinued patients might be an identified risk for the study. A number of mitigation actions might be considered for this risk:

1.    Removing one or more ancillary procedures from the protocol requirements that are not critical to support study endpoints may reduce the relative burden on patients participating in the study.

2.    Incorporate additional awareness training for sites related to this risk, including the need to check proactively with patients on their burden/comfort level with continued study participation, and what can be done to improve their experience, etc.

3.    Implement one or more key risk indicator (KRI) metrics to identify sites that may be experiencing a high rate of lost-to-follow-ups, missed or missing visits, or withdrawn consents.

These mitigation planning items need to be documented carefully, either within the risk management plan (RMP), risk log, or integrated quality risk management plan (IQRMP), where accountabilities can be assigned. The individuals accountable for managing the various risks must be identified via the operational plans, such as the central monitoring plan, the CDM plan, the site monitoring plan, or the pharmacovigilance plan. Thus, the RMP serves as a master plan, feeding into the individual operational plans and is, hence, described as an “integrated” risk management plan.

Defining KRIs

Once the key risks have been specified and mitigation plans established, the complete set of KRIs to be used in support of risk detection should be confirmed. The list should ideally be comprised of therapeutic area-specific and study-specific KRIs identified during mitigation planning, along with a standard set of generic KRIs that the organization has identified to be used across studies.⁶The study team should identify about 10 to 25 KRIs for a study. When the team finds lost-to-follow-up patients and withdrawn informed consent, it should monitor the KRI for an increasing number of missing or missed visits at sites throughout the study. Applying relative weights to each of the KRIs may be helpful in supporting decisions on which KRIs to follow up on as they emerge during the study and the urgency with which outliers need to be addressed. Not every risk in a study can or should be mitigated. This exercise, though, must be run very carefully in order to not overlook critical data or processes. Changes in local regulations is illustrative of a risk that needs to be accepted. It is also important that teams are trained not only on defining the KRIs, but on interpreting them as well.⁷

Vendor selection for the implementation of RBM

Vendor selection when implementing RBM activities is not an easy task. It involves internal and external challenges.⁸

At the start, one needs to have clarity within the organization as to what are the driving factors for implementing RBM. Is this a strategic decision with clear objectives (i.e., decreased number of site visits and increased quality) and clear milestones (i.e., decreased number of site visits) that improve quality? Many companies have not bought into this model, but are merely testing the waters. Some are recognizing the significant weightage that the FDA, EMA, TransCelerate BioPharma, and the latest ICH E6R2 guidelines have given to RBM and are making tactical decisions to perform a pilot study. However, it is important to remember that the implementation of RBM involves a significant investment in technology-and the benefits of this approach may not be evident by performing one small RBM pilot study. 

Another internal challenge is whether to follow a one-stop-shop approach or to identify smaller, more cost-effective and amenable partners. There is often a lack of clarity as to whose SOPs should be followed and whose KRI libraries should be used-the vendors or the sponsors? Even more critical is the lack of pre-defined key performance indicators (KPIs).⁶These are essential to measure the success of the RBM implementation strategy and of the partnership as well. 

A survey conducted at the 2018 DIA meeting in Boston demonstrated that around two-thirds of the participants have never been involved in RBM vendor selection, did not have internal RBM expertise, and had not defined KPIs for RBM implementation within their organizations. This shows us the challenges that the industry is facing today.

Implementing RBM is not just about analytics, technology, clinical, or RBM domain expertise. It is really about all of them. Finding a vendor who really understands the science of RBM is important. It is crucial to work with a vendor who not only has a lot of experience, but is also willing to share its learnings with the sponsor company. A vendor (CRO) may have significant clinical trial experience but does not have a rich RBM implementation experience. This needs to be validated carefully. Alignment in terms of scale and geographical footprint is also important when establishing a preferred partnership. Does the vendor actually have RBM capabilities at the required locations?

Perhaps, even more important than all of the above is the cultural fit, the alignment in vision and the flexibility and transparency offered by the vendor.  This would go a long way in terms of transforming this from a sponsor-vendor relationship to a true partnership.

Principles of change management applied to RBQM

Why do we need to address change management as part of RBQM preparation?  What do we mean by change management? John Kotter, the guru of change management, refers to the eight stages that are required for the transformation and change to take place.⁹

• Establishing a sense of urgency

• Forming a powerful guiding coalition

• Creating a vision

• Communicating the vision

• Empowering others to act on the vision

• Planning for and creating short-term wins

• Consolidating improvements and producing still more changes

• Institutionalizing new approaches 

Change management, in the context of instituting RBM and RBQM practices, refers to more than just the relationship between the sponsors and the sites. In following the stages listed above and implementing a vision, one needs to integrate a change management strategy from the risk identification stage itself and through the entire risk management planning and vendor selection stages, as well as developing a cohesive RBQM plan, addressing the needs of each stakeholder.

Consider changes within the organizations and how they are affected by RBQM. Successful change management for the adoption of RBQM is much bigger than our relationship with the sites.  It speaks to changes in our relationship and understanding of all of the clinical trial processes and study data. It speaks to transformations in how we conduct clinical trials and how we, as a community, work together and support each other. The new RBM approach is not a competition between stakeholders, but rather a way to assist each other in minimizing risk and enhancing the quality of the output of all stakeholders.

Bottom line-change management and the adoption of RBQM means disruption. RBQM offers a disruption that will improve the organization.

How do we differentiate resistance versus misunderstanding? How do we address those who see the benefits but are lacking in a commitment to the rigors required of change? Do we have time to make all the changes that are necessary? How do we prepare everyone for the bumpy first pass?

As we have seen in implementing RBQM best practices, we are involved in actualizing all eight stages at the same time. We are often not in a position to build a powerful guiding coalition before implementing the changes. Yet, the implementation of all eight stages is required for success.  There is no option to not implement RBQM. How can we use Kotter’s stages to help us in our preparation and planning?

1-Establishing a sense of urgency

Pharma is experiencing a sense of urgency for adopting the principles of RBQM. For DM preparing internally for the adoption of RBQM, questions about systems, case report form (CRF) designs, and the timeliness of implementation lend all data activities a sense of urgency. Clinical operations teams express a sense of urgency regarding training and education on the new terms and processes. For regulatory, quality, and program managers, the sense of urgency is two-fold: first, assessing preparedness for RBQM, and, second, demonstrating preparedness. This is equally important for quality assurance (QA), as QA needs to ensure that all SOPs are in place, teams are trained on the same, and training records are in place as well.

2-Forming a powerful guiding coalition

It is not only the leader who is in charge of developing the KRIs, but rather a coalition or a group that can work together. Many leaders look forward to building a guiding coalition in implementing RBQM. Who has the right skills for taking the lead? Who is writing the SOP?  

Monitors often fear that they have become dispensable. They fear that communication with the site will be centralized and personal interactions will disappear. They also fear that data managers may not understand how much the sites might skew the data. For regulatory, quality, and program managers, a guiding coalition of leaders is essential. Their underlying fear is that they will not be seen as an important voice in the leadership program. The main fear is disruption and loss of quality due to the new process.

The vendor’s experience in change management is worth assessing. Both the sponsor and the vendor need to join hands in the implementation of a strategic change management approach within the sponsor’s organization, and attempt to address the concerns of naysayers.

3-Creating a vision

Creating a vision for the implementation of RBM and RBQM depends on a succinct determination of what are considered to be best practices within the organization. With many participants offering their visions, the essential elements can be obscured or may, at times, appear too discordant to be put into practice. As discussed earlier, it is a shared vision that brings the team together. 

For the data manager, a clear vision entails updating and executing new systems for RBQM. Who is doing the testing? When does one have time for testing? How are deviations from critical data points identified, interpreted, and actioned? For monitors, there may be conflicting visions. Some individuals may not see the changes as beneficial. Changes in visit schedules and data points to be reviewed can be disruptive. RBQM may feel too open-ended-how does one check for deviations and ensure quality at the site remotely? How does one retrain the site when difficulties are noted? 

RBM results in a disruptive change, not only in monitoring methodology but also in organizational structures and processes, which is not an easy thing to accept. It is important that the sponsor recognizes the commitment that these changes will require. How does its vision for RBQM answer the requirements of the regulatory authorities as well as address the institutional procedures? 

4-Communicating the vision

In communicating the vision, we need to show value. What is the value that RBM and RBQM will bring to our organization? It is important to let everyone know what is being planned and to address all concerns. Concerns vary in nature for different stakeholders. Data managers may have concerns regarding who is going to define the KRIs and how they will be collected in the study. Monitors may be concerned about what they should be communicating to the site and whether the site will buy into that message. RA may have its own perspective regarding whether RBQM will really add value, or will it just pose one more regulatory hurdle? Sites are often not even informed about the change in approach, despite being the most important stakeholders after the subjects. Technology teams may have concerns whether their platform solution will pass muster with the end users (the central monitors) and with auditors.

An organization must have internal RBM experts (or utilize external consultants) to be able to make meaningful, well-informed decisions. A lack of internal champions may often set this up for failure.

5-Empowering others to act on the vision

All stakeholders within the company and externally should be empowered to follow and contribute to the vision. As the key in the structure of implementing RBM and RBQM in the institution, central monitors are connected to all parties within and outside the company. A clear message on the implementation strategy and how structures are changing will empower everyone that is impacted by RBQM.  

The DIA survey revealed that 40% of the participants found the lack of internal champions to be the biggest challenge within their organization, followed by a lack of vision.

6-Planning for and creating short-term wins

It is good to demonstrate success early on to show where the organization is headed and what can be accomplished. Again, showing value is essential in creating short-term wins.

The RBQM approach is integrated right from the start of the study.^6,10Central monitors can show how well its worked by identifying outliers in critical data points, and alerting monitors to take corrective action. The monitors can demonstrate short-term wins by addressing these issues remotely and actually enhance the quality of the data being received from the site. Sites that are doing well, despite fewer visits, should be recognized for their achievements. For sites that need more frequent visits, the win can be to demonstrate early identification of issues and ease of follow-up. 

7-Consolidating improvements and producing still more changes

Once one has a short-term win, it needs to keep the momentum in place for the next set of changes. This requires preparation and planning. The first short-term wins are great, but consolidating these improvements requires vigilance. One may not have all the answers to the problems and challenges along the way. For example, a KRI that is cumulative may not be sensitive enough to show us how the site is improving or worsening, but it gives us an opportunity to consolidate our improvements and produce more changes.

RBM results in a disruptive change, not only in monitoring methodology, but also in organizational structures and processes-and change is not an easy thing to accept.  It is important that the sponsor company is aware of the commitment that these changes will require.

8-Institutionalizing new approaches

The final reminder in any change management plan is that change takes time. If one rushes through the process without validating that everything is done, it is likely that the program, in turn, will suffer. We have often seen that starting and then stopping the implementation and going back to the old ways can be more harmful than waiting until everyone is prepared to get started.  

Although companies that transition an ongoing study to the RBM approach may find it more challenging than starting with an RBM approach from the beginning, it is not impossible. But it crucial to get started with RBM the sooner the better.

Institutionalizing these new approaches always takes time. The secret to success for implementing change management includes transparency and the consistent ongoing communication of a shared vision.

Mary A. Banach, Project Manager, CTSpedia.org, Vanderbilt University, School of Medicine, Dept. of Biostatistics; Nimita Limaye, Practice Lead, Life Sciences, Applied Technology Solutions, Inc.; Steve Young, Chief Operations Officer, CluePoints; Teresa Ancukiewicz, Sr. Manager, Boston Scientific Corporation;Stephen E. Wilson, Consultant-Retired FDA Director, Biostatistics; Johann Proeve, Chief Scientific Officer, Cyntegrity GmbH

References

1. ICH H. (2016). Integrated addendum to ICH E6 (R1): guideline for good clinical practice E6 (R2). Retrieved from https://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6/E6_R2__Step_4_2016_1109.pdf.

2. FDA. (2019). A Risk-Based Approach to Monitoring of Clinical Investigations. Retrieved from www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM633316.pdf.

3. Alsumidaie M, Andrianov A. Case Study: Using RBM to Evaluate Site Engagement. Applied Clinical Trials.2015, http://www.appliedclinicaltrialsonline.com/case-study-using-rbm-evaluate-and-predict-site-engagement

4. FDA. (2007). Guidance for Industry and Review Staff Target Product Profile-A Strategic Development Process Tool. Retrieved from www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm080593.pdf.

5. Telfer G. Risk-Based Monitoring Insights from the Industry: Steve Young. 2016, https://cluepoints.com/risk-based-monitoring-insights-from-the-industry-steve-young-2/

6. Limaye N, Jaguste V. Risk-Based Monitoring (RBM) Implementation: Challenges and Potential Solutions. Therapeutic Innovation & Regulatory Science. 2019;53:183-9, https://journals.sagepub.com/doi/abs/10.1177/2168479018769284

7. Limaye N. The Three Biggest Challenges in RBM Related to Key Risk Indicators. 2018, https://cyntegrity.com/the-three-biggest-challenges-in-rbm-related-to-key-risk-indicators/

8. Limaye N. Twenty Five Key Questions associated with Vendor Selection for Risk Based Monitoring. 2017, http://nymro.com/twenty-five-key-questions-associated-vendor-selection-risk-based-monitoring/

9. Kotter J. Leading Change: Why Transformation Efforts Fail. Harvard Business Review. 1995, http://www.lighthouseconsultants.co.uk/wp-content/uploads/2010/08/Kotter-Leading-Change-Why-transformation-efforts-fail.pdf

10. Alsumidaie M, Proupín-Pérez M, Andrianov A, Widler B, Johanna Schenk, MD, Schiemann P. Ten Burning Questions about Risk-Based Study Management.Applied Clinical Trials. 2015, http://cyntegrity.com/rbm-consortium-10-burning-questions-risk-based-study-management/