Can RBM Influence the Data Quality and Patient Safety Inversely?

“Never assume that something obvious is true,” a good friend of mine told me-a researcher with 20 years of experience, and one of the wisest men I have met in my life. Many years later, this advice would guide me to critically consider and analyze Risk-based Monitoring (RbM).

The FDA states clearly, “Monitoring should be tailored to your organization, the study protocol, and the product being tested.” This assumes at least, that the selection of methods and combinations of monitoring tools involves thorough analysis of a trial and its protocol, and of potential external and internal risks. Only after capturing and understanding sufficient information, should one proceed with defining the RbM strategy (Alsumidaie, M. et al., 2015).

Can RbM influence the data quality and patient safety inversely? Strange question, isn’t it? Everybody is so enthusiastic about proving that RbM works and provides data quality improvements and cost savings that almost no one said that it could make things worse.

RbM is considered a new concept (although with old principles), and many researchers are carried away with this innovation-although emotions are not always good for science, which human history proved many times. The stated question is justified; RbM is an optimization of the trial’s monitoring. From this perspective, any skipping certain steps can influence the process in both directions.

The advantages of RbM such as cost reduction and/or data quality improvement has already been claimed in certain case studies (Cramer, 2015). Nevertheless, the author considers them strongly influenced by euphoria, willingness to see certain results, without seeing them in reality. Risk management and RbM incorporate certain risks as well, which can certainly influence data quality and patient safety inversely.

This aim of the article is to summarize the thoughts of industry experts and emphasize to readers the most dangerous pitfalls that the pharma has to deal with nowadays. You can avoid them by implementing an RbM process in your organization.

Pitfall #1. SDV is reduced without any other action

Adam Baumgart, Director of Process Excellence at a CRO, sees it as a major risk: “SDV is reduced in isolation with no other action.” Under an RbM flag no other actions except the Source Data Verification (SDV) reduction is taken-a very dangerous situation-for the team and for the RbM itself. Be cautious because RbM does not mean to do less, it actually means to do smart monitoring. As Sandeep Mitra (Quality Management at Quintiles) says, “RbM systems should look into not making a monitoring visit redundant, but optimize the time spend by a site monitor on-site.” How we can achieve that? Applying the risk profiles during the monitoring visits, building the expectation about percent findings and testing these assumptions.

How to avoid the pitfall? An SDV reduction must be the last action of the change management process in RbM integration. Be sure that you have applied the change management process and the whole team understands and supports the RbM process. Only then, start reducing SDV, monitor the risk profiles of each site and write down clearly, what findings are you expecting with certain risk profile.

Pitfall #2. Risk evaluation is not objective

This pitfall may happen if human factors are involved in risk evaluation. A typical example is a well-known spreadsheet with the usual review of risks during a clinical trial. The consequence of a subjective or biased analysis is inappropriate corrective actions. In the best-case scenario, these actions will not cause harm, but in the worst case scenario, they can even become counterproductive and lead to mismanagement.

Moreover, as Steve Young former co-lead of RbM strategy at Medidata says, and the author agrees, a “Key Risk Indicator (KRI) is effective only if it is reliable and pro-active. Too many organizations are deploying KRIs for their study teams where these two requirements have not been properly considered.”

Reliable: it detects emerging risks accurately and minimizes the occurrence of false positives (i.e., yellow or red alerts that turn out to be non-issues).

Pro-active: a KRI detects emerging risks at a very early stage during a trial (so called “leading indicator”). 

How to avoid the pitfall? The author advises to base the risk evaluation on the most objective and extensive sources, e.g., on clinical data from recording systems like EDC, CTMS, pharmacovigilance, IVRS, lab data, etc. During the validation of risk indicators, evaluate such properties as “reliability” and “proactivity”. Document: How do you measure them? What assumptions you based your validation on?

Pitfall #3. Risk evaluation is biased

Bias appears when risk evaluation and quality assessment is conducted by an involved party, as for example, an EDC provider, CTMS provider. Here appears another jeopardy-“agency problem”. Because of this problem, none of the banks are allowed to conduct economic activity together with crediting. The risk ranking agencies such as Moody’s, Standard and Poor’s or Fitch are not allowed to, either. The company that captures the data is usually biased and is reluctant to identify or transparently communicate its own problems. Reputation damage is too high burden to remain unbiased.

How to avoid the pitfall? When looking for an RbM service provider, select an independent risk analysis and data quality monitoring company (e.g., Cyntegrity, JReview, etc.). Follow RbM evaluation checklists, which can help you to consider all aspects of the RbM offer. Ask a service provider how they guarantee that the “agency problem” will not happen.

Pitfall #4. Risk evaluation becomes outdated

The risk landscape changes during a trial. In other words, a site gets a new primary investigator who is not experienced with the protocol and the procedures, and immediately the risk profile of a site changes. If evaluation of a risk happens half a year later, almost nothing can be done to correct the situation. Risks are like water, sometimes calm, sometimes rough, but they flow, develop and change. Many external and internal factors influence them. The consequences (and obviously importance) of certain risks changes for a sponsor as well. Some risks can escalate or de-escalate during a trial. Therefore, continuality is essential in the risk assessment.

How to avoid the pitfall? Make risk evaluation on regular basis, e.g., bi-monthly, monthly or even weekly. Automatic solutions can help with that. Moreover, do not forget to reflect on the regular review of risk landscape in corporate Standard Operation Procedures (SOPs).

Pitfall #5. The late arrival of data

Risk evaluation is objective and unbiased, although the data, which is essential for such analysis, arrives too late-no corrective action can help to mitigate the risks. The most dangerous situation can be with patient safety. Imagine an ECG shows a pre-infarct situation and a patient can be saved this time; however, the data is sent from the site two weeks later a Serious Adverse Event (SAE) or even Suspected Unexpected Serious Adverse Reaction (SUSAR) has already happened. Here it is important to remember that the late delivery of clinical data should not be associated with the bad quality of data. Still, the faster the data achieves a central storage, the faster the reaction to an event and harmful consequences can be minimized.

How to avoid the pitfall? It is absolutely essential to motivate sites to send data as quickly as possible. The best practice should be to aim for a five to nine day turn-around. The delay in sending data can be a risk indicator in itself, which is an indicator of the site’s performance and data entry compliance.

Pitfall #6. Sites ignore RbM and do not improve

Today, sites are hardly involved into the RbM initiative. Some sites even consider that the abbreviation "RbM" means Remote Based Monitoring, just because the only effect they observe is that monitors visit them less and less. The sites are hardly involved in the procedure. No one speaks to them and no one asks their opinion.

This is one of the most dangerous practices one can do. First, any monitoring procedure must be totally transparent; only transparency drives self-improvement and self-reflection activities. The majority of the sites would like to know their position in comparison to other sites, and would be interested in monitoring their improvement.

How to avoid the pitfall? There are three points on the list: involvement, involvement, involvement. Clinical sites are the major stakeholder in monitoring and should not be ignored in this procedure. Deliver an automatic, regular, quality report about their performance and data quality (see the process scheme on Figure 1), compare the metrics between time periods and other sites, and you will observe how the system self-improves even without participation of responsible CRAs.

Figure 1. Site involvement scheme

Pitfall #7. Monitoring team does not accept the new procedure

The problem of resistance against of innovation influences not only sites, but also the monitoring team even more. Why does this happen? Everybody has his/her own reasons-mostly it is fear that one cannot fulfill his/her job, the lack of information and lack of involvement. The result is any new initiative (not only RbM) stops working and sinks in mutual resistance (P. Kotter and a. Schlesinger, 1979)

How to avoid the pitfall?

·      Education & communication: Communicate the desired switch to RbM. Once persuaded, people are often likely to implement the change with encouragement.

·      Participation & involvement: people feel more committed to making the change happen.

·      Facilitation & support: no other approach works as well with adjustment problems.

·      Negotiation & agreement: a relatively easy way to defuse major resistance.

·      Coercion: works quickly but can spark intense resentment (saves time in the short run; one should still consider the first options).

Pitfall #8. The RbM IT system produces too many messages for monitoring team

Finally yet importantly, if your KRIs are producing too many alerts, no one will take them seriously. This will send monitors and study teams on a wild-goose chase that will be very resource-inefficient. It is even dangerous, because if an important message gets lost in thousands of unimportant ones, nobody will react to it.

How to avoid the pitfall? Apply certain prioritization and escalation logic for the notifications and let them be generated only for group of events, not for each event (see the escalation scheme Figure 2).

Figure 2. RbM escalation levels.

Key take-away message

Raymond Touomou, a senior CRA at Bristol-Myers Squibb, reviewing KRIs daily, often asks the questions, “Are we sure we are looking at all relevant risks within a trial?” “Are there other risks that could be missed?” Unfortunately, RbM cannot answer these questions. As Lorne Cheeseman, CEO, Kestrel Biologic Inc., once said,  “We all have to be very careful about how we approach RbM, and in particular, the idea that a trial management team can reduce monitoring to some formula, configure it in an IT system and reduce SDV. In the end, a trial demands well-trained monitors who are able to integrate with a wide variety of information and make informed decisions about what is going on at a site using the risk profile information. So many experts in the industry do worry that if we rely too much on some sort of formula for all the reasons listed above, there can be adverse effects on the quality and patient safety.”

Even if something appears as obvious, it is not true yet. RbM makes monitoring routine more complicated and introduces its own risks. It is not a formula to configure and forget, it is smarter attitude to old tasks applying the modern technology and keeping eye on those risks, which are unknown yet.

Author:

Artem Andrianov, PhD MBA

Managing Director

Cyntegrity Germany GmbH

References:

Alsumidaie, M., Proupín-Pérez, M., Andrianov, A., Widler, B., Schiemann, P., Schenk, J., n.d., (2015). RbM Guidance Document: Ten Burning Questions about Risk-Based Study Management [WWW Document]. URL http://www.appliedclinicaltrialsonline.com/rbm-guidance-document-ten-burning-questions-about-risk-based-study-management (accessed 1.26.15).

Cramer, P., (2015). Improving Data Quality for Risk-Based Monitoring and Data Review | Journal for Clinical Studies 6,  no. 6., 50–51.

FDA. (2013) Guidance for Industry Oversight of Clinical Investigations: A Risk-Based Approach to  Monitoring. [Online]. Available from: http://www.fda.gov/downloads/Drugs/.../Guidances/UCM269919.pdf (accessed 8.15.14).

Schlesinger, Leonard A., and John P. Kotter. (March–April 1979) Choosing Strategies for Change. Harvard Business Review 57, no. 2.