The Qs of Risk-Based Monitoring

SPOTLIGHT EVENTRisk-Based Monitoring – Beyond Theory In-Depth ReviewMarch 13, 2014
Cambridge, MassachusettsDownload BrochureRegister

RELATED
- The Emergence of the Centralized Monitor
- Risk-Based Approaches
- ICON Exec Discusses ICONIK MonitoringMore in Risk-Based Monitoring

Courtney McBean, president and co-founder of Blueprint Clinical, explained in Part 1 about what made her decide to start up a technology company that specifically addresses risk-based monitoring (RBM). In Part 2, she discusses the company’s offerings.

Blueprint Clinical, the company, is one part specialty monitoring consulting and one part web-based monitoring technology--Blueprint Compass--that blends both qualitative and quantitative analysis to get a “bigger picture of what’s happening at your sites within a given study—how they’re performing relative to one another, when there are issues that need action, and when there are trends that are forming and continuing,” explained McBean.

The analytical tool behind the technology has statistical input built into it, and that engine contains quantitative data that’s collected in the study. This data can come from the EDC system, CTMS, lab database…any existing study databases are linked to the engine. McBean says, “We’ve done countless research on what data does the FDA really care about, based on warning letters, based on all of the regulatory statements that have come out, and guidances. And then we’ve also looked at what metrics point to issues at a site. Those metrics, such as deviation rates or CRF completion time—what are those variables that indicate what’s really happening at the site, which is already being collected in the study. So we collect that on the quantitative side.” In addition to the testing of the Blueprint Compass algorithm and methodology based on the FDA regulatory research, when TransCelerate issued its methodology, McBean was pleased to note that it aligns very well with the Compass methodology.

This quantitative data is combined with the qualitative data, which has three pieces—protocol specific risk-assessment, site history experience with past performance, and monitor feedback.

The monitor qualitative feedback tool is the incorporation of on-site, or what McBean calls the “gut feeling” that monitors have about what is happening at a site. McBean says, “It may be how engaged the PI is, or that there’s inappropriate delegation. But in my experience in monitoring in the past, we have had plenty of fraud, none of it has been statistical fraud. It’s all been fraud that you would have had to catch by looking at forged informed consent signatures or various things that have happened at the site, that you may never catch by doing just statistical analysis alone. So while we feel that the statistical piece of it is very important, we also feel that you cannot miss that qualitative aspect of monitoring.”

To capture that elusive gut feeling, they developed a set of questions for the monitor about their experience the last time they were at the site. This monitoring feedback is added to the tool, as well as a short piece of site feedback, which captures their interpretation of how things are going, how well-controlled they feel the study is from their perspective, and how effective they feel they can be in carrying out the responsibilities of the protocol.

The analytic tool takes both the quantitative and qualitative data and provides features such as real-time alerts and performance information, based on pre-determined key areas of variables that are chosen. In addition to the short-term or real-time alerts, the tool also features mid-term and long-term alerts. Mid-term alerts categorize site performance into five key categories that have been identified by the FDA as being very important--patient safety, data quality, regulatory compliance, protocol integrity, and site engagement. If performance declines in any of those five categories, the tool provides action recommendations to help mitigate that from becoming a greater performance issue. These actions can also be customizable to a company’s process or protocol.

The long-term alert, is actually Blueprint Quality Score, which is an overall picture of the five mid-term alert categories combined, to provide long-term picture of how this site is performing relative to the other sites in the study. According to McBean, it is this long-term picture that drives a company’s on-site monitoring frequency and scope. In this way, visit frequency and scope of data monitored can be flexible to increase or decrease based on improvement or decline in the quality score.

The consulting side of Blueprint Clinical helps clients integrate if they are in a team restructure, SOP development around monitoring, individual monitoring strategies for studies, as well as helping integrate and get organizations ready, if they choose to go to develop a risk-based monitoring strategy. “We help them through that stakeholder buy-in, getting everybody understand what risk-based means and how it’s going to be implemented it at their company, so it’s very specific,” explained McBean. “Blueprint Compass is there for people to use independently of consulting or alongside it to help them plot the strategy.”

What McBean and her colleagues at Blueprint are hearing from sponsors about the risk piece is that they know they are supposed to assess it, but they don’t know how. Or if they do, how do they know if the strategy is working. McBean believes that the use of short-, mid- and long-term alerts in Blueprint Compass works as an insurance policy in that at any given time, the tool is constantly reviewing whether there is an issue indicated at a site. “So you may not monitor as frequently as you used to, but there is a tool doing that for you. And it will indicate if there’s a reason to go there more frequently than you originally intended to,” explained McBean.