SCOPE 2024: Steve Young of CluePoints Discusses RBQM and Key Risk Indicators

ACT: Where do you think the industry currently is with adopting RBQM? Is there a growing level of comfort with it?

Young: The short answer is, yes, there is a growing comfort level with it. Adoption is definitely picking up. In fact, I would say over the last two to three years, we're seeing an inflection point where it's really accelerating. I think part of that was spurred on by the unfortunate pandemic we had. But, it's not just that, I think with any new paradigm in clinical development, it takes a while to socialize it. It's a big industry wide change management exercise. I've been really passionate about risk based quality management for probably 15 years now, and I have to say 10 years ago, I naively thought, Oh, this is going to take off immediately within a couple of years, we're going to be there. But it didn't take off. I should have known better because we're in a conservative industry, but we really have an inflection point now. There's was recently a survey done and from various different measures, it has assessed we're a little over 50% adoption of what we consider end-to-end full RBQM methodology. There's still a ways to go, but I think everyone more or less understands now that there's no turning back. If you're not on board, yet, you're going to get on board in the next couple of years. So, it's getting pretty close to business as usual.

ACT: Can you tell us a bit about key risk indicators and how they can be effectively used in centralized monitoring?

Young: Key risk indicators are of course, a key part of central monitoring. For those that may not know what it is, they're metrics or parameters to basically monitor across sites in order to look out for known areas of risks that I'm concerned about. Whether it be adverse event reporting rates, or the rate of missed key missed assessments on your trial, or even things like the cycle time to enter patient data at the sites. Those are really kind of a standard part of any central monitoring and implementation. The key risk indicators, while they're important, shouldn't be the only thing you're doing as part of central monitoring. Statistical data monitoring, goes side by side with it and is more of an unsupervised approach that will expose systemic problems in the conduct of your study. We've done metrics that are showing about 40% of all issues found from central monitoring are being found from that unsupervised approach.

ACT: As AI continues to become a large part of clinical trials, how can natural language processing be used to improve central monitoring documentation?

Young: Central monitoring documentation in particular as a use case is one that we have developed NLP machine learning based solutions for. The NLP part reads all of the study team documentation that's been entered for each risk that's been detected and followed up on. All the risks are documented by the study team. The NLP part is reading through all of that and actually predicting whether all of everything that the study team is documenting represents a confirmed issue or a non issue.

A third option that it predicts is it's unclear from your documentation. So, it's interacting with the end users to gently remind them that their documentation doesn't give a clear indication. That machine learning solution was deployed over three years ago. Since then, what we found is that there has been a 25 to 30% improvement in clear documentation. That's just one example of how you can use NLP and machine learning to make a real difference. We have a growing number of machine learning and NLP use cases that we're applying to various parts of both risk based monitoring and other areas of clinical data management processes as well.