DIA 2024: Pat Hughes of CluePoints Highlights Renewed Agreement With FDA, Utilizing AI in RBQM

ACT: What are you looking forward to working on and achieving with FDA under the renewed CRADA agreement?

Hughes: We are so delighted to renew the agreement with the FDA; our cooperative research and development agreement in its original form spanned back to 2016 and at the time, we were looking at how the statistical algorithms within the CluePoints solution could be used by the FDA within their submission process, such as selecting sites for inspection based on interrogating the data through CluePoints. And it was a really great opportunity to work with statisticians within FDA, and actually we've improved some of the ways that we do things in collaboration with their statisticians. That was the whole point of the research and development agreement, that we would work together on improving things for the FDA, but subsequently, how we could improve things within CluePoints as well.

For example, we came up with a new scoring algorithm that the FDA wanted to implement that would help them in their process, and consequently, we were able to use that within our own processes, too, if we wanted to. So, it's been a great relationship. We meet regularly with FDA. They are a big advocate of what we are doing, and as much as they're a big advocate of risk-based approaches, and that is the kind of work that CluePoints does.

With the new agreement, we're now focusing on how AI and machine learning can also assist in these types of processes. So, there's no set agenda for how that cooperation will work. FDA will still be able to leverage all the things that we currently do within the CluePoints platform, but together we'll then explore how the use of AI and machine learning is going to supercharge some of those processes. So, the thing we're looking forward to most is the collaboration moves on; we're able to examine things that are groundbreaking for the industry together, and inevitably get the buy in from the agency, with respect to all the kinds of risk-based approaches that are being advocated in the industry right now. They're a great partner. They're wonderful to work with. It's a collaboration that makes sense for us and makes sense for them, so we're really looking forward to it.

ACT: How has ICH E6 (R3) changed the risk-monitoring landscape? What can industry look forward to?

Hughes: It's never one thing that changes the industry, right? And the risk-based approaches have been advocated as part of guidance documents since 2011. FDA, in fact, were the first of the regulators to come out with a risk-based approach that was part of how they were recommending biopharma sponsors go about managing studies. So, it's not new, but with every iteration of this sort of guidance and ICH E6 (R3) is no different. There's a continued emphasis on what are the expectations of companies within the industry. And it's not often that a regulation is designed to improve efficiencies. Nobody looks at regulations and thinks, “Oh we can do things better, faster, cheaper, because we've got a new regulation.” Usually there's some hurdle to overcome, and it feels like more of a burden in a lot of cases. But in this case, very clearly, FDA and other regulators have realized that there is a much more efficient way of running clinical trials. You only have to look at other industry sectors like manufacturing industries. You don't have a production of widgets, and check every single widget that comes off the production line. You do random samples, you look at batches; you take a risk based approach. And certainly in our industry, we've been trying to look at every variable across the way, the data points that are being collected, and that's just not viable; if we're trying to do things more efficiently, that's just not viable anymore. So the regulations in R3 double down on that recommendation, that you take a risk-based approach and you concentrate on what matters most. So those are the kinds of things that we're looking forward to with the new regulations.

Aside from the regulations, there’s just so much more momentum in the industry. It takes a long time for these oil tankers to turn around, and we've worked in the industry for decades where things have been the same. It's very difficult to turn that large vehicle around when you're trying to become more nimble and agile. We've reached the point where pharmaceutical companies know that they need to do something different. This paradigm shift is now in motion, and there's definite momentum there, so the regulations help, but I think the industry in general recognizes that they need to do something different. Whenever we've spoken to pharmaceuticals companies since we first launched in 2012, everybody has always said, “This is fantastic, we need to do this.” But the next thing that comes out of their mouth is: “But how do we do it?” Because they know that they need to change to harness the power of these kinds of technologies, and it's that that the industry has a difficulty with, because change is momentous, because of the number of people involved, the silos that have existed in the past, and hopefully we're breaking down a lot of those now, but it's just a big change, and we're starting to see the industry moving completely towards this risk-based approach and not just dabbling in it, which is what I feel like the industry's been doing for, certainly the first 5 or 6 years, since the regulations were first announced. So, to answer your question, R3 is just a real reinforcement of the things that people are doing anyway, but it's helped, because now there's an expectation that there's a regulatory requirement to do them too.

ACT: Where do you see the greatest potential for the use of AI in RBQM?

Hughes: Not just in RBQM, but throughout any of the processes that exist. Certainly, the CluePoints approach is to look at use cases where it's clear that the manual intervention that's going on today can be vastly improved using these types of principles. So, what we're looking to do is to replace most of the heavy lifting that goes on in trying to find the needle in a haystack of what exists by a supercharged process to allow the machine learning, the deep learning algorithms, as we would call them, to do that for you; you need to concentrate on what matters most. If there's these types of approaches that can do the heavy lifting for you, then what we're doing is allowing you then to use your critical thinking to deal with whatever comes up. So in the case of RBQM, we've used a very advanced statistical approach, and we will continue to do that because it's proven and it's credible and it's working, but we're laying on AI and machine learning to help improve how the critical thinking can be applied. For example, when we run an analysis on a data set, we produce signals, these are potential issues. We're not saying they are definite issues, but we are saying you should definitely look at these things because they're outliers, they're discrepancies, they're anomalies that you should be dealing with, even if it's something that you can say, yes, no; when you recognize it's an anomaly, but we can explain it, and here's the explanation. When you produce those signals, there might be in some cases, and based on experience, this happens quite a lot. There might be signals that are indicative of the same problem, but they're all individual signals. What the machine learning algorithms will do is group those single signals into one signal that is the potential issue, because they were all reflective of the same potential problem. And so that's one element where that's a real time saver, because you don't need to click into every signal and say, “Well, that was the same as the last one. All right, same again.” So, there's a nice collecting and summarizing of where those potential issues might be.

And then the other thing that the deep learning, machine learning algorithms are starting to do is then potentially explain to you what might be going on and what we have seen before is the potential resolution. We're not a big fan of taking the human out of the loop. We always want there to be some expert using their critical thinking to decide what happens next, but there's no doubt that these types of algorithms—machine learning—could start to advise what might be the potential solution to the problem, because we've got a vast body of data now that we can draw from in terms of the training that would point them in the right direction. So those are a couple of examples of what's available today, but we're also looking at the things like intelligent query detection in EDC, which is an associated swim lane, in as much as it's still a risk-based approach, not quite RBQM, as you and I might know it today, but it's looking at that approach to say there's a lot of manual queries that fall out of the EDC system that need to be dealt with. We can run the algorithms on the manual queries and start to deal with those by taking those actions that are recommended, or saying, “Well, actually, we know that a lot of the manual queries don't result in any action. Perhaps the machine learning can just eradicate those at source, so we don't even get to the point where we review them.”

There's a number of intelligent aspects of machine learning that can be done in all areas of the clinical trial process, and we're trying to select the ones that just reduce the heavy lifting and roadmap items that might be taking a look at the protocol, so running the protocol through large language models, machine learning, and determining what risk might lie in the protocol and almost coming up with an automated risk assessment just by reviewing the protocol with algorithms. So, there's all sorts of exciting use cases, but what we're not fans of is just hypothesizing about what this might look like in the future. We're actually taking those particular use cases where we know there's a manually intensive, labor-intensive process right now, and taking away the labor intensive part of it and leaving you with the elements that you really want to focus on. And that's where we see the future of AI/machine learning for RBQM and other kinds of data interrogation aspects of clinical trials.