‘Hypothesis-Free’: Getting Proactive About Signal Detection

Elizabeth Smalley, director of product management, data, and analytics at ArisGlobal

Elizabeth Smalley, director of product management, data, and analytics at ArisGlobal, recently spoke with Pharmaceutical Executive, sister brand of Applied Clinical Trials, about her work at the software company in supporting the efforts oflife sciences clinical and pharmacovigilance teams in signal detection. New approaches in this area have become necessary with regulators requiring more comprehensive steps in assessing and monitoring the benefit-risk of medical products.

Pharmaceutical Executive: Can you describe your work with safety signal detection?

Elizabeth Smalley: Signal detection is all about finding patterns, correlations, and applying causality. It’s all data-driven.

Recently, we’ve been working on proactive signal detection (PSD). To understand how valuable that is, you have to understand how signal detection is done. The status quo today is very reactive. We’re trying to switch that to a proactive approach, while at the same time we’re trying to reduce false positives and detect those signals faster.

PE: Can you elaborate further on the concept of proactive signal detection?

Smalley: PSD is hypothesis-free signal detection. Often, we have a drug that’s gone through clinical trials, and we have a sense of what adverse events we might see. This is due not just to the trials, but because of the drug class and phenotypes that we’re working with. So we know what to look for; you develop a hypothesis and test it.

PSD is hypothesis-free because you don’t know what you don’t know. You might miss some of those unknowns if your work is hypothesis-driven. It also comes with what we call signal strength, which is the likelihood that correlations are causal. It’s a lot like a credit score, which is simply an objective measure of the likelihood that someone will repay a loan. It’s a similar idea, based on all of the things that we’ve already looked at, but it’s been made objectively.

PSD is a proprietary algorithm that we’ve filed a patent on that elicits those correlations unsupervised and provides a likelihood that there is causality associated to it.

PE: How will this impact the surveillance of adverse drug reactions?

Smalley: There are a few ways that this will impact things. For pharmacovigilance teams, we are already seeing that this method produces significantly fewer false positives. For those teams, that means more efficient operations, of course, but it also means spending more time on the signals that are really going to matter and end up impacting patient safety.

It impacts patients because we have our teams focused on the things that matter. We’re also finding those signals sooner, which means better patient safety.

It’s a boon for regulators as well. The primary remit is to make sure that the drugs on the market are safe and effective.

PE: Can you discuss some of the flaws with previous methods of signal detection?

Smalley: The bread and butter of signal detection today is looking at the spontaneous reported events. These are things coming from patients and doctors where they think the drug caused an adverse event. That is the primary data source, but as a data source, it is flawed.

It’s not that we shouldn’t use it, but as a single data source, it’s flawed. This is in part because things are underreported, with some estimates saying up to 95% of events aren’t reported. It’s going to take a long time for patterns to emerge when most people aren’t reporting these events.

That data source does not lend itself to detecting subtle increases in the likelihood of events. It’s not very sensitive. It’s highly lacking in details and it’s just a snapshot in time. For example, a patient is taking drug X, they start off having heart palpitations, but it’s not known if that patient had heart palpitations in the past. Have they taken other drugs? That information might be available, but it’s often not.

The methods of mining that data source are also limited. The bread and butter are methods of disproportionality. This is asking questions like whether or not there are heart attacks occurring with a drug more often than expected. There’s a slew of statistical methods to do that, but it’s not very precise.

It’s like trying to measure height with a meter stick. That method could say that a patient is somewhere between one- and two-meters tall, which isn’t precise.