Signal Prioritization

In last month’s blog post we discussed the necessity and ability of systems to identify potential safety signals in our clinical, post-marketing, and surveillance data. This act of identifying signals in different sources² is only the beginning. This month’s blog post will look at the various methods of signal prioritization that exist to assist us in the often daunting task of sifting through the identified signals so that we ensure proper distribution of resources.

The prioritization of signals is still a highly controversial aspect of signal management.⁵ It is the opinion of this author that the process should be well documented and transparent to protect the organization from audit risk and to ensure the protection of the greater public health. There are a few examples of the type of decision support that is used to determine the priority of signals. The WHO uses a method which analyzes seriousness, unexpectedness, strength of disproportionality score, the temporal displacement of disproportionality score, age of the product, signaling in multiple countries, positive re-challenge, or existence of the signal term on a list of targeted terms.² These aspects of the signal are each evaluated and then an overall priority is placed on the signal that drives the formal evaluation of the finding (a methodology we will cover in next month’s Signal Evaluation blog entry).

Another known formal method of signal prioritization is from the United Kingdom’s regulatory authority the MHRA. They have borrowed the triage concepts from emergency rooms for their signal prioritization method. Essentially they analyze the immediate and high risk signal attributes first and allow those aspects to drive formal evaluation while lower signal attributes are postponed for formal evaluation until a later time.⁵

Their methodology results in the formation of a final priority score built of two main parts; Evidence score and Public Health score. The evidence score is an algorithmic score based largely on the strength of the disproportionality metric value, plausibility of causal relationship and the strength of the evidence provided.⁵ This is a different focus than the public health score which is compromised of the number of cases per year, anticipated health consequences, and the reporting rate in relation to the level of drug exposure.⁵ The combination of this score is used in an undisclosed manner to categorize the signal into a final category of High, More information required, Low, or No action. Other companies have introduced software systems known as decision support systems into their organization to aid in the automated prioritization of signals. Researchers at Johnson & Johnson Pharmaceutical Research & Development, L.L.C. have introduced a systematic prioritization algorithm built into their signal management. In their methodology, multi-criteria decision analysis (MCDA) is incorporated into their signal detection tool so that a seamless evaluation of each signal occurs sending only those of highest priority into the formal evaluation process while others are allowed to remain in a holding pattern for further information.³

Regardless of the specific methodology selected, these works demonstrate the need of a formal, transparent, and reproducible signal prioritization aspect to our signal management procedures. In closing the ability for our field to make sense out to the vast amount of collected information continues to be a high priority. These systems and examples can aid in our continued protection of the public health and our collective companies.

Note: This post was edited by Gina Choi. By Rodney L. Lemery, MPH, PhD, Vice President, Safety and Pharmacovigilance, BioPharm Systems, Inc.

References

1. Levitan, B., Yee, C. L., Russo,L., Bayney, R., Thomas, A. P. and Klincewicz, S. L.. (2008). A Model for Decision Support in Signal Triage. Drug Safety. 31 (9), pp. 727-735
2. Council for International Organizations of Medical Sciences (CIOMS). (2010). Practical Aspects of Signal Detection in Pharmacovigilance. Report of CIOMS Working Group VIII, Geneva .
3. Levitan, B., Yee, C. L., Russo, L., Bayney, R., Thomas, A. P., and Klincewicz, S. L. (2008). A Model for Decision Support in Signal Triage. Drug Safety, 31 (9)
4. Lindquist, M. (2007). Use of triage strategies in the WHO signal-detection process. Drug Safety, 30:635-7
5. Waller, P. (2010). An Introduction to Pharmacovigilance. Wiley-Blackwell. Oxford, UK