Observational Data Under the Microscope

In clinical development, the gold standard for establishing a therapy's safety and effectiveness is, of course, the controlled clinical trial. Yet we all know that the body of data accumulated during trials is not sufficient to adequately predict a product's future performance in the real world.

Wayne R. Kubick

Clinical trials generally involve only a very small, carefully selected patient population, which is hardly indicative of the much larger and diverse expanse of patients who may be exposed to a new product after launch.

Thus, the spontaneous reporting system was established to help fill the gap by monitoring potential drug-related events for marketed products. But despite its undeniable value for pharmacovigilance, it too suffers from such limitations as underreporting, bias, and uneven data quality. And the spontaneous reporting system can't really compare the number of reports received to the number of patients who have been exposed to a drug.

Taken together, these two data sources, while necessary and important, are hardly sufficient to fully understand the safety and effectiveness profile of a new therapy. That's why the promise of using longitudinal health care data for research is so appealing.

While not exactly a new idea—epidemiologists have been conducting formal, protocol-driven, noninterventional studies using observational health care data for quite some time—it's an idea that has begun to captivate a much broader community of researchers in government, academia, and industry.

Mounting attention

It seems like everyone is getting interested in using real-life observational patient data for multiple purposes, such as evaluating drug safety and effectiveness, and improving both the quality and economy of health care treatments.

We're seeing evidence of this mounting interest all around us:

• The FDA Amendments Act of 2007 included establishment of the Reagan Udall Foundation, which among other goals mandates a postmarket risk identification and analysis system to analyze safety data from at least 1 million patients by July 1, 2012.

• The FDA is initiating or participating in a number of Critical Path Initiatives such as the Sentinel Network and the eHealth Initiative.

• The public/private Observational Medical Outcomes Partnership (OMOP) is establishing a laboratory to explore the potential of using such data sources for drug safety evaluation purposes.

• The American Recovery and Reinvestment Act, among its many other features, includes funding to accelerate the adoption of electronic medical records (which will provide much of this observational data), and an intriguing investment for the "comparative effectiveness" study of health care treatments.

Data power

Why the recent spike in interest? A primary factor is that more electronic health data are now becoming accessible, and the increased traction of rapidly improving data standards promise to make data more suitable for different types of research and analysis than could be contemplated previously.

This means that more data will be coming at us from many different directions. But what types of things do people want to do with these data? Well, for starters, some typical objectives include:

• Providing early warnings and supporting rapid assessment of potential safety issues

• Exploring and verifying safety signals in real-world patient populations

• Better understanding of disease characteristics, disease progression, and treatment outcomes, which in turn may help guide the development of new or improved therapies

• General data mining for interesting new relationships (e.g., dependencies, interactions) that may be suggested by the data.

• However, there are many issues to confront first. Reliable de-identification is critical, yet by de-identifying data, it often becomes difficult to link together related records from different data sources—such as tying patient records with prescriptions and lab results.

And health care data are a valuable commodity, often expensive to acquire, from many different providers using many different proprietary formats. Such formats are not always suitable for analysis, with less than optimal contemporaneity.

Obstacles ahead

There are many challenges in translating streams of observational event data into structured, organized databases optimized for data mining and analysis, such as coding issues, accounting for coverage gaps, and a variety of confounding factors.

While we've seen a number of promising developments with data standards (particularly by HL7, which is leading the development of many of the standards relevant to electronic medical records), many of the standards we need haven't trickled down to the stage where they're being applied to the data that are currently available in operational systems.

And the most mature standards seem to be more oriented toward messaging rather than analysis. Yet some of the newest FDA technology initiatives are already preparing to receive clinical data and safety reports as HL7 messages that will be organized more like this electronic health care data than the CRF-oriented data domain tabulations prepared today, in the hope that some day all of this information will fit together for a wide range of regulatory purposes.

So how do all the various initiatives already in progress fit together? Which ones should sponsors pay most attention to? And what should they be doing on their own?

Again, too early to place your bets, but you should be watching the other bettors at the table closely.

Not all of this is good news to everyone. Not everyone wants to accept the conclusions of evidence-based medicine. There are legitimate fears that increased access to data may compromise patient privacy, and that the availability of such data will lead to faulty, misleading, and possibly even threatening conclusions. But there's no reason to assume that the results of such initiatives must end badly.

Right now when physicians find a condition, they prescribe a drug based on their previous experience.

Having a better understanding of the collective experience of how drugs are used, how they're working, and which ones should work better under different conditions for different subpopulations is something that should benefit everyone.

That's not to say you should put such powerful data resources in the hands of dilettantes and expect them to draw reasonably valid conclusions about anything either.

We'll first need a much better understanding of the myriad complications and an extensive library of contextual knowledge and sophisticated tools to use this information effectively and confidently. Which is what the research community is now working on.

Progression continues

What are researchers learning from the data so far? Well, it's kind of like the first few discoveries from an archaeological dig. You can already tell that some very valuable things are buried in there—you're just not sure exactly how to get to them yet.

As with any new frontier, we've only begun to scratch the surface, and it's not yet clear where we'll end up. So what impact may this have on clinical research? Dr. Helen Thomas offers one perspective in a recent paper for the Heritage Foundation: "Comparative effectiveness research must move beyond randomized clinical trials and embrace practical clinical trials. It should include observational data, and its methodologies should fully address issues such as the validity and applicability of findings."¹

So while tapping into these new data sources won't replace controlled clinical trials or the spontaneous reporting system, it promises to give us a much better picture of what's really happening in the world all around us with regard to the safety and effectiveness of health care treatments.

Meanwhile, as the interest and momentum builds, new start-up companies are looking at novel ways to get more contemporaneous, complete, and useful data available in a format that's more analysis-ready, or identifying new ways to collect such data by mobilizing communities of physicians and patients using Web collaboration technologies.

And extensive development of analytics, visualizations, and reusable research modules is proceeding rapidly.

With so many things going on, some of the best minds in the industry are now starting to converge on these shared issues. And that's how real breakthroughs occur.

So what can we observe from observational health data? Just watch and see. As Yogi Berra said: "You can observe a lot by watching."

Wayne R. Kubick is Senior Vice President and Chief Quality Officer at Lincoln Technologies, Inc., a Phase Forward company based in Waltham, MA. He can be reached at wayne.kubick@phaseforward.com

Reference

H. Thomas, "Comparative Effectiveness in Health Care Reform: Lessons from Abroad," February 4, 2009, http://www.heritage.org/Research/HealthCare/bg2239.cfm.