The Balance Between Data and Diversity

Despite years of effort, many clinical trials fall short in collecting and evaluating how an experimental medical product may affect a broad spectrum of patient populations, including women, the elderly and ethnic groups. FDA addressed this challenge at a workshop in early December 2015 on the importance of, and methods for, assessing the impact of medical interventions on different patient subgroups. Hosted by FDA and the Johns Hopkins University Center of Excellence in Regulatory Science and Innovation (JHU-CERSI), representatives of industry, academia, and patients examined the importance of diversity in medical research and how to design studies that include an adequate representation of people likely to use an approved intervention.*

The meeting follows up on an FDA Action Plan to Enhance the Collection and Availability of Demographic Subgroup Data issued in 2014. That examined ways to improve the completeness and quality of demographic subgroup data collection, reporting and analysis. It sought to identify barriers to enrolling subgroups in clinical trials, strategies to encourage greater participation, and ways to make demographic subgroup data more available and transparent. 

Keith Ferdinand, professor of medical practice at the Tulane University School of Medicine, opened the workshop by explaining concerns about the generalizability of clinical trial findings when they are not derived from studies of diverse populations. The failure to enroll sufficient subgroups in clinical trials may provide only average results that don’t inform clinical practice.

In a keynote address, FDA Deputy Commissioner for Medical Products and Tobacco, Robert Califf, described the many challenges to finding a balance between clinical trials large enough to assess all relevant populations and small enough to include deep data on each patient. He said that it seems “inevitable to me” that comprehensive evidence will include small focus studies to support precision medicine, along with very large trials using data generated from medical practice and personal devices “to get the inclusiveness you are talking about today.”

Califf suggested that sponsors and researchers need to explore how to “weave together evidence” from classical clinical trials, pragmatic trials and population studies to derive conclusions meaningful to patients. In thinking about inclusiveness, Califf said that it isn’t just who is included in the trials, but how diverse populations are included in the design of the trials and to identify what is important to them.

A key issue, Califf said, is whether there is sufficient evidence to indicate that heterogeneity is frequent enough that it needs to be looked at systematically in every case. He noted that it’s important to assess when it’s reasonable to depend on clinical pharmacology and biology to inform about heterogeneity and treatment effect, and to assess if it’s possible to use that information to make decisions without doing “the big study.”

Califf also commented on how “the dramatic computational revolution” and availability of ‘omics in systems biology is changing methods for evaluating clinical research. “We’re just on the cusp of beginning to understand what the Big Data means,” he observed. “It may be useful to tap non-randomized epidemiological data to draw causal inferences about the presence or absence of a safety issue in a sub-population, or to extrapolate data from a study to support efficacy and effectiveness," and said cloud computing makes computation easier.

In thinking about diversity in clinical trial participants, Califf speculated whether the same principles apply to medical devices as for drugs. He also noted that more and more evidence is coming from outside the United States, given that 96% of people do not live here. Califf put forth the following queries in those cases:

How and when do trials need to require U.S. populations?

What happens when differences are found according to geography?

When differences in results are found, how are they assessed for reality vs. chance?

How can these issues be overcome by not requiring much larger sample sizes?

Social media may be helpful in identifying meaningful outcomes for diverse populations, Califf added. New technology now provides ways to go directly to people to find out what’s important to them, but he advised researchers also to understand how access to social media may create disparities-or reduce them.

Califf noted the important tradeoff between clinical trial internal validity and generalizability: that the more precise a trial, the harder to prove that results of a study pertains to the next patient. He suggested that this kind of “inherent tension” could be alleviated by greater use of electronic health records to obtain information on treatment impacts. Most importantly, however, is to have well-designed clinical studies that can support the use of a product based on good evidence. The diversity in methods for collecting data, he observed, now is matching the diversity of people.

Author’s note: Comments excerpted from remarks and slides presented Dec. 2, 2015 at Clinical Trials: Assessing Safety & Efficacy for Diverse Populations Workshop.