Using Pragmatic Randomization in Real-World Studies

Real world observational studies are gaining prominence as a way to gather valuable information about the impact of a treatment outside of the highly controlled clinical trial environment. Observational studies track the behavior and real world outcomes of diverse patients at various stages of disease, helping researchers learn which treatments result in clinically meaningful improvement, for which type of patients and in what situations. Observational studies can also provide information about treatment heterogeneity that is needed to develop a more personalized approach to medical care. By definition, observational studies do not interfere with decisions about which treatments patients will be offered. Instead of dictating which treatments will be used, “observational” studies rely instead on information that is captured through the normal course of treatment, whatever that may be, and require limited, if any, additional testing.

The challenge with these studies is the unintended bias that determines how patients are treated.

When a new treatment comes to market, it is often first prescribed to patients who have failed to respond to other treatments, and who may start on a new drug as a last resort, when their disease progressed beyond reversal or they have tried every other treatment without any benefit. As a result, they may have outcomes that are quite different than would be encountered without the bias that can result from “channeling” sicker patients to new treatments.

To overcome this bias, some researchers use pragmatic randomized clinical trials (PCT), where treatment randomization is used to balance baseline characteristics and risk factors between treatment groups. Then information collected from observing encounters in everyday medical practice is used to gauge practical impacts on patients’ diseases. PCTs are gaining popularity as a way to quickly and less expensively evaluate the real world impact of marketed products, at lower cost and with less complexity than randomized clinical trials (RCTs).

Once physicians and patients agree to participate in a PCT, the investigator follows a randomization schedule to assign treatment and then conducts follow-up according to customary clinical practice. Treatments are not blinded. Comparators are generally whatever is used as standard of care in the regional setting, but placebos or sham treatments are never used in PCT.  Sometimes additional clinical assessments may be requested to help gauge improvement but they must be consistent with real-world practice. Patient-reported outcomes also may be collected in order to evaluate the impact on daily living and quality of life.

While it may sound simple, it isn’t.

In the absence of blinding patients and observers as to which treatments are being used, it is critical to evaluate if and how assessments of outcomes will be impacted by patients and physicians knowing what treatments they are receiving. It is also essential to determine whether the outcomes of interested will be detectable and recorded in accessible real-world data, like electronic medical records or health insurance claims. Nonetheless, with appropriate evaluation and planning, PCT can be used to address clinical questions of importance to many stakeholders, including patients and health care providers as well as regulators and payers.

Changing the way we answer questions

While this is a relatively new research model for the pharma industry, PCT are catching on. Consider ADAPTABLE, a three-year PCT funded by PCORnet, the National Patient-Centered Clinical Research Network, to study whether a daily dose of aspirin will prevent heart attacks and strokes in individuals living with heart disease. The ADAPTABLE study, just starting, will randomize treatments to one of two doses of daily aspirin in 20,000 participants to assess cardiac benefit. Investigators will use existing data sources to track major adverse cardiac events, including hospitalizations and death. The cost of ADAPTABLE is expected to be about $14 million whereas a classical RCT of similar size was recently completed at a cost of roughly $420 million. So, for the same price as a single classical cardiovascular outcomes randomized trial, researchers could conduct 30 different pragmatic trials, pursing many questions of real-world importance.

There have been several examples of successful PCTs, including the Salford Lung Study (SLS) in Manchester, England which used electronic health record data and National Health Service (NHS) data to recruit and randomize treatments to a population of 2800 consenting patients to examine the safety and effectiveness of a new treatment for chronic obstructive pulmonary disease (COPD). The study ultimately helped the sponsor demonstrate the value of this new treatment for clinicians, patients and payers.

Regulators are also beginning to show interest in PCTs as a way to establish real world effectiveness and possibly even for label expansions. The US Food and Drug Administration hasn’t issued guidance for using PCTs, however, in May, 2016 the National Institutes of Health (NIH) sponsored a workshop on the Ethical and Regulatory Issues of PCTs; and in January, 2017 the NIH Office of Disease Prevention (ODP) released an online course on how to design and analyze pragmatic and group-randomized trials. The European Medicines Agency (EMA) has also shown a willingness to consider data from PCTs, and references their use in its Scientific guidance on post-authorization efficacy studies.

While there is no guarantee that regulators will accept PCT data as proof of effectiveness that is sufficient to broaden a label for a product already on the market or to support a health technology assessment, sponsors interested in using this cost-effective approach should meet with regulators and/or payers before starting a PCT to discuss the goals and proposed plan in order to address any concerns those stakeholders may have. Also PRECIS-2, a checklist to help evaluate all the parameters by which a trial can be pragmatic, should be consulted during the design of a study, since there are many design features that can influence how pragmatic a trial is, and not an absolute dichotomy between classical and pragmatic RCT.

This study design can have a profound impact on the way we answer real world questions around efficacy, effectiveness and the safety of marketed products going forward - the cost benefits and demonstrated quality that can be achieved with these PCT make them too valuable to ignore.

Nancy Dreyer, MPH, PhD is Global Chief of Scientific Affairs and Senior Vice President, Real-World & Late Phase Research at QuintilesIMS