Clinical Trials vs Registries

As health care stakeholders, from regulators to providers to payers, demand more evidence on the safety and effectiveness of drugs and medical devices in real-world use, they are also increasingly asking life science companies to develop patient registries to fulfill this need.

(PHOTOGRAPHY: MARTIN POOLE, GETTY IMAGES ILLUSTRATION: PAUL A. BELCI)

Since registries are similar in many ways to clinical trials, on first review a CRO partner—strong in trials—would seemingly also be strong in patient registries. However, it is in fact the subtle (and not so subtle) differences that make this a potentially risky assumption for a purchaser.

For a number of reasons, an organization's need for designing and operating high-quality registries at a reasonable cost is best met by partnering with service providers that are experienced in this rapidly evolving research space.

Quality data

Registries are a valuable pre- and postapproval tool. In general, registries provide complementary data to that obtained from trials and are an increasingly important tool in safety surveillance.

In addition to safety evaluation, they are particularly useful for generalizing the findings of clinical trials to populations not studied in those trials and for evaluating the impact of physician decision making and actions on patient care.

Registries are also being utilized for observational comparative effectiveness studies because they can provide useful information at a lower cost than alternative methods.

These efforts require specialized knowledge in design and evaluation. More importantly, whether for review by a regulator evaluating a risk evaluation and mitigation strategy (REMS) or by a journal editor considering a registry manuscript for publication, registries are now judged on quality.

Since the 2007 release of the handbook Registries for Evaluating Patient Outcomes: A User's Guide¹ by the Agency for Healthcare Research and Quality, the ability to review a registry on its quality has become common.

The right partner

Many CROs have extensive experience with Phase I–III and sometimes Phase IV trials, but have not managed a proportionate number of registries, cohort studies or quasi-experimental designs. In choosing your partner, consider:

• Does the CRO have specific standard operating procedures (SOPs) and personnel focused on registries or are they borrowed from clinical trials?

• Have you reviewed those procedures?

• Have you reviewed the training and experience of the team that will be working on your registry?

While some functions are similar between clinical trials and registries, there are two key areas that are not: scientific skills (including protocol development, advisory panel guidance, biostatistics, and publications) and operations (including project and site management). Look for specialists in these critical areas—CRO personnel that focus entirely on registries and similar real-world studies and do not cross between study types.

Registry standards

A patient registry is "an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition or exposure, and that serves a predetermined scientific, clinical or policy purpose."¹

While clinical trials focus on efficacy—the extent to which medical interventions achieve health improvements under ideal circumstances—registries provide strong evidence for the extent to which medical interventions achieve health improvements in real practice settings.

A single standard does not exist for the conduct of observational research, but there are a series of publications and consensus efforts that can serve as guides to registry design and evaluation. These include AHRQ and their 2007 handbook mentioned earlier, the GRADE Working Group,² and GRACE Initiative.³ These resources serve as useful guides in shifting from a trial focus to an observational strategy.

Scientific guidance

An effective CRO partner will field a scientific team that will guide your organization through key stages of the registry, including developing a good protocol and case report forms; making an advisory board effective; and, generating high-level publications for top tier journals.

A quality registry begins with a good protocol. If a protocol is hard to understand or provides irrelevant background information, it can confuse or alienate physicians. As a consequence, it can compromise the study's ability to recruit sites and enroll eligible patients. A registry protocol that is forced into a clinical trial template can cause incorrect, and often unnecessary, safety reporting.

A scientific partner must have the ability to write a clear, simple protocol that will provide guidance on the different safety requirements that exist for a registry—not a clinical trial—and then facilitate the regulatory team in determining the appropriate procedures for the study.

In addition to protocol complexity, there are several other telltale signs of an inexperienced CRO working on a registry. Identifying the right data to collect requires an understanding of the analytical needs in the future; data sets can be broader and are generally focused differently than for a clinical trial. A CRO must understand what the minimum data set should be and how much data is too much.

Beyond the protocol, the scientific partner must also be able to educate registry advisors on the purpose and power of a registry; that education is typically ongoing throughout the registry. If the advisory board and CRO are inexperienced with registries and do not fully understand the benefits of the design, the reasons it was chosen over a trial, and what the pitfalls are along the way, all too often the project will drift to the more comfortable clinical trials paradigm—sometimes with disastrous consequences.

A third common problem with an inexperienced scientific team is that the benefits of the registry in terms of output are not realized. While certainly the lack of appropriately sophisticated analytic support for observational data will limit success, even more commonly, inexperienced CROs and advisors set expectations too low for registry outputs.

Good registry data and quality manuscripts from them are in high demand from top tier journals. Lower tier journals should not be the expectation for manuscripts produced from a high-quality study. If the CRO isn't suggesting top scientific journals for your registry, that could be a red flag.

Sponsors should review the publication lists of potential partners and ensure that the CRO has successfully published results from patient registries in high-quality journals.

Project and site management

The project management and site communication teams must also understand the unique considerations of observational research and be able to explain these to the enrolling physicians and other data contributors. The issues they encounter are frequently different than in a clinical trial.

For example, site managers may need to explain to a site why a registry does not pay for health care services. The CRO registry team must have the ability to build strong relationships with sites typically with lower payment levels than in a trial. The CRO must have methods to engage the practices or they will lose motivation and enrollment will lag.

A good understanding of technology and physician practice workflow is important in implementation, as busy practices commonly ask for workflow recommendations that are not commonly seen in clinical trials—which rarely take place in small, private practices. Appropriate technology, particularly technologies that provide workflow tools and can integrate with billing systems or electronic health records, can also be vitally important to successful implementation.

Quality assurance is a key area where registry operations and ultimately registry data will be judged. Registry experts have advocated for a risk-based approach to identifying the most likely causes for error and diligently creating processes and procedures to both avoid likely errors and to evaluate and report the types of errors that occur.

Because the types of bias that enter observational research are different than in a randomized trial, the quality assurance procedures also may need to be different. Simply borrowing QA processes from clinical trials can both dramatically increase costs and, at the same time, fail to provide the assurance of quality that is being sought.

This is a common and often very costly error of CROs inexperienced in high-quality registries.

Global experience

With increasing global requirements for sponsors, leveraging a registry across multiple countries is both efficient and often necessary. Experience in conducting registries in the countries targeted for the registry is often critical.

Ethical regulations, in particular, vary internationally for observational research, and they are not the same as those that govern clinical trials. For example, each of the 27 member states in the EU defines its own legislation pertaining to when ethical review is needed. Not all of these are clearly defined, and some countries allow local regions to outline their own specific requirements. Some countries, such as Germany, have additional regulatory requirements for observational research that one might not expect if accustomed to managing U.S. studies.

If a CRO partner does not fully understand the requirements of the regions in which an observational study is being conducted, it will, at best, result in spending unnecessary time and funds on regulatory approvals. Even worse, lack of global knowledge could lead to a violation of international ethics regulations.

CRO partners

A robust understanding and background in observational research, beyond clinical trials, is essential for a successful registry. Key components of the niche expertise in patient registries that should be sought in a CRO partner include:

• Strong scientific guidance

• Focused project management

• Site management and technology capabilities

• Global regulatory experience in observational studies.

Registries are large investments with tremendous opportunities to generate return on those investments. Selecting the right partner is more often than not the primary determinant in the ultimate success of the registry. Choose wisely.

Richard Gliklich,* MD, is president and Christina DeFilippo Mack is research coordinator of Outcome, 201 Broadway, Cambridge, MA 02139, email: richg@outcome.com

*To whom all correspondence should be addressed.

References

1. R.E. Gliklich and N.A. Dreyer, "Registries for Evaluating Patient Registries: A User's Guide," Agency for Healthcare Research and Quality, publication No. 07-EHC001 (AHRQ, Rockville, MD, 2007).

2. G.H. Guyatt, A.D. Oxman, G, Vist et al., "Rating Quality of Evidence and Strength of Recommendations GRADE: An Emerging Consensus on Rating Quality of Evidence and Strength of Recommendations," British Medical Journal, 336, 924-926 (2008).

2. N.A. Dreyer, Do We Need Good Practice Principles for Observational Comparative Effectiveness Research? GRACE Initiative www.graceprinciples.org (August 2008).