Enrollment Performance: Weighing the "Facts"

In 2000, a curious metric on investigative site performance was presented at a small conference in Philadelphia and it quickly invaded the industry's collective conscience becoming a widely cited "fact." That metric, based on interviews among clinical operations managers, held that 20% of all investigative sites fail to enroll a single patient; 30% under-enroll; 30% of investigative sites meet enrollment targets; and 20% exceed target enrollment levels.

Kenneth A. Getz

Clinical operations professionals to this day—now more than a decade later—frequently cite this metric to convey the adverse affects of poor investigative site identification, selection, and performance, and the highly risky nature of patient recruitment and retention.

Results of a 2012 quantitative study conducted by the Tufts Center for the Study of Drug Development (Tufts CSDD) updates this metric. Using a robust quantitative methodology, the results of this new study provide a more nuanced set of conclusions and insights.

For this study, 10 pharmaceutical companies and two major contract research organizations provided performance data on nearly 16,000 investigative sites that participated in 151 Phase II and III global clinical trials between 2008 and 2010. The typical clinical trial in this study matched industry-wide benchmarks with regards to protocol scope and complexity and was conducted in 11 countries, on average, among adult patients.

Primary performance measures gathered in this 2012 study included: enrollment activation rates, the percentage of investigative sites ready to enroll patients that actually do enroll at least one patient; enrollment achievement rates, the proportion of patients actually completing the clinical trial compared to the planned completion level of patients; and actual to planned study enrollment timelines.

Ready to, but don't enroll

Whereas the earlier metric held that 20% of investigative sites fail to enroll a single patient, the Tufts CSDD study found that a considerably lower percentage—11%—of investigative sites ready-to-begin recruitment failed to enroll a single patient.

The 2012 Tufts CSDD study also found wide variability in enrollment activation rates by geographic region: investigative sites in Western Europe, Eastern Europe, and Asia Pacific had the lowest non-activation rates at 7%, 8%, and 9% respectively. Thirteen percent of North American investigative sites failed to enroll a single patient. Investigative sites in Latin America had the highest non-activation rates with 20% failing to enroll a single patient.

In sum, the vast majority—89%—of investigative sites are activated and begin enrollment in a Phase II and III global study with wide differences observed—from 80% up to 93% enrollment activation rates—across geographic regions.

Achieving target enrollment level

Overall, investigative sites enrolling at least one patient achieve 85% of the target enrollment number of patients. Of the total investigative sites activated (i.e., ready-to-enroll patients) the Tufts CSDD study found that 41% were unable to achieve the target enrollment number of patients. This percentage includes sites that enroll only one patient up through those that miss the enrollment target level by 10% or more. Four out of 10 investigative sites achieved the target enrollment number and 15% exceeded the target number.

Enrollment achievement rates also varied widely by geographic region and show an inverse pattern to that noted earlier: Investigative sites in regions that had the highest enrollment activation rates tended to have the lowest enrollment achievement rates. This suggests that regional activation rates are not an indicator of overall success.

North American and Latin American sites in the 2012 Tufts CSDD study hit or exceeded target enrollment levels at 98% and 104% respectively. Investigative sites operating in the Asia Pacific region achieved 107% of the target enrollment number of patients. Sites in Western and Eastern Europe had the lowest enrollment achievement levels at 69% and 75% of target respectively.

Setting the denominator to all investigative sites in the study, not just those "activated," 48% of the 16,000 sites included in the 2012 Tufts CSDD study under enrolled. Thus, nearly half of all investigative sites identified and selected to participate in clinical trials fail to enroll a single patient or under-enroll. This compares with 50% of investigative sites under-enrolling proposed in 2000. Other proprietary benchmarks (e.g., CMR and KMR) have noted that approximately one-third of all investigative sites enroll zero or one patient in a typical clinical trial.

Enrollment Activation and Achievement Rates

The Tufts CSDD study found that for the 151 global clinical trials evaluated, on average, 928 patients were screened, a mean of 622 study volunteers were enrolled, and 516 completed participation. One in six (17%) enrolled in each clinical trial dropped out before completion.

Slipping timelines

A relatively high percentage of investigative sites are activated, begin enrollment, and recruit the expected (or more) number of study volunteers. At issue is when they do so. Results of the 2012 Tufts CSDD are troubling as they suggest that sponsors and CROs must permit study timelines to slip in order to give sites more time to hit their target enrollment numbers.

Sponsors and CROs report that less than half (47%) of their studies completed enrollment within the planned timeline; 53% of studies had extended timelines, with one of six studies taking more than twice as long as originally planned.

These figures are likely underrepresenting the problem. It was difficult for companies participating in the study to discern the frequency and extent to which planned timelines were adjusted to accommodate protocol amendments, changes in study scope, and other factors.

Past studies conducted by CenterWatch and others have solicited investigative sites to report their perceived delays in completing clinical trials. Several hundred investigative sites in a recent CenterWatch study reported that upwards of 90% of clinical trials are delayed and fail to meet the original timeline indicated in the study contract.

Interpreting the "facts"

The results of this 2012 study provide new, updated metrics characterizing investigative site enrollment performance: One in 10 sites, on average, fails to enroll a single patient in a given clinical trial. Nearly six out of 10 (59%) activated investigative sites meet or exceed their target enrollment levels. And more than half (53%) of studies take longer than planned to give activated sites in a global clinical trial enough time to achieve 85% of their target enrollment numbers.

Sponsor and CRO focus on the selection and better management of investigative sites is clearly one major lever to improve enrollment performance. But it is not the only area demanding attention. Other key levers have been discussed regularly in this column and they require sponsors and CROs to be more introspective and proactive in finding ways to support investigative site success. These areas include:

Simplifying protocol designs in order to optimize scientific and operating complexity, reduce stringent volunteer eligibility criteria, and minimize study delays. Tufts CSDD studies have consistently shown a positive association between protocol complexity and longer timelines, poorer patient recruitment and retention rates, and a higher incidence of protocol amendments. Updated metrics in this 2012 study indicate that timelines are a collective problem and are substantially challenged by ambitious clinical trials.

Expanding and better educating the population of potential study volunteers and other stakeholders drives higher levels of volunteerism. Online clinical trial matching services and social media offer new ways to reach target patient populations. But these approaches primarily pursue patients in the immediate participation window. In order to establish a sustainably higher and steadier level of patient support, broader outreach and education is necessary to engage healthcare providers and other trusted sources of health information and to raise overall public awareness and support of clinical research.

Improving the accuracy and feasibility of study timelines to help sponsors and CROs set more realistic expectations internally and among investigative site partners. Poor site performance is not the only explanation for the high percentage of studies in which timelines must be extended. Improved planning and forecasting gives sponsors and CROs greater confidence in study timelines; instills more credibility and ensures higher levels of adherence; and provides a better basis to understand factors that most impact variance from planned performance.

Kenneth A. Getz MBA, is a Senior Research Fellow at the Tufts CSDD and Chairman of CISCRP, both in Boston, MA, e-mail: kenneth.getz@tufts.edu