Is Investigative Site Feasibility Feasible?

Sponsors, CROs, and investigative sites widely agree that site feasibility assessments are frequently exercises in futility. Sponsors and CROs are known to routinely, and grossly, discount investigative site projections. Anecdotal reports from sponsor companies indicate that even these "adjusted" site feasibility assessments are accurate less than 10% of the time. Yet, sponsors base their selection decisions and invest expensive personnel and substantial resources to initiate and engage investigative sites based on these inaccurate assessments.

Kenneth A. Getz

With the majority of clinical trials running late by an average of six weeks, and less than half of all sites typically meeting or exceeding study enrollment goals, improvements in the feasibility assessment process could translate into millions of dollars in optimized direct investment and far more in faster enrollment performance.

A catch-22

On the surface, the process appears straightforward enough: Sponsors and CROs send out feasibility surveys to prospective sites asking how many subjects they could enroll in a particular protocol. Busy investigators and study coordinators provide figures that are used to aid in the selection of sites to conduct the protocol.

Historically, sites derived figures that were more reflective of their abilities and sponsors worked with the figures provided. Today, sites know that sponsors adjust feasibility figures based on personal experience and rudimentary guidelines. Sponsors describe having been burned in the past by what they consider overly optimistic and unrealistic enrollment projections. Sponsor companies believe that sites arbitrarily inflate their enrollment projections in order to win study grants. But competition alone does not explain investigative site behavior.

Another major cause of the problem has to do with the fact that sites are given incomplete and changing information with which to base their feasibility. "Sites are often asked to declare their interest and willingness to participate in a study without the benefit of having a full protocol and without the budget," says Deidre BeVard, who built site networks before joining ClearTrial as senior director of clinical services in March. "The protocol is still being developed, and hasn't been finalized. In what other business sector is this acceptable?"

Moreover, the line of questioning in the feasibility survey tends to be either irrelevant or impossible for the site to answer accurately with the information available to them and given the realities of their operating environment. Protocol eligibility criteria, for example, increasingly make it very difficult for sites to gauge patient availability. According to a recent Tufts CSDD study, the typical protocol has nearly 50 inclusions and exclusion criteria. Frequently, sites must look beyond their immediate pool of patients and speculate on those that will respond to recruitment advertising and promotion. As a result, the best that sites can do is guess.

Also, sponsors neither provide sufficient detail about the protocol for sites nor allow sufficient time to prepare more reliable figures. Research sponsors also don't customarily validate feasibility projections. "The top 10% of sites consistently do a good job on studies and the bottom 10% repeatedly under-perform," says Dave Zuckerman, president of St. Louis, MO-based Customized Improvement Strategies. "Enrollment success for the 80% in the middle is strongly protocol dependent. Yet sponsors almost never interview sites about the particulars of the protocol, press them for objective information to support their enrollment claims or give them enough time or compensation to plow through the feasibility process."

"For the most part, site feasibility is a very reactive and qualitative process," explains BeVard. "Investigators, whether they've been conducting studies for one year or 20, will generally estimate probability of success based on their personal knowledge of their patients. Their estimates essentially come from their gut intuition."

BeVard adds, "Investigators and study coordinators are equipped to provide a more educated projection had they the time to spare between revenue generating activities. When coordinator time is compensated, sites can take a more structured, deliberate approach...a consideration often not pondered."

Many sites report that sponsors have conditioned them to exaggerate their enrollment projections due to the arbitrary nature in which sponsors select sites. Sites reason that sponsors are motivated to select the lowest cost providers with the "best" enrollment projections. Sites are more attractive to companies if they report, erroneously or not, that enrollment goals can be met. Even if sites invested the effort to improve the accuracy of their enrollment projections—assuming this is possible—there's no guarantee that they'll get the study.

The process can be demotivating for many sites. They complain that they must reinvent the wheel every time a study is initiated. Investigative sites dedicated to clinical research are particularly frustrated. None of the information that they have submitted on dozens of studies appears to be shared between, and in some cases even within, sponsor and CRO companies. The minority of organizations that keep an investigator database don't leverage it much, if at all. Sites are forced to answer the same generic questions over and over again rather than verify information they previously shared. For high enrolling, experienced sites, being a proven performer on a past study apparently counts for little. "It's another example of the failure of sponsors to demonstrate that they value successful performers," lamented an investigator.

Sites also doubt the objectivity of the entire feasibility process, in part because they have never been shown how the site selection decision-making process really works. When sites get passed up they rarely receive feedback as to why. As a result, they are not learning from the experience.

Enhanced feasibility

Investigative sites need prompting and sufficient response time to validate their enrollment potential. Good Clinical Practice Guidelines put forth by the International Conference on Harmonization (ICH) state that sites should systematically demonstrate their ability to recruit the required number of subjects in the agreed upon recruitment period. Experts claim that sponsors indirectly encourage noncompliance by failing to require adherence to ICH guidelines and by neglecting to pay sites for undertaking the enrollment validation exercise. Several sponsors are changing their processes to require and reimburse a more robust enrollment validation assessment.

It is also critical that sponsors and CROs revise the feasibility assessment process to solicit better investigative site responses. Beth Harper, president of Dallas, TX-based Clinical Performance Partners, has spent the past 15 years advocating that site feasibility be accomplished by "asking the right question of the right individual in the right format at the right time."

"It's a continuum and a conversation...not a form," says Harper. "This initially means asking investigators about their interest in the medical question at hand, not just whether or not they have a minus 20 degree refrigerator or an efficient ethics committee review process. Understanding the characteristics that make sites better suited to one type of study versus another can form the basis of investigator profiles to guide future site qualification selection efforts."

In Zuckerman's experience, meaningful information can be derived from sites by asking "What's the largest number of subjects you could enroll or under what circumstances could you enroll 30 subjects rather than two?—in lieu of stock questions about total potential volume."

"To a large extent, it comes down to the way pharmaceutical and biotechnology companies design their studies," says Zuckerman. "There's no incentive for collaboration between clinical operations and therapeutic departments to ensure that a protocol is doable as well as scientifically sound."

Investigative sites, for their part, could begin by adhering to a tougher business development mindset and seeking out companies with trials for which their patient population is appropriate and their enrollment projections are more accurate. Unfortunately, few sites are either comfortable or can sustain being "conservative" to the point where they lose too many new projects.

Real collaboration

A growing number of sponsor companies are turning to site selection databases. These tools may provide objective information to help identify sites that have a higher likelihood of operating among a particular concentration of patients and a solid track record of performing certain studies. Companies are also using more sophisticated metrics to better predict and forecast site performance. Databases and predictive metrics provide useful input into the site selection process, but they do not replace the critical need for sponsors and CROs to nurture more effective communication and collaboration with potential and engaged sites.

Stronger site relationships offer invaluable insight for the sponsor and CRO into real performance issues and the support required to succeed. Stronger collaborations also convey value in the partnership.

There is no question that accurate site feasibility would be invaluable in improving site selection and ultimately in accelerating study conduct performance, quality, and cost. High interest among sponsors and CROs to improve their investigative site relationships holds promise for addressing a wide range of study conduct inefficiencies. Among them: today's flawed feasibility assessment process.

Kenneth A. Getz MBA, is a Senior Research Fellow at the Tufts CSDD and Chairman of CISCRP, both in Boston, MA, email: kenneth.getz@tufts.edu