New Benchmarks for Trial Initiation Activities

Sponsor and contract research organization (CRO) companies conducting global clinical trials face numerous operational and logistical challenges. With more than 3,000 active global clinical trials conducted annually at approximately 40,000 investigative sites dispersed worldwide, companies are looking for more efficient ways to streamline their processes in order to speed up timelines.¹ A number of factors impact study cycle times, including protocol amendments and patient recruitment and enrollment challenges. Tufts Center for the Study of Drug Development (CSDD) research found that substantial amendment presence in a protocol significantly increases study cycle times. Protocols with at least one substantial amendment had a study duration of three months on average longer than those protocols lacking substantial amendments.² In addition, research reveals a high turnover rate among investigators conducting trials, with about 40% each year choosing not to conduct any further trials, citing regulatory compliance burden and a challenging operating environment.¹ Recruitment challenges are typical, and findings from a recent study among examining 151 global clinical trials from 12 companies indicated that 11% of sites failed to enroll a single patient.³

Companies are also looking to increase efficiencies as they face more complex study protocols in therapeutic areas such as oncology (with the use of targeted therapies) and neuroscience (diseases such as Parkinson’s and Alzheimer’s). Prior research examining protocol complexity and burden on clinical site staff found that the investigative site work-effort to administer each protocol had increased 64% between 2002 and 2012.^4,5

Activities associated with site selection, study start-up, and site activation also pose challenges and company practices contribute to inefficiencies in these areas. There are hurdles within the regulatory environment, and timelines for site contracting, budget negotiations, and submissions.⁶ Organizations are continually looking to improve their study start-up cycle times through investments in technology, including data analytics and other software. These processes can potentially be streamlined through electronic cloud-based solutions, online clinical document exchange portals, and shared investigator databases.^7,8

Tufts CSDD research suggests that the early stages of the site initiation process accounted for the majority of cycle time and the highest variance (the pre-study visit to contract execution), while little variation was observed at the end of the process (contract execution to first patient in) across therapeutic area, type of site, and geographic region.⁸ In examining over 100 global clinical studies by therapeutic area, Tufts CSDD found that oncology and central nervous system therapeutic areas represented the longest cycle times to first patient in, while cardiovascular, infectious disease, and metabolic and endocrine studies had significantly shorter times to first patient in.⁸ In addition, academic institutions and government-funded sites took longest to first patient in, while physician practices were fastest. In a further analysis by global regions, sites in Latin America took twice as long as sites in North America to enroll the first patient.⁸

Study start-up is similarly viewed by investigative site staff as an inconsistent and inefficient process. Over 200 clinical research professionals who responded to a survey identified certain key areas that could be improved upon including contract and budget negotiations, regulatory processes, document management and communication between sponsors and sites.⁹

Based on the previous research on benchmarking clinical trial initiations and the wide variation and inconsistency among organizations, Tufts CSDD conducted a follow-up study examining practices and inefficiencies in site selection, study start and site activation. In-depth interviews were conducted with senior level pharmaceutical and CRO executives to better understand the challenges and overall strategies that organizations are using to improve these activities. The study was funded by goBalto, a technology solutions company.

A total of 26 interviews were conducted across 21 companies. In some cases, more than one representative from a company was interviewed across functions to gather more complete information and better understand the approaches for activities associated with site selection through site activation. Respondents represented 13 biopharmaceutical companies and eight CROs across 13 large and eight mid-sized and small organizations. Interviewees were experienced and were primarily senior level executives occupying director level roles or above. A few respondents held newly created roles within separate study start up functions.

Interviews examined the key criteria utilized to qualify and identify investigative sites and how practices vary across organizations. Use of repeat sites, experience level of sites, and therapeutic expertise were factors investigated. In addition, novel practices adopted by companies and any solutions currently being planned and implemented were identified. Use of performance indicators, including historical site data, and perceived or measured impact on cycle times and cost were also explored. Last, respondents reported the greatest challenges to study start up, site selection and activation and resources or investments that have improved these areas.

Site qualification and identification

The results suggest that sites are primarily identified based on therapeutic expertise, phase of research, and by global region. Respondents reported that the most critical factors are an organization’s previous experience with a site, site experience, performance, and capabilities, including sufficient staff and resources to conduct a study.

One respondent from a large pharmaceutical company discussed, “If it’s an area we know well, we go to sites we know. We use benchmarking data to help teams and affiliates. We get a download of all investigators in indications (globally) and rank them. We also do a therapeutic area scan and by country. We ask how active investigators are in the trial landscape. We take a list of top 10 investigators in the field.”

Another respondent from a small biopharmaceutical company noted that “we are a hybrid house. We outsource some of our trials and some we run ourselves. Outside the U.S. we don’t have boots on the ground; we work with a CRO partner.”

Global expertise becomes important, as there is regional variation in cycle times to first patient in. These data reflect similarities to the findings of an earlier Tufts CSDD study that found sites in Latin America and Eastern Europe had longer cycle times than sites in other regions, and North American sites took the least amount of time to first patient in (See Figure 1).

Through feasibility studies, organizations are able to discern a site’s ability to execute the proposed study and complexity of the protocol. A detailed on-site assessment was conducted in some cases. Past experience with a site is a strong indicator and is guided by intelligence gathered both internally and externally. Large companies tend to have more consistent and strategic approaches to gathering central and local intelligence on sites, and follow global processes. Sources used included CiteLine or TrialTrove and intelligence gathered by site management organizations (SMOs) or CROs. Smaller companies reported reliance on vendors that provide solutions searchable by indication or outsource all site selection. Respondents mentioned a number of other solutions to support site selection, including their own internal tools, clinical trial management systems (CTMS), and IMS’s Study and Site Optimizer (see Table 1).

A site’s compliance with good clinical practice (GCP) and the ability to meet protocol-specific requirements were also noted as site selection criteria. Respondents reported that site selection could be improved by gathering more intelligence on site

performance and leveraging data from past experience as well as program planning early on. One interviewee indicated that “we are using data more effectively to make decisions. We rely on performance; the number of protocol deviations; number of escalated issues for GCP non-compliance. We rely on the upper percentile of sites. To improve site selection, we can arm our teams with more intelligence.”

Another respondent reported that “site selection can be improved by earlier planning at the program level. Sites and country decisions are key. As you refine the needs for the protocol, you hone in on the investigator.”

Site selection process

Across 15 companies, on average, the results indicated that the proportion of sites is comprised of 70% familiar and 30% new sites. An overwhelming majority of companies use familiar sites rather than new sites, but usage may vary based on indication and geographic area (global location) of a study.

On average, site selection was reported to take approximately 3.2 months and could potentially range from two weeks to six months. In all, the process of site identification to activation is a one-year cycle. Factors impacting site selection timelines vary by therapeutic area, indication, geographic area, size, and phase of study. Type of site is also of particular influence since academic site selection

typically takes longer than community-based sites. Academic sites have several layers of review, including institutional review boards (IRBs) or ethics committees and scientific review committees. Some larger academic sites in the U.S. also have operational review committees, which can delay site activation and impact study start-up.

In particular, companies conducting oncology trials typically report lengthy timelines as they rely on large academic sites to conduct their trials. These results are comparable to earlier research examining first-patient-in cycle time by type of site and by therapeutic area.⁹ Academic or government sites have longer cycle times and physician practices have the shortest cycle times to first patient in. In addition, oncology and CNS or neuroscience studies had longer cycle times to first patient in than either metabolic/endocrine or infectious disease studies. In examining cycle times for Phase I compared with Phase II, III, and IV studies, a Phase I study took 6.5 months on average from pre-study visit to first patient in, while Phases II to IV took 10.4 months on average.

Use of historical site data to guide future site selection activity

Historical data are gathered at the study, site, and country levels and used to guide future site selection. Metrics collected include protocol approval to first site activated, first country activated, and average number of days to have a site up and running typically measured as time from site selection to final protocol approval to ready to enroll patients. Organizations like KMR Group and CMR provide benchmarks for company comparisons with internal data.

In addition to the key metrics cited, respondents reported that contracting and budgeting approval and execution cycle times were tracked. In particular, cycle times for a contract and budget sent to a site to contract execution, and contract execution to site initiation, are also measured. Findings from a prior Tufts CSDD study indicate that these early stages of the site initiation process account for the majority of cycle time and explored these results through further analyses. The results revealed that the cycle times associated with activities occurring from the pre-study visit to first patient in revealed more variation than those cycle times associated with site identification and with last patient last visit to database lock (see Table 2). In addition, site initiation

to first patient in had the largest coefficient of variation, indicating the greatest variability.

Other metrics cited by company representatives interviewed included: first patient enrolled or first patient first visit (FPFV) and number of patients randomized; IRB and ethics approvals and country submissions; and investigator and site staff performance, including turnover. Timelines around data entry, serious adverse event reporting (SAEs), and query resolution rate (or number of queries) were also part of historical data gathered on a site’s performance. Virtually no qualitative data was reported gathered by individuals interviewed, such as assessments of a working relationship with a site; the quality of the data; or number of queries relative to other sites.

New practices implemented by organizations

Of the 21 companies examined in this study, six companies were implementing new practices. Four companies (including pharmaceutical and CROs) have developed a new start-up function or created study start-up roles within their organizations. These organizational changes were designed to streamline all activities occurring from site selection to study start-up. Focusing on study start-up enables a dedicated group to provide needed support on trials and make continuous improvements. Cluster training has also been established at one company where core groups of sites are identified based on IRB approvals prior to the investigator meeting, and training is then provided on a site initiation visit (SIV).

Another company has eliminated the time between a site being protocol ready and the receipt of site documents. This process involves having all supporting documents sent to a site within 24 to 48 hours of a final protocol. A few companies also had recently implemented master service agreements (MSAs) between sponsors and sites and creating standard language for contracts.

Investments in technology are enabling novel practices in a number of ways. Some organizations are moving toward more data-driven site selection and are implementing centralized systems, using data visualization software, or other data analytics tools. Quintiles Infosario Site Gateway and further customization of the IMS Study and Site Optimizer were also mentioned as solutions.

Despite the new practices being instituted among organizations, only about one-third of companies indicated that there has been an impact on their non-enrolling or unproductive sites as well as greater percentages of sites being activated faster relative to internal benchmark data. The remainder did not report impact on cycle times or costs or have not yet assessed the impact of recently implemented practices. One interviewee noted an impact on non-enrolling sites: “Before the implementation of local dedicated study start-up staff, we had about one-third of our sites that were not delivering. We reduced that number to 10%.” This improvement was accomplished by adding dedicated staff and regional therapeutic area experts to manage site selection. This company representative serves as a liaison between the sites and the sponsor company.

Greatest challenges to site selection, study start-up and site selection

The results suggest that the most critical challenges are not only increased competition for sites but also fierce competition for other studies at sites in therapeutic areas such as oncology or rare disease. Challenges with site feasibility occur in the ability to obtain quality completions and having sufficient time to complete each part of the set-up process. Enrollment issues add additional pressures to the site selection process. Determining whether or not sites will be able to enroll patients within a tight timeline are continual challenges.

A major hurdle to study start-up is the contracting and budget negotiation process. Key issues cited by respondents include the lack of standardization on informed consent and site contract language, the absence of master service agreements (MSAs) with sites, and the need for country-specific templates to expedite contracts. Unpredictable timelines by IRBs and ethics committees were also cited as major delays to study start-up and some companies reported more efficiency and speed with the use of central IRBs.

Organizational resources and process changes

Expanding use of technology to implement process changes was a key theme across companies. The use of electronic medical records (EMRs), data warehousing, and data mining is used by companies to inform numbers of patients that meet inclusion/exclusion criteria. Some cited the use of the Investigator Databank or TransCelerate Biopharma’s Shared Investigator Platform. Other solutions mentioned were site feasibility software, internal investigator dashboards, and the use of goBalto technology.

Contracts and budgets functions were also identified as areas where resources were added. Some companies added staff primarily to negotiate and execute contracts and worked to establish MSAs with sites and improve legal agreements. Establishing fair market value for budget negotiations was mentioned as standard practice. Informed consent was also an area being improved upon through the development of more standard, “user-friendly” templates.

Some respondents indicated that their trial optimization or strategic planning groups contributed to the efficiency of start-up activities. These groups established an increased focus on site selection and start-up or in some cases developed an integrated site activation plan.

Conclusion

Despite the attempts to streamline and speed study start and improve site selection and activation, these activities are still fraught with challenges and many of the findings from this study are corroborated by an earlier Tufts CSDD analyses on study start-up. The earlier study found that sites conducting oncology trials represented the longest cycle times to first patient in and academic- and government-funded sites took longest to enroll the first patient.⁸ Companies in our current study also noted the lengthy and cumbersome review process occurring at academic sites. Also, these were typically large academic sites conducting oncology trials with lengthy timelines.

Additionally, one of the areas with the greatest cycle times in the earlier Tufts CSDD study focused on the pre-study visit to contract execution.⁸ Similarly, the results of the current study indicate that one of the major challenges to study start-up is the contracting and budget negotiation process-and it is an area where new processes are being implemented across organizations. In addition, the results on cycle time variance suggest that contract execution to site initiation and site initiation to first patient in may potentially be improvement areas for organizations.

According to the results of our study, despite new practices and process changes being instituted, the impact on improving study start-up cycle times has been fairly small. The investment of new technology and other solutions has made incremental improvements and has provided added intelligence on site performance globally, specifically start-up cycle times and site activation. These solutions have enabled companies to implement more efficient start-up and site selection processes. Adoption of these solutions, however, has been inconsistent as is the ability to integrate data from multiple sources. Some organizations are continuing to strengthen their cycle times and improve efficiencies through a focus on study start and site activation either with adding new functions or additional resources in combination with using data and other solutions more efficiently.

Mary Jo Lamberti, PhD, is Senior Research Fellow; Ranjana Chakravarthy is Research Analyst; and Ken Getz, MBA, is Director of Sponsored Research and Chirman of CISCRP; all with the Tufts Center for the Study of Drug Development

References

1. Getz, K.A., Lamberti, MJ Global site landscape remains highly fragmented with variable performance. Tufts Center for the Study of Drug Development Impact Report. March/April 2013; 15 (1-3).
2. Getz K, Stergiopoulos S, Short M, Surgeon L, Krauss R, Pretorius S, Desmond J, Dunn D. The impact of protocol amendments on clinical trial performance and cost. Therapeutic Innovation & Regulatory Science. 2016; Published online before print February 22, 2016 http://dx.doi.org/10.1177/2168479016632271.
3. Lamberti, MJ, Mathias, A. Myles JE, Howe D, Getz K. Evaluating the impact of patient recruitment and retention practices. Drug Information Journal. 2012; 46(5): 573-580.
4. Getz, KA, Kim J. Stergiopoulos, S, Kaitin, KI New governance mechanisms to optimize protocol design. Therapeutic Innovation & Regulatory Science, 47 (6) 651-655 (2013).
5. Getz KA., Stergiopoulos S., Marlborough M, Whitehall J., Curran M., Kaitin KI. Quantifying the magnitude and cost of collecting extraneous protocol data. American Journal of Therapeutics, 22(2) 117-124 (2015).
6. Morgan C. The need for speed in clinical study startup. Clinical Leader. Link accessed February 10, 2016 http://www.clinicalleader.com/doc/the-need-for-speed-in-study-startup-0001.
7. Schimanksi C, Kieronski M. Streamline and Improve Study Start up Applied Clinical Trials 22(9) 22-27 (2013).
8. Lamberti, MJ, Brothers C, Manak D, Getz KA. Benchmarking the study initiation process. Therapeutic Innovation & Regulatory Science 47(1) 101-109 (2013).
9. Clinical Trials Survey Results. Quantia Communications, Inc., 2011, a division of Aptus Health.