Efficient Pre-implementation of Multi-site Clinical Trials

Increasing the speed with which promising pharmacological treatments move from pre-clinical research to patient-care utilization has received increasing focus in recent years¹ and is a critical goal of the National Center for Advancing Translational Sciences (NCATS), an NIH institute established in 2011.² Some streamlining efforts focus on issues of great scientific appeal, such as the development of chips mimicking human tissues to be used to screen for toxic drug effects. Other efforts are focused on more mundane, yet important, issues such as ensuring that investigators have needed access to regulatory support and data management tools.³ Achieving more efficient translational research will entail reducing the time and expense required to conduct clinical trials in the United States.^4,5 A report from the Institute of Medicine in 2010 delineated several key challenges to the efficient completion of multi-site clinical trials.⁶ This paper addresses one of the issues raised, which is the relatively long time that the pre-implementation phase, the phase between sponsor approval of a protocol and initiation of recruitment, can take to complete.

A long pre-implementation phase is problematic in that it can significantly increase the costs of conducting the trial and reduce the speed with which findings are available, and hence, their potential relevance.⁶ The pre-implementation phase for multi-site clinical trials is complex and typically entails coordination among multiple institutions and among individuals with varying areas of expertise. There can, thus, be a number of potential reasons for a lengthy pre-implementation period. For example, the time required to obtain all regulatory approvals can significantly prolong the pre-implementation phase.⁵ There are three primary causes of prolonged pre-implementation periods, only one of which, the occurrence of unforeseeable events, cannot be avoided. A second cause is the human tendency to underestimate the amount of time a project will take to complete, which is a well-documented human bias; indeed, understanding the factors that contribute to the bias, and potential methods for countering them, has received a good deal of study.^{7,8,9,10,11,12,13,14,15} A third cause is problematic coordination among the entities that play a key role in the clinical trial. This paper describes the Pre-implementation Timeline Calculator, which is a tool designed to help investigators plan and coordinate the completion of the pre-implementation phase in a timely manner and presents data suggesting that the Calculator may be helpful in reducing the time and cost needed to complete the pre-implementation phase.

Context

The Pre-implementation Timeline Calculator was developed in working with the National Drug Abuse Treatment Clinical Trials Network (NIDA CTN). The CTN was established in 1999 in response to an Institute of Medicine report delineating the relative lack of information exchange between substance abuse researchers and substance abuse community treatment providers.¹⁶ In the CTN, substance abuse researchers work with treatment providers to design clinically meaningful trials and implement them in "real-world" settings utilizing existing staff as interventionists and clinic patients as participants. The CTN links substance abuse researchers and treatment providers through nodes, with each node consisting of a regional research and training center (RRTC), which is affiliated with a university, and multiple community treatment programs (CTPs) that work with the RRTC. Oversight of the CTN is provided by the Center for the Clinical Trials Network (CCTN). The CCTN contracts with a clinical coordinating center (CCC) to provide regulatory guidance, quality assurance monitoring, central pharmacy and laboratory services, and study supplies. The CCTN also contracts with a centralized data and statistics center (DSC), which provides statistical support and data management for CTN trials. Study oversight is provided by the RRTC of each participating CTP and by the study's executive committee, which is comprised of the lead investigator (LI), members of the LI's team, and representatives from the CCTN, CCC, and the DSC.

The Pre-implementation Timeline Calculator

Overview. To help manage, and successfully complete, pre-implementation in a timely manner, the Pre-implementation Timeline Calculator was developed for pharmacological multi-site clinical trials. The Calculator divides the pre-implementation stage into six general tasks.

The Calculator itself is a Microsoft Excel worksheet and is available for free at the CTN Dissemination website (http://ctndisseminationlibrary.org/resourcespolicies.htm/). Cells with dotted borders require direct entry of the actual date associated with the task being referenced. Most cells in the Calculator are pre-filled with formulas and locked; time-to-completion targets may be adjusted as needed by changing the value in the associated cell. Based on the date entered for "sponsor approval of protocol," recommended dates for target site initiation and the investigators' meeting are generated.

As noted earlier, a long pre-implementation phase is problematic.⁶ For CTN trials, the target is to initiate recruitment eight months after receiving sponsor approval of the study protocol. While the goal is to complete the overall pre-implementation phase as efficiently as possible, particular attention is given to minimizing the time between completing staff training and initiating participant recruitment, since staffing costs constitute the majority of costs for a clinical trial; thus, it is important to minimize staffing costs at sites that are not open for enrollment. The target of holding the investigators' meeting one month prior to site initiation is based on the time generally needed to complete site preparation and quality assurance (QA) visits following the meeting and prior to site initiation.

Instructions for using the Calculator. Utilization of the Pre-implementation Timeline Calculator to establish the pre-implementation timeline for the trial entails three steps:

• The investigator enters dates for sponsor approval of protocol, target for site initiation, and investigators' meeting.

• The investigator and his/her team review each task and determine if there are any tasks that are not relevant for the trial or which were completed prior to the sponsor approving the protocol. For these tasks, a "dummy" date is entered into the Calculator in the "Actual Date Completed" column. For site selection tasks, the dummy date should be the date that the sponsor approved the study protocol. For all other tasks, the date completed should be the "Target Date of Completion" generated when the three dates listed in step 1 were entered.

• The investigator and his/her team determine if the "Estimated Time to Complete" needs to be adjusted for any of the tasks and adjusts the estimated number of weeks to complete as necessary. The appearance of an "X" in the "Timeline Problem" column indicates that site initiation eight months following sponsor approval may not be feasible and a later date for "Target Date for Site Initiation" should be entered.

Commitment to the timeline should be obtained from all parties playing a key role in the pre-implementation phase (e.g., investigator, data management center, parties responsible for securing study medication and supplies, QA monitoring, etc.). The investigators' meeting should then be scheduled, avoiding both holidays and relevant professional conferences, to ensure the availability of the venue for the meeting. At the end of this process, the pre-implementation timeline for the trial has been established and should be shared with all parties playing a key role in pre-implementation, including the study sites, once they have been selected. The provision of the timeline allows each entity time to complete their piece according to schedule.

General pre-implementation tasks. The following is a description of the six general tasks involved in the pre-implementation stage:

1. Site selection. The goal is to have the sites selected at least one month prior to the expected initiation of regulatory activities. Many clinical trials require that an institutional review board (IRB) submission be completed at the level of the site, and many sites will require that a contract/subaward be established before any work, including preparing IRB submissions, can commence; allowing a month for contract/subaward establishment can help avoid delays in initiating needed regulatory activities. Since the Calculator was created for use in the CTN, a network in which information about candidate sites, including prior performance, typically is readily available, the tool reflects the time needed to obtain a completed study questionnaire from sites and to conduct telephone interviews with promising sites. When the Calculator is used for trials requiring site visits for site selection, the "Estimated Time to Complete" number needs to be increased for the "Interview Sites" task.

2. Regulatory approvals. Obtaining all needed regulatory approvals can require a substantial amount of time. In the case of U.S. pharmacological trials, this will include obtaining an investigational new drug (IND) or an IND waiver from the FDA. For many multi-site clinical trials, it will also necessitate obtaining approvals from multiple IRBs. The goal, therefore, should be to have all regulatory approvals obtained prior to the investigators' meeting so that regulatory-related delays in initiating recruitment can be avoided.

3. Data management. Clinical trial data management system development often takes significantly more time than expected. The timeline for creating the data management system is based on the time required for past multi-site trials.

4. Study materials, medications, supplies. The target dates for completing the operations manual and having the study medication and supplies on site are entered into the Calculator. A thorough, user-friendly operations manual is a key piece of a successful multi-site trial and generally should be completed as early in the pre-implementation process as possible, since its completion can delineate feasibility issues with the protocol that should be corrected prior to IRB submission. Clear delineation of responsibility for obtaining study medication and supplies, and securing commitment to meeting the timeline, needs to be completed as part of determining the pre-implementation timeline for the trial.

5. Site staffing. As noted, staffing constitutes a significant portion of the cost of conducting clinical trials and, thus, the site staff should be hired so that the time elapsing between hiring and initiation of subject recruitment is the amount of time needed to complete training and site preparation. As tracked in the Calculator, research assistants (RAs) typically are hired earlier than the remaining staff, which reflects the additional training requirements for the RAs. The other study staff can be hired approximately two weeks before the investigators' meeting, which allows enough time to make travel arrangements to attend the meeting. The amount of time required to hire staff can vary dramatically, from very little time for sites moving staff from one trial to the next, to several months for sites that need to establish a new position and complete the hiring process.

6. Investigators' meeting and site initiation. As noted, the goal is to hold the investigators' meeting approximately one month prior to the target date for initiating subject recruitment.

Use of the Calculator to condense the pre-implementation phase

From a budgetary perspective, the lengthening of the pre-implementation period can result in staff being hired and the investigators' meeting being held without the trial being ready to start (e.g., the data management system is not complete, needed study materials have not been finalized, etc.). Evaluating whether the Pre-implementation Timeline Calculator reduces pre-implementation phase length would require that clinical trials be randomly assigned to Calculator use or to a control condition, which has not been done. However, preliminary data on the Calculator's potential to reduce the length of the pre-implementation phase is available and has been used for several multi-site clinical trials, including three pharmacological trials; these trials are referred to as Calculator trials.

The Calculator trials were all CTN studies initiated between 2004 and 2012. Trials with which to compare the Calculator trials were samples of convenience—substance abuse pharmacological trials conducted during approximately the same period as the Calculator trials for which information about the pre-implementation phase length was available. The non-Calculator trials are four pharmacological substance abuse trials initiated between 2003 and 2011, which include both pharmaceutical and NIDA (including CTN)-sponsored trials. The time required for pre-implementation can vary according to several factors, including the complexity of the trial and the amount of staff time devoted. Thus, to provide a more stable estimate of the pre-implementation timelines between the Calculator and non-Calculator trials, average times for each trial type are provided.

Figure 1 provides the average times taken for two segments of the pre-implementation phase: the period between having a sponsor-approved protocol and the initiation of recruitment at a study site and the period between the investigators' meeting and the initiation of recruitment at a site. As illustrated, the Calculator trials were about six months shorter than the non-Calculator trials for the first period and three months shorter for the second period. To provide an estimate of how elongation of the second period can contribute to the cost of a clinical trial, the staffing cost associated with the additional three months was calculated for a hypothetical trial conducted at 10 sites with site staffing, as outlined in Table 1. On average, there were 48 days between the investigators' meeting and the initiation of recruitment for the Calculator trials and 145 days for the non-Calculator trials. With a cost of $400 per day, per site for staff costs, the staffing costs for the Calculator trials would be $192,000 for 10 sites for the period between the investigators' meeting and the initiation of recruitment and would be $580,000 for the non-Calculator trials, a difference of $388,000.

Conclusion

The goal of streamlining translational science, of which streamlining multi-site clinical trials is a component, is critical.¹ Achieving this goal will entail the dedication of many investigators, as well as attention to factors that, while mundane, can have a significant impact on the speed with which promising therapeutic treatments reach patient care.³ This report addressed one challenge to the efficient completion of multi-site trials—the relatively long period of time that it can take to complete the pre-implementation phase. It discussed the use of the Pre-implementation Timeline Calculator, which was developed to help investigators to plan for, and complete, the pre-implementation phase in a timely manner. Preliminary data suggest that using this tool may significantly reduce the time and financial resources required for pre-implementation. An important limitation of the present data is the reliance on a sample of convenience for the Calculator and non-Calculator trials. However, investigators who have completed multi-site pharmacological trials have access to the length of the pre-implementation phase for their trials and, thus, can determine whether the use of a tool like the Pre-implementation Timeline Calculator may lead to time and cost savings. In general, given that the Calculator has no discernible disadvantages, its more widespread use in multi-site clinical trials seems warranted.

Theresa Winhusen, PhD, is Professor of Psychiatry and Behavioral Neuroscience; Director of Addiction Sciences Division, University of Cincinnati College of Medicine, email: [email protected].

References

1. Zerhouni EA. Translational and Clinical Science—Time for a New Vision. New Engl J Med 2005; 353: 1621-3.

2. Insel TR. Invited Response to: Preparedness of the CTSA's Structural and Scientific Assets to Support the Mission of the National Center for Advancing Translational Sciences (NCATS). Clin Transl Sci 2012; 5: 130-1.

3. CTSA Principal Investigators, Shamoon H, Center D, et al. Preparedness of the CTSA's Structural and Scientific assets to support the mission of the National Center for Advancing Translational Sciences (NCATS). Clin Transl Sci 2012; 5: 121-9.

4. Dilts DM. Early Warning: An Ailing Canary in the Mine. J Clin Oncol 2010;28:3799-800.

5. Wang-Gillam A, Williams K, Novello S, et al. Time to Activate Lung Cancer Clinical Trials and Patient Enrollment: A Representative Comparison Study Between Two Academic Centers Across the Atlantic. J Clin Oncol 2010; 28: 3803-7.

6. Institute of Medicine (US) Forum on Drug Discovery, Development, and Translation. Transforming Clinical Research in the United States: Challenges and Opportunities: Workshop summary. Washington (DC): National Academies Press; 2010.

7. Norris KP. The Accuracy of Project Cost and Duration Estimates in Industrial R&D. R&D Management 1971; 2: 25-36.

8. Roy MM, Christenfeld NJ, McKenzie CR. Underestimating the Duration of Future Events: Memory Incorrectly Used or Memory Bias? Psychol Bull 2005; 131: 738-56.

9. Jorgensen M, Grimstad S. Software Development Estimation Biases: The Role of Interdependence. Software Engineering, IEEE Transactions 2012; 38: 677-93.

10. Kutsch E, Maylor H, Weyer B, et al. Performers, Trackers, Lemmings and the Lost: Sustained False Optimism in Forecasting Project Outcomes — Evidence from a Quasi-Experiment. Int J Project Management 2011; 29: 1070-81.

11. Sanna LJ, Schwarz N. Integrating Temporal Biases: The Interplay of Focal Thoughts and Accessibility Experiences. Psychol Sci 2004; 15: 474-81.

12. Jorgensen M. Identification of More Risks can Lead to Increased Over-Optimism of and Over-Confidence in software development effort estimates. Information and Software Technology 2010; 52: 506-16.

13. Sanna LJ, Parks CD, Chang EC, et al. The Hourglass is Half Full or Half Empty: Temporal Framing and the Group Planning Fallacy. Group Dynamics-Theory Research and Practice 2005; 9: 173-88.

14. Buehler R, Griffin D, Ross M. Exploring the Planning Fallacy—Why People Underestimate Their Task Completion Times J Pers Social Psychol 1994; 67: 366-81.

15. Halkjelsvik T, Jorgensen M. From Origami to Software Development: A Review of Studies on Judgment-based Predictions of Performance Time. Psychol Bull 2012; 138: 238-71.

16. Lamb SJ, Greenlick MR, McCarty D. Bridging the Gap Between Practice and Research: Forging Partnerships with Community-based Drug and Alcohol Treatment. Washington (DC): National Academy Press; 1998.