Method for Efficient Pre-implementation of Multi-site Pharmacological Trials

Increasing the speed with which promising pharmacological treatments move from pre-clinical research to patient-care utilization has received increasing focus in recent years¹ and is a critical goal of the National Center for Advancing Translational Sciences (NCATS), an NIH institute established in 2011.² Some streamlining efforts focus on issues of great scientific appeal, such as the development of chips mimicking human tissues to be used to screen for toxic drug effects. Other efforts are focused on more mundane, yet important, issues such as ensuring that investigators have needed access to regulatory support and data management tools.³ Achieving more efficient translational research will entail reducing the time and expense required to conduct clinical trials in the United States.^4,5 A report from the Institute of Medicine in 2010 delineated several key challenges to the efficient completion of multi-site clinical trials.⁶ This paper addresses one of the issues raised, which is the relatively long time that the pre-implementation phase, the phase between sponsor approval of a protocol and initiation of recruitment, can take to complete.

A long pre-implementation phase is problematic in that it can significantly increase the costs of conducting the trial and reduce the speed with which findings are available, and hence, their potential relevance.⁶ The pre-implementation phase for multi-site clinical trials is complex and typically entails coordination among multiple institutions and among individuals with varying areas of expertise. There can, thus, be a number of potential reasons for a lengthy pre-implementation period. For example, the time required to obtain all regulatory approvals can significantly prolong the pre-implementation phase.⁵ There are three primary causes of prolonged pre-implementation periods, only one of which, the occurrence of unforeseeable events, cannot be avoided. A second cause is the human tendency to underestimate the amount of time a project will take to complete, which is a well-documented human bias; indeed, understanding the factors that contribute to the bias, and potential methods for countering them, has received a good deal of study.^{7,8,9,10,11,12,13,14,15} A third cause is problematic coordination among the entities that play a key role in the clinical trial. This paper describes the Pre-implementation Timeline Calculator, which is a tool designed to help investigators plan and coordinate the completion of the pre-implementation phase in a timely manner and presents data suggesting that the Calculator may be helpful in reducing the time and cost needed to complete the pre-implementation phase.

Context

The Pre-implementation Timeline Calculator was developed in working with the National Drug Abuse Treatment Clinical Trials Network (NIDA CTN). The CTN was established in 1999 in response to an Institute of Medicine report delineating the relative lack of information exchange between substance abuse researchers and substance abuse community treatment providers.¹⁶ In the CTN, substance abuse researchers work with treatment providers to design clinically meaningful trials and implement them in “real-world” settings utilizing existing staff as interventionists and clinic patients as participants. The CTN links substance abuse researchers and treatment providers through nodes, with each node consisting of a Regional Research and Training Center (RRTC), which is affiliated with a university, and multiple Community Treatment Programs (CTPs) that work with the RRTC. Oversight of the CTN is provided by the Center for the Clinical Trials Network (CCTN). The CCTN contracts with a Clinical Coordinating Center (CCC) to provide regulatory guidance, quality assurance monitoring, central pharmacy and laboratory services, and study supplies. The CCTN also contracts with a centralized Data and Statistics center (DSC), which provides statistical support and data management for CTN trials. Study oversight is provided by the RRTC of each participating CTP and by the study's Executive Committee, which is comprised of the lead investigator (LI), members of the LI's team, and representatives from the CCTN, CCC, and the DSC. 

The Pre-implementation Timeline Calculator

Overview. To help manage, and successfully complete, pre-implementation in a timely manner, the Pre-implementation Timeline Calculator was developed for pharmacological multi-site clinical trials, which can be seen in Figure 1. The Calculator divides the pre-implementation stage into six general tasks.

Information about the columns comprising the Calculator is provided in Table 1. The Calculator itself is a Microsoft Excel worksheet and is available for free at the CTN Dissemination website (http://ctndisseminationlibrary.org/resourcespolicies.htm). As displayed in Figure 1, cells with dotted borders require direct entry of the actual date associated with the task being referenced. Most cells in the Calculator are pre-filled with formulas and locked; time-to-completion targets may be adjusted as needed by changing the value in the associated cell. Based on the date entered for "sponsor approval of protocol," recommended dates for target site initiation and the Investigators' Meeting are generated.  As noted earlier, a long pre-implementation phase is problematic.⁶ For CTN trials, the target is to initiate recruitment eight months after receiving sponsor approval of the study protocol. While the goal is to complete the overall pre-implementation phase as efficiently as possible, particular attention is given to minimizing the time between completing staff training and initiating participant recruitment since staffing costs constitute the majority of costs for a clinical trial; thus, it is important to minimize staffing costs at sites that are not open for enrollment. The target of holding the Investigator's meeting one month prior to site initiation is based on the time generally needed to complete site preparation and quality assurance (QA) visits following the meeting and prior to site initiation.

Instructions for using the Calculator.Utilization of the Pre-implementation Timeline Calculator to establish the pre-implementation timeline for the trial entails three steps:

• The investigator enters dates for sponsor approval of protocol, target for site initiation, and investigators' meeting.

• The investigator and his/her team review each task and determine if there are any tasks that are not relevant for the trial or which were completed prior to the sponsor approving the protocol. For these tasks, a dummy date is entered for "Actual Date Completed." For site selection tasks, the "dummy" date should be the date that the sponsor approved the study protocol. For all other tasks, the dummy date completed should be the "Target Date of Completion" generated when the three dates listed in step 1 were entered.

• The investigator and his/her team determine if the "Estimated Time to Complete" needs to be adjusted for any of the tasks and adjusts the estimated number of weeks to complete as necessary. The appearance of an "X" in the "Timeline Problem" column indicates that site initiation eight months following sponsor approval may not be feasible and a later date for "Target Date for Site Initiation" should be entered.

Commitment to the timeline should be obtained from all parties playing a key role in the pre-implementation phase (e.g., investigator, data management center, parties responsible for securing study medication and supplies, QA monitoring, etc.). The Investigators' Meeting should then be scheduled, avoiding both holidays and relevant professional conferences, to ensure the availability of the venue for the meeting. Since the goal is to have one month between the Investigators' Meeting and site initiation, a later-than-intended meeting would lead to a later target date for site initiation being entered. At the end of this process, the pre-implementation timeline for the trial has been established and should be shared with all parties playing a key role in pre-implementation, including the study sites, once they have been selected. The provision of the timeline allows each entity time to complete their piece according to schedule.

General pre-implementation tasks. The following is a detailed description of the six general tasks that encompass the pre-implementation stage:

• Site selection

As noted in Figure 1, the goal is to have the sites selected at least one month prior to the expected initiation of regulatory activities. Many clinical trials require that an institutional review board (IRB) submission be completed at the level of the site and many sites will require that a contract/subaward be established before any work, including preparing IRB submissions, can commence; allowing a month for contract/subaward establishment can help avoid delays in initiating needed regulatory activities. Since the Calculator was created for use in the CTN, a network in which information about candidate sites, including prior performance in clinical trials, typically is readily available, the Calculator reflects the time needed to obtain a completed study questionnaire from sites and to conduct telephone interviews with promising sites. When the Calculator is used for trials requiring site visits for site selection, the “Estimated Time to Complete" needs to be increased for the "Interview Sites" task.

• Regulatory approvals

As noted earlier, obtaining all needed regulatory approvals can require a substantial amount of time. In the case of U.S. pharmacological trials, this will include obtaining an investigational new drug (IND) or an IND waiver from the Food and Drug Administration (FDA). For many multi-site clinical trials, it will also necessitate obtaining approvals from multiple IRBs. As noted in Figure 1, the goal should be to have all regulatory approvals obtained prior to the Investigator's Meeting so that regulatory-related delays in initiating recruitment can be avoided.

• Data Management

Data management system development often takes significantly more time than expected by clinical trialists. The timeline for creating the data management system outlined in Figure 1 is based on the time required for past multi-site clinical trials. As can be seen in the example provided in Figure 1, paper copies of the approved case report forms (CRFs) needed to be provided to the data management center in mid-to-late June of 2013 in order for the system to be ready before the Investigator's Meeting in early January 2014.

• Study materials, medications, supplies

The target dates for completing the operations manual and having the study medication and supplies on site are provided in Figure 1. A thorough, user-friendly operations manual is an important piece of a successful multi-site clinical trial and generally should be completed as early in the pre-implementation process as possible since its completion can delineate feasibility issues with the study protocol that should be corrected prior to IRB submission. Clear delineation of responsibility for obtaining study medication and supplies, and securing commitment to meeting the timeline needs to be completed as part of determining the pre-implementation timeline for the trial.

•  Site staffing

As noted, staffing constitutes a significant portion of the cost of conducting clinical trials and, thus, the site study staff should be hired so that the time elapsing between hiring and initiation of participant recruitment is the amount of time needed to complete training and site preparation. As can be seen in Figure 1, the research assistants (RAs) typically are hired earlier than the remaining staff, which reflects the additional training requirements for the RAs. The other study staff can be hired approximately two weeks before the Investigators' Meeting, which allows enough time to make travel arrangements to attend the meeting. As noted previously, the pre-implementation timeline should be provided to the study sites once they have been selected, which enables the sites to plan for staff hiring and for the Investigators' Meeting. In particular, the amount of time required to hire staff can vary dramatically from very little time for sites moving study staff from one trial to the next to several months for sites that need to establish a new position and complete the hiring process. 

•  Investigator's meeting and site initiation

As noted above, the goal is to hold the Investigator's Meeting approximately one month prior to the target date for initiating participant recruitment.

Use of the Calculator to condense the pre-implementation phase

As discusses earlier, a lengthy pre-implementation phase is problematic. Of particular importance from a budgetary perspective, the lengthening of the pre-implementation period can result in staff being hired and the Investigators' Meeting being held without the trial being ready to start (e.g., the data management system is not complete, needed study materials have not been finalized, etc.). Evaluating whether the Pre-implementation Timeline Calculator reduces pre-implementation phase length would require that clinical trials be randomly assigned to Calculator use or to a control condition, which has not been done. However, preliminary data on the Calculator's potential to reduce the length of the pre-implementation phase is available in that it has been used for several multi-site clinical trials, including three pharmacological trials; these trials are referred to as Calculator trials.

The Calculator trials were all CTN trials initiated between 2004 and 2012. The pre-implementation period for one of the trials was nearly complete prior to the CCC being established, while the other two utilized the same CCC. Two trials were managed by the same DSC, while the third was managed by a different DSC. Trials with which to compare the Calculator trials were samples of convenience — substance abuse pharmacological trials conducted during approximately the same period as the Calculator trials for which information about the pre-implementation phase length was available. The Non-Calculator trials are four pharmacological substance abuse trials initiated between 2003 and 2011, which include both pharmaceutical and NIDA (including CTN)-sponsored trials. These were the only pharmacological substance abuse trials for which the needed information was readily available. The time required for pre-implementation can vary according to several factors, including the complexity of the trial, and the amount of staff time devoted. Thus, to provide a more stable estimate of the pre-implementation timelines between the Calculator and Non-Calculator trials, average times for each trial type are provided.

Figure 2 provides the average times taken for two segments of the pre-implementation phase: The period between having a sponsor-approved protocol and the initiation of recruitment at a study site and the period between the Investigator's Meeting and the initiation of recruitment at a study site. As can be seen, the Calculator trials were approximately six months shorter than the Non-Calculator trials for the first period and three months shorter for the second period. As noted above, the second period is particularly important from a budgetary perspective since this is a period in which the study staff have already been hired. To provide an estimate of how elongation of the second period can contribute to the cost of a clinical trial, the staffing cost associated with the additional three months was calculated for a hypothetical trial conducted at 10 sites with site staffing, as outlined in Table 2. On average, there were 48 days between the Investigators' Meeting and the initiation of recruitment for the Calculator trials and 145 days for the Non-Calculator trials. With a cost of $400 per day, per site for staff costs (see Table 2), the staffing costs for the Calculator trials would be $192,000 for 10 sites for the period between the Investigators' Meeting and the initiation of recruitment and would be $580,000 for the Non-Calculator trials, a cost difference of $388,000.

Conclusion

The goal of streamlining translational science, of which streamlining multi-site clinical trials is a component, is of critical importance.¹ Achieving this goal will entail the dedication of many investigators and will entail scientific breakthroughs, as well as attention to factors that, while mundane, can have a significant impact on the speed with which promising pharmacological treatments reach patient care.³ This paper addressed one challenge to the efficient completion of multi-site clinical trials, the relatively long period of time that it can take to complete the pre-implementation phase. It discussed the use of the Pre-implementation Timeline Calculator, which was developed to help investigators to plan for, and complete, the pre-implementation phase in a timely manner. Preliminary data suggest that using this tool may significantly reduce the time and financial resources required for pre-implementation. An important limitation of the present data is the reliance on a sample of convenience for the Calculator and Non-Calculator trials. However, investigators who have completed multi-site pharmacological trials have access to the length of the pre-implementation phase for their trials and, thus, can determine whether the use of a tool like the Pre-implementation Timeline Calculator may lead to time and cost savings. In general, given that the Calculator has no discernible disadvantages, its more widespread use in multi-site pharmacological clinical trials seems warranted.

Theresa Winhusen, PhD, Department of Psychiatry and Behavioral Neuroscience, University of Cincinnati College of Medicine, 3210 Jefferson Ave., Cincinnati, OH 45220,

[email protected]

References

1. Zerhouni EA. Translational and clinical science--time for a new vision. New Engl J Med 2005; 353: 1621-3.
2. Insel TR. Invited response to: Preparedness of the CTSA's structural and scientific assets to support the mission of the National Center for Advancing Translational Sciences (NCATS). Clin Transl Sci 2012; 5: 130-1.
3. CTSA Princiapal Investigators, Shamoon H, Center D, et al. Preparedness of the CTSA's structural and scientific assets to support the mission of the National Center for Advancing Translational Sciences (NCATS). Clin Transl Sci 2012; 5: 121-9.
4. Dilts DM. Early warning: an ailing canary in the mine. J Clin Oncol 2010;28:3799-800.
5. >Wang-Gillam A, Williams K, Novello S, et al. Time to activate lung cancer clinical trials and patient enrollment: a representative comparison study between two academic centers across the atlantic. J Clin Oncol 2010; 28: 3803-7.
6. Institute of Medicine (US) Forum on Drug Discovery, Development, and Translation. Transforming clinical research in the United States: challenges and opportunities: workshop summary. Washington (DC): National Academies Press; 2010.
7. Norris KP. The accuracy of project cost and duration estimates in industrial R&D. R&D Management 1971; 2: 25-36.
8. Roy MM, Christenfeld NJ, McKenzie CR. Underestimating the duration of future events: memory incorrectly used or memory bias? Psychol Bull 2005; 131: 738-56.
9. Jorgensen M, Grimstad S. Software Development Estimation Biases: The Role of Interdependence. Software Engineering, IEEE Transactions 2012; 38: 677-93.
10. Kutsch E, Maylor H, Weyer B, et al. Performers, trackers, lemmings and the lost: sustained false optimism in forecasting project outcomes — evidence from a quasi-experiment. Int J Project Management 2011; 29: 1070-81.
11. Sanna LJ, Schwarz N. Integrating temporal biases: the interplay of focal thoughts and accessibility experiences. Psychol Sci 2004; 15: 474-81.
12. Jorgensen M. Identification of more risks can lead to increased over-optimism of and over-confidence in software development effort estimates. Information and Software Technology 2010; 52: 506-16.
13. Sanna LJ, Parks CD, Chang EC, et al. The hourglass is half full or half empty: temporal framing and the group planning fallacy. Group Dynamics-Theory Research and Practice 2005; 9: 173-88.
14. Buehler R, Griffin D, Ross M. Exploring the planning fallacy - why people underestimate their task completion times J Pers Social Psychol 1994; 67: 366-81.
15. Halkjelsvik T, Jorgensen M. From origami to software development: a review of studies on judgment-based predictions of performance time. Psychol Bull 2012; 138: 238-71.
16. Lamb SJ, Greenlick MR, McCarty D. Bridging the gap between practice and research: forging partnerships with community-based drug and alcohol treatment. Washington (DC): National Academy Press; 1998.