Analytics and Metrics Help Pinpoint Costs of Study Start-Up

The clinical trials sector has done a good job tracking costs associated with study execution, but costs linked to study start-up are conspicuously absent. A quick look at industry research confirms this gaping hole in cost accountability. For Phase III clinical trials, for example, it is widely recognized that total costs are substantial, ranging from an estimated $11.5 million to $52.9 million, depending on therapeutic indication.¹ Upon closer examination, it is apparent that identified costs are almost exclusively tied to study execution, and rarely include details on the hefty costs incurred early on, namely those tied to study start-up and site overhead. In fact, data from an often-quoted 2014 study published by the Department of Health and Human Services (HHS) indicate that unaccounted for costs may represent upwards of 43% of the total cost of conducting a clinical trial.

This statistic is calculated from the HHS study, which focused on numerous cost drivers in each phase of clinical trials and the percentage they represent of total costs.² For Phase III studies, among the many known costs, the top three were:

• Administrative staff (11.7% of total costs)

• Clinical procedure costs, such as electronic data capture (11.3%)

• Site monitoring costs, such as risk-based monitoring (8.2%)

Collectively, these top-known costs, plus all of the other known factors, rang it at $11.38 million, yet the average of all Phase III costs totaled $19.89 million. It is concerning that the large $8.51 million difference between these two figures lacks detail and remains unaccounted for.

This lack of accountability is indicative of how the industry in general underestimates the activities associated with the initiation of clinical trials. The Trial Master File (TMF) Reference Model,³ a DIA document and records management community project, corroborates the significance of study start-up activities. It categorizes 53% of artifacts that ultimately end up in the TMF as being associated with study start-up, including ongoing maintenance activities (e.g., protocol and contract amendments, principal investigator [PI] changes and related submissions, etc.).

How is this possible?

It can be hypothesized that the undefined study cost elements in the HHS study are caused by an industry largely tracking significant elements (i.e., operational and regulatory) of a study in paper and unsophisticated spreadsheets and not dedicated systems/databases. In fact, the tracking of regulatory elements truly begins at investigational product release, at which point all systems/databases are traditionally implemented, hence there is cost-associated data for these elements. This article explores the factors that make up unaccounted expenses, especially those tied to the many complex steps that define study start-up, such as protocol writing, site identification, selection, and activation, budgets and contracts, institutional review board (IRB) and ethical reviews, and more. As part of this effort, the following issues are considered:

• How a workflow-based system helps gather quality data and metadata from multiple sources to help track costs and improve study start-up operations

• Why it is critical to consider using a parallel approach to the multi-step study start-up process to cut timelines and costs

By embracing a systematic, data-driven approach, it is possible for metrics to identify more accurately the best sites, steps causing delays, the associated costs, and why this is happening.

Effective study start-up

An effective study start-up process is fundamental to the overall operational success of a clinical trial, as it is the most impactful phase of the trial’s lifecycle. The impact comes from a thoughtful upstream study start-up approach that influences downstream performance through the use of purpose-built workflows. Essentially, workflows streamline and organize operational aspects in real-time, enable the collection of data and metadata, help identify problems quickly, and allow detailed study start-up costs to be calculated.

This represents a sea change, as the HHS study shows study start-up costs as vague and incomplete, basically reflecting that, traditionally, these costs have not been clearly identified. Instead, they have been overlooked or lumped into a handful of categories, such as the cost of IRB approvals, site recruitment and retention costs, and patient recruitment costs. As a result, critical multi-step tasks ranging from pre-study visits through to contract or budget execution have relatively unknown costs, although they embody almost 60% of study start-up cycle time.⁴

In the DHHS study, the largest category is “Site Overhead,” which lacks detail entirely, but is still estimated as responsible for nearly $2.6 million of the $19.89 million in Phase III study costs. Similarly, the “All Other Costs” category is also undefined yet is associated with almost $6 million in Phase III expenditures. Moreover, the report states that these two large expenses were determined by extrapolation from cost components with available estimates-not task-specific data (see Table 1 below).

A good example of how workflow technology can bring greater precision to study start-up costs includes taking a look at how it can impact tightly guarded siloes. Typically, stakeholders rarely interact with others along the clinical trial continuum, so they are often unaware of which delays are throwing the timeline off course or what the associated costs might be. With workflows, however, pinpointing tasks need to be completed and when, and by whom, can be more easily achieved, improving the likelihood of the study’s timeline staying on track.

This approach is a major departure from the still-too-common department-to-department handshakes and continued reliance on departmental metrics meant to evaluate performance while generally lacking the ability to measure costs and activity at the granular level. For instance, attempting to measure “time to execute all site-level contracts” is not an accurate metric unless it is based on its sub-steps, all of which play a role. These include:⁵

• When site contract language is ready

• When first contract language package is sent

• When last contract language is ready for execution

• Completion of regulatory submissions and approvals

• When all contracts are executed

There are additional benefits to a workflow approach. It optimizes the handling of amendments, which determines their subsequent impact on study start-up functions, such as site activation and study execution. Research from the Tufts Center for the Study of Drug Development (CSDD) indicates why the limiting of amendments and their proper management are vital actions. More than half of all protocols, 57%, across all phases, have at least one substantial amendment, which results in a study taking three months longer to complete, on average, as compared to studies without amendments.⁶ Moreover, approximately 40% of all amendments occur before the first study volunteer receives the first dose in the clinical trial.⁷

It is noteworthy that workflow solutions align with the 2016 Good Clinical Practice (GCP) guideline from the
International Conference on Harmonization [ICH-GCP E(6)R2)], which calls for a heavy emphasis on quality from the beginning of a study, and extending it throughout the life cycle.⁸ Specifically, the guideline has an entire

section dedicated to quality management, which states that to achieve quality, efficient tools are essential for proper data collection and decision-making (see box on right).

This regulatory input and the impact of workflow-based tools are driving significant improvements to how study progress is often tracked, namely Excel spreadsheets, Word documents, and PDFs that generally are department-specific and lack access to upstream and downstream information. As a result, reliable metrics are usually difficult to calculate, and stakeholders cannot easily track costs or study status, or make improvements.

Working in parallel

Taking the steps to improve data collection from study start-up activities means figuring the best way to maximize the impact of workflows. Certainly, implementing workflows drives efficiencies by automating manual processes, but significant reductions in timelines and cost require more-they require changing the underlying processes. For starters, this may involve switching to working in parallel, instead of continuing with the traditional sequential method of study conduct. In particular, working to evaluate sites, then selecting and activating them, are activities that can take place in parallel.

The parallel approach has become somewhat of a battle cry in recent years, as industries from computer science to manufacturing to chemical synthesis have adopted it to shorten timelines. A simple Google search reveals a cadre of articles describing use of parallel technique dating back 20 years, but for the life sciences sector, it is a fairly new endeavor. A 2004 article in Applied Clinical Trials comments on the then-growing use of electronic data capture (EDC), which would change the trial process from a series of sequential tasks to a parallel process, enabling continual review of data to improve data quality.⁹ Other articles describe conducting Phase I and Phase II trials in parallel to evaluate the safety and efficacy of combination dose levels simultaneously, to more quickly select the optimal combination dose.¹⁰

What role does a parallel technique play in creating a better study start-up experience? A system that will help identify high-quality sites that are most likely to meet enrollment targets and ensure adherence to GCP is one area. Research suggests that for site identification, there is no single source of data used to find the right sites, and the approaches are not evidence-based. Consequently, it is hardly surprising that 11% of selected sites are never activated, a statistic that has not changed meaningfully in more than a decade.¹¹

With the help of data analytics, sponsors and contract research organizations (CROs) can do more than identify the best sites. They can activate them quickly, allowing them to start patient recruitment activities, while continuing to identify sites in parallel.

The need for parallel processes is evident. A survey conducted in 2017 by the Tufts CSDD, with 591 respondents, found that average cycle time for study start-up is quite lengthy, averaging more than 36 weeks (~ 8.4 months) for new sites and 26 weeks (~6 months) for existing ones (see Figure 1).¹⁰ This timeframe poses challenges. In particular, with a sequential approach, the “Sites Selected” list is typically finalized months before any sites are activated. And for sites that have identified patients as part of the feasibility process, expecting those patients to wait months to enroll is a poor management strategy.

A parallel approach allows those sites that have already completed a confidentiality disclosure agreement, and are ready to recruit and enroll, to actually do so. Using a system to collect and analyze study start-up data, this approach can enable site feasibility and activation to take place in parallel. This also can help ensure that patients can be found and enrolled much closer to release of the investigational product to the site (see Figure 2).

A systems approach

The clinical trials sector has invested heavily in recent years in cloud-based tools for study execution, such as EDC, clinical trial management system (CTMS), risk-based monitoring initiatives, interactive response technology, and more. As a result, there are more opportunities for parallel operations associated with these established solutions, given the existence of metrics-related data and good cost information, unlike the vague cost landscape that has typified study start-up.

Part of the challenge in improving operations stems from the fact that until recently, there have been no systems to enter study start-up performance data, so it has been quite difficult for companies to perform a data-driven analysis to identify high-quality sites most likely to meet enrollment targets. Study start-up workflow systems with open application program interfaces (APIs) can exchange data from tools such as EDC, CTMS, and others, allowing data to flow among them. This enables stakeholders, including sites, to enter data that can eventually be used for performance analysis. With the help of these systems, a sponsor or CRO can identify sites, and continue identifying them until analytics indicate with 90%–95% probability that they will meet the enrollment target. As discussed, those sites that are ready to activate can do so. This creates an environment in which sites can compete to be selected and compete to reach enrollment targets.

Figure 3 depicts how this end-to-end workflow system works, using a parallel model.

This analytical approach is more exacting than defaulting to arbitrary milestones based on sequential methods that not only lack the data to find the best sites, but also cannot indicate when sponsors or CROs should stop identifying sites. There are available databases of site information, but those data do not necessarily lead to use of parallel process. A better approach for stakeholders looking to start a study is to review the protocol, and then use a parallel-based systemic approach to generate data and metadata specific to that study.

Better understanding of costs

With nearly half of clinical trial costs unaccounted for, including known administrative costs, and much of those linked to study start-up, it is telling that the industry is finally moving toward identifying those costs with the help of purpose-built tools that can determine, with a high degree of accuracy, when it is time to stop identifying sites. The vast wealth of data that reside in technologies designed for study execution, such as EDC, CTMS, and more, can be integrated with workflow-driven solutions for study start-up, allowing stakeholders to gain insight at a granular level into how a study process is unfolding and the associated costs.

The parallel method, using data and applying data analytics, could possibly shave up to two months from the timeline. This change in approach will continue to highlight how improving the study start-up process is a great opportunity, yielding a major impact on quality, timeline, and overall cost of clinical trials.

References

1. Sertkaya A, Wong H-H, Jessup A, Beleche T. Key cost drivers of pharmaceutical clinical trials in the United States. Clinical Trials. 2016. 13(2):117-126. http://journals.sagepub.com/doi/pdf/10.1177/1740774515625964. Accessed May 4, 2018.
2. Sertkaya A, Birkenbach A, Berlind A, et al. Examination of clinical trial costs and barriers for drug development. US Department of Health & Human Services. July 2014. https://aspe.hhs.gov/report/examination-clinical-trial-costs-and-barriers-drug-development. Accessed May 4, 2018.
3. Trial Master File Reference Model, https://tmfrefmodel.com/
4. Lamberti MJ, Brothers C, Manak D, Getz KA. Benchmarking the study initiation process. Therapeutic Innovation & Regulatory Science. 2013;47(1):101-9.
5. The gateway to study startup (SSU) optimization: Performance metrics. goBalto. May 2017. https://www.gobalto.com/whitepaper_study_startup_optimization_performance_metrics. Accessed June 5, 2018.
6. Peters S, Lowy P. Protocol amendments improve elements of clinical trial feasibility, but at high economic and cycle time cost. Press Release. Tufts Center for the Study of Drug Development. January 14, 2016. http://www.marketwired.com/press-release/protocol-amendments-improve-elements-clinical-trial-feasibility-but-high-economic-cycle-2088135.htm. Accessed June 1, 2018.
7. Protocol amendments: A costly solution. Applied Clinical Trials. 2011. http://www.appliedclinicaltrialsonline.com/protocol-amendments-costly-solution. Accessed June 7, 2018.
8. Integrated Addendum to ICH E6(R1): Guideline for Good Clinical Practice E6(R2). ICH Harmonized Guideline. November 2016. https://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6/E6_R2__Step_4_2016_1109.pdf. Accessed June 6, 2018.
9. Transforming EDC – The emerging eR&D model. Applied Clinical Trials. 2004. http://www.appliedclinicaltrialsonline.com/transforming-edc-emerging-erd-model. Accessed June 6, 2018.
10. Huang X, Biswas S, Oki Y, Issa JP, et al. A parallel phase I/II clinical trial design for combination therapies. Biometrics. 2007;63(2):429-36.
11. Harper B, Wilkinson M, Lamberti MJ, Getz KA. Tufts CSDD-goBalto Site Selection Market Research (START II). June 30, 2017. https://www.gobalto.com/research-report-tufts-2017-full-report. Accessed July 5, 2018.

Craig Morgan, Head of Marketing, Study Start Up, Oracle Health Sciences