Scrutinizing Non-Core Protocol Procedures

The period from 2010 to 2020 may well come to be recognized as the decade of protocol design optimization. Having spent the past 20 years implementing strategies and tactics that have had a marginal-at-best impact on reducing study cycle time and costs, sponsors and CROs are now looking in earnest at protocol design optimization as the key to driving fundamental and sustainable operating efficiency and speed.

Kenneth A. Getz

Initiatives to optimize design and improve protocol feasibility include modifying authoring templates and inviting input from investigative sites and patients. Recent research from Tufts CSDD assists sponsors and CROs in targeting their focus on those procedures that are less essential to the objectives of the protocol and determining whether it is critical to perform them.

Protocol scope creep

Results of the 2012 study suggest that sponsors are spending more than $5 billion annually in direct costs to administer protocol procedures that are not tied to primary or key secondary endpoints and regulatory requirements. The incidence of these less essential or "non-core" procedures has nearly doubled during the past decade.

Non-core procedures are added to protocols for a variety of reasons: Clinical scientists and statisticians, for example, may want to collect more contextual data to help interpret study findings and guide development decisions. Context-setting variables may not appear in any statistical plan but they may provide clinical validation and explanation for unusual and unexpected results that may be observed during a clinical trial. Clinical scientists also collect additional study data hoping that, should the study fail to meet its original objectives, post-hoc analyses might reveal useful new insights into the characteristics and treatment of disease.

The presence of non-core protocol procedures may also be a function of authoring processes and practices and as an insurance policy against risk. Medical and protocol writing professionals may permit outdated and unnecessary procedures into new protocols because they are routinely included in legacy authoring templates and policies. Clinical teams collect additional data in cautious anticipation of requests for data from regulatory agencies, purchasers, and payers.

Study methods

Medidata Solutions sponsored the research study and 15 mid-sized and large pharmaceutical and biotechnology companies participated. Each company provided data on approximately 10 Phase II and III protocols targeting diseases across multiple therapeutic areas and executed by investigative sites dispersed globally since 2009. To minimize unusual and atypical designs, pediatric, medical device, orphan drug, and extension studies were excluded from the sampling frame. In all, 116 unique Phase II and III protocols having at least one procedure tied to a primary endpoint were analyzed. Participating companies classified each protocol procedure according to the objective and endpoint it supported as defined by the clinical study report (CSR) and the study's specific statistical analysis plan (SAP). In total, 25,103 procedures were classified along the following lines:

• "Core" procedures. Those that support primary and/or secondary study objectives, or primary or key secondary and safety endpoints.

• "Required" procedures. Those that support screening requirements and compliance-related activity including drug dispensing, informed consent form review, and study drug return.

• "Standard" procedures. Those that are commonly performed during initial and routine study participant visits including medical history, height and weight measurement, adverse event assessment, and concomitant medication review.

• "Non-Core" procedures. Those that support supplemental secondary, tertiary and exploratory endpoints, and safety and efficacy procedures not associated with a study endpoint or objective.

The direct cost to implement each protocol procedure was also analyzed using Medidata Solutions PICAS® database. Direct cost data for 16,607 procedures was analyzed.

Figure 1. Incidence of non-core procedures overall and by phase and

Incidence and direct cost

Characteristics of the protocols analyzed in this study—i.e., number of countries, sites and patients; total number of procedures and eligibility criteria—were consistent with industry benchmarks. Overall, half of the total procedures per protocol were classified as "Core" to the study. These procedures supported primary or key secondary endpoints. More than one out of every five procedures (22.3%) overall were "Non-Core" as they supported supplemental secondary, tertiary, and exploratory endpoints. One out of every four (24.7%) procedures performed per Phase III protocol and 17.9% of all Phase II procedures were classified as "Non-Core." "Core" procedures made up approximately half of all procedures by phase—47.9% of Phase III and 54.3% of Phase II studies. Variability across therapeutic areas was observed with endocrine protocols having the highest incidence of non-core procedures.

The distribution of direct costs was similar to that of procedure count. Overall, an average of $2.9 million of the total direct procedure costs per study—47.9%—was spent to administer "Core" procedures. An average of $1.1 million per protocol (17.9%) was spent to cover the direct cost of performing "Non-Core" procedures. The direct cost to administer "Required" and "Standard" procedures for Phase II and III protocols cost an average of $1.3 million (21.7%) and $.8 million (12.5%) respectively.

For Phase III protocols, the average total direct cost to administer all procedures was $9.4 million. Approximately half of the total direct cost (46.0% or $4.3 million on average) was spent to administer "Core" procedures; 18.5% or $1.7 million on average was spent to administer "Non-Core" procedures. The direct cost to administer procedures supporting screening requirements and regulatory compliance were $2.2 million or 24% of the total. "Non-Core" procedure administration costs were on average $.3 million or 13.1% of the total direct costs for Phase II protocols.

Discussion

Sponsors and CROs have long noted the expansive and increasing scope of their study designs and the rapid growth in the amount of data collected and analyzed per protocol. This recent Tufts CSDD study, however, provides hard metrics quantifying the incidence and direct cost of non-core procedures—a major source of expanding protocol scope. The marginal cost of including a single non-core procedure may be very small relative to the overall total study budget. But in the aggregate, non-core procedures consume 20% of the entire study budget.

Given that an estimated 2,578 Phase II programs and 1,079 Phase III programs were active worldwide last year, the total direct cost of collecting data from non-core procedures supporting these programs in 2012 was an estimated $4 billion. This is a very conservative estimate as it only counts an investigational drug in clinical development once, despite the fact that many drugs are in active clinical trials for multiple indications. The estimate also only counts one clinical trial per active compound per phase, when multiple trials are often conducted simultaneously.

These direct cost estimates also do not include any of the costs associated with having personnel capture, monitor, clean, analyze, manage, and store tertiary and exploratory procedure data. Indirect costs may be four to six times higher than the direct costs. The Tufts CSDD study also did not attempt to estimate the ethical costs of exposing study volunteers to unnecessary risks associated with conducting non-core procedures.

The primary takeaway of this study is not that clinical trials are a waste of resources as was suggested by a journalist reporting on a presentation of the results that I gave several months ago. The key message is that study performance and efficiency are highly associated with protocol feasibility. Any and all attempts to simplify protocol design and reduce complexity will reduce cost, shorten cycle time, and improve patient recruitment and retention effectiveness.

Non-core procedures are the place to begin. These procedures should be more carefully scrutinized and the trade-off between their benefits and cost assessed. As part of that assessment, sponsors and CROs can determine whether to delay or remove non-core procedures if the cost of doing so outweighs their benefit.

The recent Tufts CSDD study provides a framework for sponsors and CROs to identify and challenge non-core procedures. This framework may prove invaluable in helping sponsors prioritize and redirect scarce resources and capital. As we enter the decade of protocol design optimization, more prudent and active evaluation of non-core procedures is an unusual win-win opportunity for sponsors and CROs to dramatically improve performance and efficiency while lowering costs.

Kenneth A. Getz MBA, is the Director of Sponsored Research at the Tufts CSDD in Boston, MA, e-mail: kenneth.getz@tufts.edu.