Closing in on Opportunities to Simplify Protocol Design

Sponsor companies face intense pressure to deliver higher levels of efficiency and drug development performance. A growing number of sponsors are now acting on the belief that improvements in protocol design feasibility hold the key to addressing and easing some of these pressures. Although changing legacy protocol design practices and operating procedures is extremely difficult, sponsor companies are implementing new mechanisms and approaches- beginning with a focus on reducing non-core procedures, protocol amendments, and excessively administered core procedures.

Kenneth A. Getz MBA

A recent 2014 study by the Tufts Center for the Study of Drug Development (CSDD) provides an update on the prevalence of non-core procedures and sheds new light on their causes. Key takeaways include:

• Sponsors report that a higher percentage of protocol procedures now support supplemental, tertiary, and exploratory endpoints compared to that reported in 2012.

• A closer look at non-core procedures shows that a high percentage is associated with safety and efficacy endpoints.

• Most non-core data is source data verified and included in regulatory submissions.

About the 2014 study

This follow-up study sought to understand more about the purpose of non-core procedures, their characteristics, and how non-core data is used. Eight sponsor companies participated, each providing approximately 15 protocols. In total, 137 unique Phase II and III protocols conducted since 2009 and having at least one procedure tied to a primary endpoint were analyzed. The protocols targeted diseases across multiple therapeutic areas and were executed by investigative sites dispersed globally. To minimize unusual and atypical designs, pediatric, medical device, orphan drug, and extension studies were excluded from the sample. The scope and characteristics of protocols analyzed in this study were generally consistent with industry benchmarks (i.e., number of countries, sites, and patients; total number of procedures and eligibility criteria).

Medidata Solutions not only sponsored the 2012 study but also sponsored this subsequent effort and provided a custom e-solution to collect each company's data and to tie protocol procedures to their direct costs. Participating companies classified a total of 25,287 procedures according to the objective and endpoint that each supported as defined by the clinical study report (CSR) and the study's statistical analysis plan (SAP).

The procedure classification scheme used in this study was identical to that used in the original 2012 study:

1. "Core" procedures-are those that support primary and/or secondary study objectives or primary or key secondary and safety endpoints.

2. "Required" procedures-are those that support screening requirements and compliance-related activity, including drug dispensing, informed consent form review, and study drug return.

3. "Standard" procedures-are those that are commonly performed during initial and routine study participant visits, including medical history, height and weight measurement, adverse event assessment, and concomitant medication review.

4. "Non-core" procedures-are those that support supplemental secondary, tertiary, and exploratory endpoints, and safety and efficacy procedures not associated with a study endpoint or objective.

Characterizing non-core procedures

Nearly one-third of all Phase III protocol procedures (30.6%) supported non-core endpoints and objectives, up from 24.7% observed in the earlier study. Nearly one in five Phase II procedures (20.7%) supported non-core endpoints and objectives, approximately three percentage points higher than the 2012 study level. The proportion of procedures supporting core endpoints also increased: from 54.3% to 64.9% of Phase II procedures, and from 47.9% to 58.6% of Phase III procedures. In the later study, a smaller relative proportion of Phase II and Phase III procedures were classified as standard and required. Similar to the 2012 study, variability across therapeutic areas was observed, with protocols targeting endocrine and central nervous system (CNS) disorders having the highest incidence of non-core procedures.

Procedures supporting non-core endpoints were evenly distributed across the entire schedule of assessments. Twenty-two percent and 32.6% of all non-core procedures in Phase II and III studies, respectively, supported safety endpoints. And 13.7% and 21.6% of all non-core procedures in Phase II and III trials supported efficacy endpoints. Less than 20% of all procedures that collected non-core data supported outcomes-related endpoints, such as quality of life assessments and reimbursement. Only 3% of non-core procedures collected biomarker data supporting regulatory filings.

Costs of collecting non-core data

The proportion of each study budget supporting the direct cost of administering non-core procedures also increased. Approximately one-quarter (23%) of each Phase II and III study budget covered the direct cost of administering non-core procedures. This proportion is up from an estimated 18% in the 2012 study.

The direct cost proportion of the study budget is only part of the total cost picture. The in-direct costs (e.g., the cost of collecting, monitoring, cleaning, analyzing, managing, and storing non-core procedure data) must also be considered. Although organizations have difficulty measuring fully loaded in-direct costs accurately, participating companies looked at a number of factors to begin characterizing these in-direct costs.

This recent Tufts CSDD study found that 80% of all data from Phase II non-core procedures and 87% of all data from Phase III non-core procedures was source data verified by study monitors. And although non-core data does not support primary or key secondary endpoints, participating sponsor companies reported that nearly all of their non-core data was included in the clinical study report (92%) and in the tables, listings, and figures (95%) in the regulatory submission.

Simplification steps

Non-core procedures are added to protocols for a variety of reasons: Clinical teams and statisticians often want to collect more contextual data to help interpret study findings and guide development decisions. These context-setting variables provide clinical validation and explanation for unusual and unexpected results that may be observed during a clinical trial. Clinical teams also collect additional study data hoping that, should the study fail to meet its original objectives, post-hoc analyses might reveal useful new insights into the characteristics and treatment of disease. Most companies can point to exploratory data that led to the discovery of a novel therapy.

But a growing number of sponsor companies now admit that the presence of non-core protocol procedures may also be due to habit and risk-avoidance. Professionals involved with protocol authoring may permit outdated and unnecessary procedures into new protocols because they are routinely included and copied from old authoring templates and policies. And clinical teams are collecting additional data in cautious anticipation of requests from regulatory agencies, health authorities, purchasers, and payers.

The higher reported percentage of protocol procedures now supporting supplemental, tertiary, and exploratory endpoints compared to that reported in 2012 is a curious finding. Sponsor companies may be continuing to add more non-core procedures to their study designs. The higher reported proportion may also be due to greater awareness and more aggressive identification of non-core procedures as a growing number of companies look for ways to simplify their protocol designs and improve study feasibility.

Indeed, a separate 2014 Tufts CSDD study found that a number of companies are using new protocol authoring techniques that include more robust evaluations (e.g., SPIRIT authoring checklist and the Metrics Champion Consortium Protocol Quality Scoring Tool) to better tie procedures to core endpoints and objectives.

Most major and mid-sized pharmaceutical and biotechnology companies are now taking formal steps to routinely examine the operational feasibility of their study designs. Many organizations have established feasibility review committees to challenge non-core and excessively conducted core procedures. Reports to date on the effectiveness and impact of these committees are very positive: 93% of companies report that their feasibility review committees are "somewhat" or "very" effective. And a high percentage of companies with established committees report that they have seen fewer protocol amendments (68%); a reduction in investigative site burden to administer the protocol (53%); and faster study cycle times (44%).

Non-core procedures represent an important area of initial focus for companies seeking opportunities to optimize study design. Non-core procedures should be more carefully scrutinized and the trade-off between their benefits and cost assessed. As part of that assessment, sponsors and CROs can determine whether to delay or remove non-core procedures if the cost of doing so outweighs their benefit. The recent Tufts CSDD study provides further insight into the purposes of, and practices associated with, procedures supporting non-core endpoints and objectives. The marginal cost of including a single non-core procedure may be very small relative to the overall total study budget. But in the aggregate, non-core procedures consume up to a third of the total direct procedure cost in a Phase III study budget, and many magnitudes more in in-direct study costs.

Kenneth A. Getz MBA, is the Director of Sponsored Research at the Tufts CSDD and Chairman of CISCRP, both in Boston, MA, e-mail: kenneth.getz@tufts.edu