Sponsors Ask: Is Our Trial Plan Realistic? Toward a Data-Driven Feasibility Assessment

Feasibility studies are remarkable for the degree to which they can reduce pharmaceutical companies’ risk in their research and development efforts. Clinical trial protocols have become more complex, with the typical Phase III protocol having 50 eligibility criteria in 2012 compared to 31 in 2002.¹ Further, improved feasibility is among the interventions that can positively reduce the risk associated with more complex clinical trials.¹ Today, a data-driven approach to feasibility assessment, which includes using electronic health records (EHR) data, can help ensure that a sponsor’s clinical research plan is designed to enroll the right patients, rely on the right investigators, and take place in the right countries for success. By performing such an assessment early, sponsors can embark on their research with achievable goals and avoid having to rescue trials or suffer costly delays in the development timeline.

A comprehensive feasibility assessment draws on extensive data resources. It involves several analytical steps and should be undertaken with experienced internal staff or a partner that has access to the necessary data, methodologies, expertise and technology platforms to make it easy for study planners to make critical decisions on the design and operation of a trial.

Enough Eligible Patients?

The efficiency of studies can be drastically improved by ensuring that the protocol targets a patient population that can be recruited. EHR data can be used to “pressure test” preliminary subject eligibility criteria to estimate the size of the available patient population within a geography. Such a sensitivity analysis uses large volumes of data, together with powerful analytics and visualization technology platforms to estimate the number of available patients as Inclusion/Exclusion (I/E) criteria are adjusted. The data types include demographics, lab values, diagnosis, procedures, and medications.

Seemingly minor adjustments to certain I/E criteria can yield significant changes in eligible patient counts without compromising the scientific integrity of a study and the ability to assess the study endpoints. For example, ICON analyzed the impact of the inclusion criterion “Worsening chronic heart failure (CHF) requiring hospitalization or an unscheduled outpatient visit in the last three months” in a trial being planned for a cardiovascular treatment. This criterion proved very limiting, so we extended the period for previous hospitalizations from three to six months and re-ran the analysis of EHR data. This simple change increased the number of eligible patients from 7,980 to over 14,000. (See Figure 1.)

Figure 1:  Small Changes in I/E Criteria Can Increase Eligible Patient Populations Significantly

How Available Will Investigators Be?

In virtually every therapeutic area, competition for site resources can make it difficult to enlist investigators and find patients. Thus, it is essential for sponsors to be aware of the current competitive environment as they plan the geographic footprint of their trials and consider investigator sites.

A competitive assessment should be conducted at the country level and then for individual sites, tailored to the indication, patient segment, drug class and mechanism of action. Data should be gathered on the timing of planned and ongoing trials that will be vying for the same investigators and patients. This type of data come from clinical trial registries, such as clinicaltrial.gov or commercial, subscription databases. At the same time, sponsors should be attuned to pending drug approvals and access to marketed products.

Some tools allow for results of this analysis to be reported in visually appealing ways for decision makers. For example, a heat map can depict concentration levels of competing trials by geography, pie charts can show the distribution of other sponsor’s studies across sites, and a timeline can clarify when competing trials will be enrolling. With such an understanding, sponsors can capitalize on sites rolling off of similar studies. (See Figure 2.)

Figure 2: Enrollment Periods for Competing Trials

Where Should We Run the Trial?

Mapping out the best mix of countries in which to run a trial is a complex process, as it should take factors into consideration such as: the competitive landscape, past performance of sites in each country and the available patient population. A best-practice approach is to assign weights to each of these factors based on its importance to the study. An analytical engine can then apply algorithms to those weighted factors to produce a rank-order listing of countries based on their suitability. (See Figure 3.)

Figure 3: Rank-Order List of Country Options

Which Sites Should We Approach?

The next step is to arrive at a preliminary list of possible sites which will later be confirmed/modified based on the results of direct contact with investigators.

Sites should be ranked according to an array of factors, with information gleaned from the contract research organization (CRO) or sponsor’s own intelligence on sites and supportive data from external sources. Decision criteria might include the site’s past performance working with the sponsor on related studies, the site’s recent activities in the indication, historical study initiation timelines and current competitive activity for all indications as well as the target indication. Each factor should be weighted based on its importance to the study to produce a rank-order list of potential sites or to place the sites in performance categories. (See Figure 4.)

Figure 4: Summary Analysis of Top 200 Sites

Subsequent due diligence includes further site identification activities. For instance, the trial’s I/E criteria can be applied against de-identified patient data from providers’ EHR systems to determine eligible patient counts by provider. With this insight, sponsors can proceed more strategically with a well-informed recruitment model, focusing on those sites that are serving a patient population that aligns with the protocol criteria.

Can We Complete Recruitment in Time? 

All of the insights gathered in the previous steps come together in the final stage of a comprehensive feasibility analysis, the aim of which is to estimate with great precision how long it will take to recruit the required number of patients and to verify the assumptions about the number of sites required.

The recruitment rates that analogous studies have achieved can give sponsors a general sense of what might be expected, country by country. Of course, the closer the match in terms of the target patient population, study design and phase, competitive pressure, current standard of care, recruitment target, and number of countries, the better. It may be that only a few studies prove to be sufficiently comparable to serve as meaningful benchmarks.

The final step should be to run a Monte Carlo Simulation (MCS) analysis to determine the optimal mix of operational parameters needed to enroll a study with a certain probability of success. Each input (such as the result of the recruitment rate analysis, the number of sites in each country, screen failure rates, etc.) into the application is given a range, and the application randomly selects a value from the range for each parameter. The standard practice is to run 5,000 virtual simulations to represent the range of possible outcomes. The output is displayed as a distribution chart showing the probability of completing enrollment within a given period.

Figure 5, for example, shows that in this case, the study has an 85 percent probability of completing enrollment in 37 months, earlier than the sponsor’s specifications. This shorter duration is a built-in contingency giving the sponsor room to meet its original timeline even if the trial events do not accrue as planned.

Figure 5:  Probability of Completing Enrollment as Planned

Despite its immense value, assessing a trial’s feasibility need not be a time-consuming exercise; with the right data, tools, and expertise, it can be completed in a matter of days. It is time well spent, as it gives sponsors a roadmap to follow in planning their studies. The process is most successful when sponsors

• Make it a team effort. Feasibility works best as a cross-functional team effort in which the CRO, if involved, is regarded as the sponsor’s partner/collaborator.

• Are transparent with their CRO. The results of the feasibility assessment will be more accurate when they are based on the combined data and expertise of the CRO and the sponsor. Sponsors should share their experiences with the compound and with specific sites with their CRO. They should also share key elements of the development plan-such as whether the study is on the critical path-with the CRO so that the CRO can tailor the feasibility planning accordingly.

• Engage their CRO early in development planning. Fundamentally, the feasibility exercise aims to prevent a sponsor from going down the wrong path with a study-before it is too late to make modifications to the protocol or before resources have been expended on the wrong sites.

Conducting a thorough data-driven feasibility assessment in this manner sets sponsors up for success in the operational aspects of their trials. They know precisely what will be required and how long it will take-two traditional sources of uncertainty in trial planning that have been overcome by a data-led approach.

Reference

1 Getz K, “The cost of clinical trial delays,” 2015. Accessed at https://www.ctti-clinicaltrials.org/files/pgct-session5.1-getz.pdf

Otis Johnson

Dominique Gard Muszinski

Otis Johnson, PhD, MPA, is Vice President, Feasibility & Clinical Informatics; and Dominique Gard Muszinski, is CNS Aligned Director, Feasibility Strategy, both for ICON.