In November 2019, the International Council on Harmonization (ICH) released the final version of an addendum (R1) to ICH E9 guidance addressing statistical methods for use in clinical trials. In this document, a structured statistical framework based on the use of estimands was outlined for use in clinical trials.¹The objective of the estimand framework is “to align the clinical study objective with the study design, endpoint, and analysis to improve study planning and the interpretation of analysis.” 

On Dec. 6, 2019, FDA held a public workshop to address the topic of patient-focused drug development as they update the 2009 guidance on patient outcomes.²This workshop addressed the development of Guidance 4-Incorporating Clinical Outcome Assessments into Endpoints for Regulatory Decision-Making. As part of the planned updated guidance, FDA has refined the ICH E9 (R1)estimand framework previously used in statistical plans in clinical trials and made it relevant for all trials with clinical outcome assessment (COA) endpoints. During the workshop, FDA provided guidance on how to implement the estimand framework in clinical trial study design with COA endpoints. Below is an overview of the estimand framework proposed for use in clinical trials with COA endpoints, and a summary of the key points presented in the December FDA workshop.  

What is an estimand?

According to the ICH report¹, an estimand is a description of the treatment effect associated with a clinical trial objective. More specifically, it summarizes what the outcomes would be in “the same patients under different treatment conditions being compared.” The targets of estimation must be defined in advance, allowing design of a trial to estimate treatment effect.

The description of an estimand involves precise specifications of certain attributes, which should be developed based on clinical considerations and how intercurrent events (defined below) are reflected in the clinical question of interest. 

The characteristics of an estimand include the following four attributes:

• Definition of the targeted study population

• Statement of the endpoint of interest

• Details of any intercurrent events

• Population level summary of the variable of interest

These attributes should be clearly defined prior to developing a protocol and included in both the protocol and statistical analysis plan.

Target study population 

Defining the patients who are targeted by the scientific question is key in determining how the data will be analyzed. The study population is typically defined by the inclusion/exclusion criteria in a protocol. In some cases, there may be multiple study populations of interest that can be analyzed to address different study questions, such as all patients, patients that completed the baseline visit, and patients that completed all visits. These populations should be clearly defined in the study protocol and the statistical analysis plan.

Endpoint of interest

An endpoint is defined as a variable intended to reflect an outcome of interest to answer a research question. When establishing an endpoint, it is important to define the types of assessments made, timing of the assessments, and how multiple results might be combined to produce a composite endpoint. Measurement properties, such as the reliability, content validity, construct validity, and the ability to detect change are critical when determining an endpoint. FDA has stressed that any COAs used in clinical trials are determined to be “fit-for-purpose” (measuring what it is intending to measure for the applicable population). 

Intercurrent events 

A key component of the estimand framework considers intercurrent events. These events are defined as “events occurring after treatment initiation that affect either the interpretation or the existence of the measurements associated with the clinical question of interest.”¹This includes discontinuation of study medication, use of alternate therapies, or additional health issues encountered during a clinical trial. For example, if a patient was in a clinical trial for eczema and contracted poison ivy in the same area where their eczema was being measured, it would be considered an intercurrent event. 

Other examples would be a patient initiating over-the-counter pain medication during a trial, or the learning effect associated with a cognition assessment as the patient becomes familiar with the test procedure. The ICH and FDA recommendation is to acknowledge these types of occurrences at the outset of the trial and document a statistical analysis plan to account for any anticipated intercurrent events.  

Population-level summary of variable

Selecting how the data are summarized forms the basis for comparisons between treatment arms, treatment conditions, and other groups. For example, in a weight loss study investigating a new treatment, it would be appropriate to summarize the data by looking at the average weight loss in the population, and then comparing the mean weight loss between treatment groups. In this case, the population-level summary is the mean weight loss over the duration of the study.  

There are many considerations for how data should be summarized and analyzed, and the considerations for COA-based endpoints are similar to those of any other endpoint. Some analysis considerations include:

• Timepoints for interim and final analysis (e.g., 16 weeks): Justification should be provided as to why the chosen timepoint was selected, considering duration of treatment onset and expected time to response.

• Analyzing ordinal data: Ordinal data (such as rating pain on a scale from 1 to 10) requires the use of appropriate statistical methods for analysis.

• Time-to-event analysis: In this case, a justifiable, predefined, clinically relevant threshold should be defined for events such as symptom deterioration, maintenance, or improvement.

• Responder analyses and percent change from baseline: Analyzing continuous or ordinal COA data is preferred to dichotomous (yes or no) data. If a responder endpoint is appropriate, it should be well-defined with evidence to support that the predetermined threshold for response is clinically meaningful. 

Simplified example of an estimand 

Shown below is a simplified example of an estimand for a health-related quality of life endpoint in a clinical trial of a weight loss treatment in obese patients. This example is not meant to be all inclusive but is used only to illustrate the components of an estimand associated with a clinical trial. A complete estimand would include additional detail and a more complete description of the statistical analysis plan, details on sensitivity analyses, and potential secondary endpoints-additional details are available in the ICH E9( R1) addendum.¹

Trial objective

To determine the effectiveness of a weight loss treatment in obese patients on their health-related quality of life.

Estimand

Mean change from baseline to week 52 in the health-related quality of life score measured by SF-36 health survey in the treatment group versus control group for a weight loss treatment in obese adults.

• Target population: Obese adults

• Variable of interest: Absolute change in SF-36 score from baseline to week 52

• Intercurrent events: Study discontinuation, onset of diabetes

• Population-level summary of variable: Comparison of mean SF-36 score between treatment groups

Using estimands during trial design   

It is critical that the objectives of the research are clearly stated at the start of the study design, prior to developing an estimand. Both the natural history of the disease as well as the goal of the investigational product should be considered when developing the research objective. After the research objectives have been clearly defined, the estimand framework should be used to determine study procedures, such as when data are collected, the statistical analysis plan, and communication of the trial results.

According to section A.6 of the ICH guidance¹, the clinical trial protocol should very clearly specify a primary estimand that addresses the primary study objective. Estimands for secondary trial objectives that support regulatory approval should also be specified on the protocol. Exploratory endpoints should be defined by an estimand as well; however, it is not a regulatory requirement to document an estimand for each exploratory objective in the protocol.

Any changes to an estimand after the study has started will affect study credibility and will require an amendment to the protocol, so it is critical to define the estimand properly before study initiation. In addition, study reports should reference the estimands used in the study protocol and follow the framework for analyzing and reporting results to regulators.

Conclusions

With the final approval and release of ICH E9 (R1), the use of estimands in clinical trials is expected to become required by ICH member organizations (including FDA and the European Medicines Agency). The FDA workshop and planned updated guidance on patient-focused drug development will certainly include recommendations on the use of estimands as well.  

Clinical trial sponsors should prepare now to include estimands in their protocols as well as their statistical design documents and study reports. Adopting the estimand framework and methodology now will facilitate future protocol design and ensure protocols are developed in compliance with new regulatory guidances.

Kenneth G. Faulkner,PhD is VP, eCOA Science; Rinah Yamamoto, PhD, is a Senior Scientific Advisor; Elisa Conrad, MA, is a Scientific Advisor, all of ERT.

References:

1. Addendum on Estimands and Sensitivity Analysis in Clinical Trials to the Guideline on Statistical Principles for Clinical Trials E9(R1) Final version (Step 4), Adopted on 20 November 2019. (https://database.ich.org/sites/default/files/E9-R1_Step4_Guideline_2019_1203.pdf)
2. Public Workshop on Patient-Focused Drug Development: Guidance 4 – Incorporating Clinical Outcome Assessments into Endpoints for Regulatory Decision Making.  December 6, 2019.  (https://www.fda.gov/drugs/development-approval-process-drugs/public-workshop-patient-focused-drug-development-guidance-4-incorporating-clinical-outcome)