The 21st Century Cures Act was enacted in 2016 to both accelerate medical product development, and it expresses the need to include both patient experience and patient perspective in the drug development process. In accordance with the Act as well as the Prescription Drug User Fee Act, the FDA is currently developing four new methodological Patient-Focused Drug Development (PFDD) guidance documents for industry that incorporate patient experience data into drug development. The key concepts of each guidance-and their current status-are summarized below.

Although the updated guidance documents are still in development, researchers should start planning now to incorporate the basic features outlined in the draft guidances and discussion documents, as these concepts will likely be required in future research programs.

Overview of Guidance Documents:

• Guidance 1: Whom do you get input from, and why? How do you collect the information?
• Status: Public meeting held; draft guidance available (Published June 2018)

• Guidance 2: What do you ask, and why? How do you ask non-leading questions that are well-understood by a wide range of patients and others?
• Status: Public meeting held; draft guidance available (Published September 2019)

• Guidance 3: How do you decide what to measure in a clinical trial and select or develop fit-for-purpose clinical outcome assessments (COAs)?
• Status: Public meeting held October 2018; meeting discussion document available

• Guidance 4: Once you have a COA measurement tool and a way to collect data using it, what is an appropriate clinical trial endpoint?
• Status: Public meeting held December 2019; meeting discussion document available

Guidance 1: collecting comprehensive and representative input 

Guidance 1 lays the foundation for the Patient-Focused Drug Development series by defining patient experience data and outlining whom to get patient experience data from and how to collect, analyze and report the information obtained. Key points include:

What is patient experience data? 

Includes the “experiences, perspectives, needs, and priorities of patients related to:

• the symptoms of their condition and its natural history

• the impact of their condition on their daily function and quality of life

• their experience with treatments

• input on which outcomes are important to them

• patient preferences for outcomes and treatments

• the relative importance of any of these issues as defined by patients.”

Who to get input from?

• Define the target population:  Obtain information from the specific population you wish to include in your study. This should compare to the inclusion/exclusion criteria in your protocol.

• Determine who will be providing patient experience data: FDA generally recommends that the patient self-report their experience, unless they cannot reliably do so (e.g. young children, cognitively impaired individuals).  In such cases a clinician or caregiver can report on patient experience data that is observable.

How to collect information?

• Determine the study design and research setting: e.g. study type (clinical trial, observational, survey), sampling method, representativeness of the study population, and sample size required.

• Determine the methods for collecting and analyzing patient experience data: qualitative, quantitative, and mixed methods (both qualitative and quantitative).

• Operationalize and standardize data collection and data management:  Considerations include locating patients, sampling strategy, and collecting and managing data.

• Data collection methods include:
• Interview and Focus Groups

• Observations

• Documents (includes medical records, scientific publications)

• Questionnaires

• Audiovisual materials

• Social Media and Identifiable Patient Communities

• Digital Health Technology (includes wearables such as accelerometers, heartrate trackers, etc.)     

For full methods on collecting and analyzing COAs, the guidance document refers to the FDA Guidance for Industry “Patient Reported Outcome Measures: Use in Medical Product Development to Support Labeling.”¹

Guidance 2: methods to identify what is important to patients

Guidance 2 further expands on the methods to collect representative patient experience data, especially how to identify what matters most to patients in terms of disease burden and treatment and informs COA development and selection. Key points include:

Background research: Literature reviews and consultation with subject matter experts should be used to develop research questions and select the appropriate methods, listed below.

Qualitative research: Patient experience in their own words.

• Methods:
• One-on-One Interviews: semi-structured, structured, unstructured interviews

• Focus Groups: 5-10 participants per group 

Note: “FDA does not have a single recommended interview mode for eliciting patient input”

• Approaches to Asking the Right Questions: Critical for obtaining unbiased patient input
• Spontaneous responses are ideal, but sometimes participants will need to be prompted

• Avoid leading questions by using a semi-structured interview guide to facilitate discussion, do not suggest an answer, do not assume you know what the subject is feeling or thinking, do not imply that you prefer participant to respond a certain way

Quantitative research: Collection of quantifiable data and application of statistics to summarize patient experience data

• Methods:
• Survey administration: self-administered or interviewer-administered

Note: “Using survey instruments in a clinical trial for screening and/or exit visits may add greater depth to understanding the burden of disease and treatment”

• Considerations for Developing a Survey
• Survey instructions and items should be well understood by participants, translated and culturally adapted, tested through interviews of target population to make sure the survey is interpreted as intended, and tested for usability if administered electronically

• Avoid poorly defined terms, double-barreled questions, double negatives, or leading questions

Mixed methods research: This includes research that uses both qualitative and quantitative methods.  

Note: “FDA encourages researchers to consider the goals and objectives of using a mixed-methods approach and how the results from both qualitative and quantitative research components are intended to be used together.”

When selecting methods, special populations (children, cognitively impaired, rare diseases) should be taken into consideration.  For example, factors such as limited attention span of children, use of caregivers for patients who are unable to self-report, remote assessment for rare diseases where patients are geographically diverse should be considered when developing surveys and data collection methods.

Finally, Guidance 2 discusses considerations for using social media. Different social media communities appeal to different segments of the population and therefore a variety of social media communities should be used to obtain the most generalized information.  Other considerations include whether the participant is anonymous, and limitations with ability to verify patient characteristics such as identity and diagnosis.

Guidance 3 (discussion document): selecting, developing or modifying fit-for purpose clinical outcome assessments

After identifying what is most important to patients as outlined in Guidance 1 and 2, Guidance 3 outlines how to select, develop or modify a “fit-for purpose” COA that captures patient experience data. Currently, only the discussion document for Guidance 3 is available. This discussion document provides guidance on all four COA types: Patient-Reported Outcome (PRO), Clinician Reported Outcome (ClinRO), Observer-Reported Outcome (ObsRO) and Performance Outcome measures (PerfOs) including mobile health technologies. The discussion document aligns closely with the FDA Guidance “Patient Reported Outcome Measures: Use in Medical Product Development to Support Labeling.”² Key points include:

Roadmap to COA selection: What to consider when selecting or developing/modifying a COA:

Figure 1: Roadmap to COA Selection Source: Adapted from FDA PFDD Discussion Document 3

Evaluation of COAs: When requesting review and advice on COAs for use in medical product development programs, the following should be submitted to the FDA:

• intended use and endpoint for the COA

• copy of COA

• conceptual framework of COA

• evidence to support content validity and other measurement properties

• scoring information for COA

Note: “FDA generally reviews COA data as part of the totality of evidence to inform benefit-risk assessment, whether or not the labeling claims are granted.”

Below is a summary of evidence recommended to be submitted to the FDA to demonstrate a COA is “fit-for-purpose”:

• Conceptual Framework-details of the concepts being measured

• Evidence that COA measures the concepts of interest (content validity).  This evidence can be obtained from qualitative studies (e.g. interviews or focus groups), quantitative studies and/or published literature.  Examples of information submitted to establish content validity include:
• Literature reviews

• Expert Input

• Qualitative Study protocols and summary reports for concept elicitation interviews and cognitive interviews to test the COA

• Evidence to support item relevance, item stems, response options, recall period, and meaningful change

• Evidence of Other Measurement Properties:
• Reliability

• Construct Validity

• Ability to Detect Change

• Interpretation of Meaningful Change:

FDA recommends anchor-based methods supplemented with both empirical cumulative distribution function (eCDF) and probability density function (PDF) curves

Recommended anchors:  Patient Global Impression of Severity (PGIS) (at a minimum), Patient Global Impression of Change (PGIC), or well-established clinical outcome

Note: “From a regulatory standpoint, FDA is more interested in what constitutes a meaningful within-patient change in scores from the patient perspective (i.e., individual patient level).  A treatment effect is different than a meaningful within-patient change. The terms minimally clinically important difference (MCID) and minimum important difference (MID) do not define meaningful within-patient change if derived from group-level data.”

Electronic modes of administration: The discussion document outlines the advantages to using electronic data capture, including:

• No need for manual secondary data entry

• Direct transmission

• Alarm or reminder capabilities

• Time and date stamp capabilities

• Real-time data recording and transmission

• Remote data capture

Regarding Bring Your Own Device (BYOD), the FDA discussion document acknowledges the increasing interest in BYOD but still recommends using a single platform throughout a clinical trial to reduce variability.  If a sponsor decides to proceed with BYOD, they should provide a detailed plan to the FDA to review and comment on to ensure that the instrument will function as intended across devices. 

Paper-electronic Migration and Equivalence: Mode equivalence testing is not always necessary and depends on the magnitude of changes.  The discussion document refers to Coons et al. 2009 ISPOR Task Force paper on “Evidence Needed to Support Measurement Equivalence between Electronic and Paper-Based Patient-Reported Outcome (PRO) Measures.”³ The FDA recommends any device usability testing results, instrument screenshots, and training materials be submitted for FDA review. 

Device Validation: The eCOA system should undergo system validation as outlined in the ISPOR Task Force paper “Validation of Electronic Systems to Collect Patient-Reported Outcome (PRO) Data,”⁴ including user acceptance testing (UAT). In addition, sponsors are encouraged to perform usability testing with cognitive interviews (also referred to as Cognitive Debriefing and Usability Testing (CD/UT)), to ensure COA comprehension and ease of use in the intended patient population.

Special patient populations

Rare Disease Populations: In rare diseases, well-characterized endpoints and existing COAs that are suitable for the patient population may not be available.  Due to the small number of patients available, traditional COA development and validation may not be feasible and therefore the FDA is open to other approaches (e.g. combined concept elicitation and cognitive interviews).

Pediatric Populations: Pediatric populations are a high priority for the FDA.  It is important for sponsors to determine whether the COA can be reliably and validly completed by young children.   

Cognitively Impaired Populations: For patients who are cognitively impaired, the FDA recommends using ObsRO, ClinRO, or PerfO rather than a PRO.

Guidance 4 (discussion document): incorporating clinical outcome assessments into endpoints for regulatory decision making

The goals of Guidance 4 as outlined in this discussion document are to explain what the FDA considers when evaluating a COA as part of regulatory decision-making. The discussion document outlines the estimand framework which aims to align “the clinical study objective with the study design, endpoint and analysis to improve study planning and interpretation of analysis.” 

The discussion document also outlines methods to interpret study results to evaluate what constitutes a meaningful within-patient change. 

Overview of estimand framework

The estimand framework should be clearly defined prior to developing a protocol and included in the protocol and Statistical Analysis Plan (SAP).  The estimand framework includes:

• Target study population
• Generally, the intent-to-treat population (ITT) should be used.  Sponsors should provide justification if using a COA analysis population different from the ITT population.

• Endpoint of Interest
• The outcome being tested or measured within the target population; this may include data from multiple variables.  

• Intercurrent Events
• Events precluding observation or affecting interpretation of endpoint of interest

• Intercurrent events should be accounted for in the SAP

• Examples: dropouts, use of rescue medication, not following prescribed regimen (medication adherence)

• Population-Level Summary
• The population-level summary is the comparison between treatment arms, treatment conditions, or other groups

Figure 2: Overview of Estimand within Context Source: Adapted from FDA PFDD Guidance Document 4

Example Estimand in Prostate Cancer Source: ERT

Meaningful within-patient change

As mentioned in Discussion Document 3, the FDA recommends determining meaningful within-patient change using anchor-based methods (anchors such as PGIS, PGIC), and also using Empirical Cumulative Distribution Function (eCDF) and Probability Density Function (PDF) Curves. Finally, the FDA recommends that sponsors ensure that COAs intended for approval or to support labeling claims are appropriately positioned in the endpoint hierarchy, the trial’s protocol and SAP should state each COA-based endpoint as a specific clinical trial objective, and address multiplicity concerns and plans for missing data.

Throughout all guidance documents it is mentioned repeatedly to discuss plans early with the FDA to obtain feedback from the relevant FDA division.

Conclusions

Since the release of the 2009 FDA Guidance on the use of Patient-Reported Outcome Measures, there have been many advances in the field of outcome research and data capture technology.  The FDA updates currently in development are an important step forward in the use of outcome measures for medical product development.  The focus on identifying outcomes which are meaningful to the patients themselves is an important evolution in outcomes research-particularly since these outcomes may vary between patients.

Although the updated guidance documents are still in development, researchers should start planning now to incorporate the basic features outlined in the draft guidances and discussion documents, as these concepts will likely be required in future research programs.

Sarah T. Gary, PhD, is a Sr. Scientific Advisor; Nadeeka R. Dias, PhD, is a Sr. Scientific Advisor; Kenneth G. Faulkner, PhD, is Vice President, eCOA Science, all with ERT.

References

1. FDA Guidance for Industry “Patient Reported Outcome Measures: Use in Medical Product Development to Support Labeling”. December 2009. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-reported-outcome-measures-use-medical-product-development-support-labeling-claims
2. Ibid
3. Coons SJ, Gwaltney CJ, Hays RD, Lundy JJ, Sloan JA, Revicki DA, Lenderking WR, Cella D, Basch E; ISPOR ePRO Task Force. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report. Value Health. 2009 Jun;12(4):419-29. doi: 10.1111/j.1524-4733.2008.00470.x
4. Zbrozek A, Hebert J, Gogates G, Thorell R, Dell C, Molsen E, Craig G, Grice K, Kern S, Hines S. Validation of electronic systems to collect patient-reported outcome (PRO) data-recommendations for clinical trial teams: report of the ISPOR ePRO systems validation good research practices task force. Value Health. 2013 Jun;16(4):480-9. doi: 10.1016/j.jval.2013.04.002.