The First eCOA Forum Examines BYOD and Item Skipping

A compelling benefit of membership of Critical Path Institute’s (C-Path) Electronic Clinical Outcome Assessment (eCOA) Consortium is the ongoing opportunity to interact in a neutral environment with sponsors within the Patient-Reported Outcome (PRO) Consortium. There is much fertile ground to explore in the context of eCOA implementation, and much of what the eCOA Consortium does is founded on this relationship. What C-Path offers in addition is the pivotal interface with FDA through the public-private partnership between C-Path and the agency, a relationship that has driven the publication of important and authoritative works, such as the recently published paper on ePRO dataset structure and standardization.

Across both consortia, members have yearned to have open, non-committal, and exploratory discussions on specific eCOA-related matters with regulators. Thanks to the collaborative mindset of FDA colleagues, especially within the Division of Clinical Outcome Assessment, an invitation to join a cross-section of eCOA Consortium and PRO Consortium members to discuss two specific eCOA-related topics was accepted, and the inaugural eCOA Forum was held immediately after the close of the 14th Annual PRO Consortium Annual Workshop in Silver Spring, MD, on April 20.

Two specific topics that were examined; bring-your-own-device (BYOD) and item skipping. The first discussion explored what additional evidence is necessary to allow for BYOD approaches for PRO data collection to be incorporated into registrational trials, assuming PRO measures within an ePRO implementation are fit-for-purpose. The discussion referenced established industry-best practices and their applicability to BYOD models, the perceived risks of this approach, and the importance of usability alongside measurement comparability.

Much of the concern around the BYOD approach focuses on the impact of smaller device screen sizes and the understanding that an element of variability in data may be introduced by BYOD that is mitigable through training and good design principles.

The discussion focused on the fact that the requirements are the same for BYOD approaches as with data collected by other methods to be used in registrational trials. The group consensus concluded that eCOA providers should develop a package of evidence—generated in a population at least similar to that being studied—that usability is assured, as this will be of benefit to sponsors exploring BYOD for registrational trials.

The second more provocative topic asked the group to answer this question: Is it ever appropriate to require a trial participant to complete every item within a PRO measure? In preparing for this discussion, it was apparent that different stakeholders take a range of contrasting positions. The core concern could be reframed as simply not wanting respondents being forced to make up answers. A distinction between the evidence to be gathered during instrument development and then instrument application was also raised because the skipping of individual items is informative during instrument development.

There was acknowledgement of the fact that participants may decline to complete study procedures and should be appropriately consented but also advised of the reasoning and value of contributing their data. Development of novel measures may allow for identification of—and subsequent removal or revision of—potentially challenging questions, and perhaps incorporation of “not applicable” or “prefer not to answer” options, but that isn’t an option for legacy PRO measures.

The counter-argument assessed whether allowing for item skipping actually has an impact on the data obtained, and allowing skipping may help inform the qualitative assessment of those more challenging questions. The use of item libraries that enable very specific and pertinent questions to be selected is a methodology that may negate the issue altogether.

On this matter, the eCOA Forum concluded without a resolution, but collectively accepted that outcome. If good design principles are followed and active skipping is required—to avoid accidental missed responses—data quality impact should be minimized. Moreover, the opportunity to reflect on the perspectives of key stakeholders on challenging issues was both stimulating and productive, and the eCOA Consortium will continue to drive such vital discussions.

Authored on behalf of Critical Path Institute’s eCOA Consortium by Scottie Kern, Executive Director.