ePRO in Diabetes Studies

As such, biopharmaceutical companies and researchers around the world are working to identify new, effective and safe treatments to help this growing patient population.

Diabetes clinical research often includes patient reported outcomes (PROs) to measure symptoms and safety over the course of the trial, quality of life, and the economic burden of the disease. Data are collected directly from patients on diaries and/or questionnaires, which are either printed on paper or available on electronic devices.

The use of electronic patient reported outcome (ePRO) data in clinical research is increasing as regulatory authorities acknowledge the credibility of such data, particularly in comparison to paper and pencil self-reports completed in unsupervised settings. Many sponsors leverage technologies by collecting symptom, event, and quality of life (QOL) data on the same electronic platform. This simplifies and streamlines the entire data collection and management process.

Transitioning a diary or questionnaire from paper to electronic formats requires physically modifying the appearance of the diary or questionnaire to fit on the screen of a device. The primary goal when transitioning from the use of paper questionnaires to versions administered electronically is to preserve psychometric validation by minimizing the degree of change between the paper version and the electronic one. The likelihood of comparability or measurement equivalence of the ePRO version to the paper one depends on the degree to which there is modification of the content, format, and usability of the PRO items and scales.

This article highlights the conversion from paper to ePRO for a PRO instrument designed to assess the impact of diabetes on the quality of life of patients with diabetes.

Diabetes programs are similar to other therapeutic areas, but often require these additional components:

• Patient data collection at various locations-the site, at home, at work, at school, etc.

• Study designs with inclusion and exclusion criteria that involve daily measures during screening.

• Complex randomization schemes and various defined treatment regimens.

• Requirements for treatment and management of emergent events anticipated with these patients.

• Safety alerts and reminders.

For many sponsors and clinical research organizations the decision to collect patient data on paper diaries or electronic ones is an economic issue. When the trials or studies are not collecting endpoint data, the disadvantages of paper methods may be manageable. If the data are intended to support an endpoint with the diabetes trial or study collecting data for submission or post-marketing verification, the following issues with paper become too expensive in terms of risk and resources:

• Real-time data are not available to support timely enrollment decisions, to manage compliance, to detect emerging problems in the trial, or to support adaptive trial designs.

• Incomplete diaries and reports are common.

• Safety review teams and regulatory authorities must work from low quality records.

• Logical inconsistencies in the records occur as subjects fail to follow conditional branching properly, usually necessitating the expense and difficulty of source data verification and clarification.

• There are no real-time automated alerts if symptom thresholds or other criteria are exceeded.

• Contingent messages and reminders to participants are more difficult to generate and are delayed.

• Side effects and unexpected problems that deserve review by investigators are unstructured and unavailable in a timely fashion.

• Recall bias is introduced when subjects do not complete PROs according to the protocol's schedule. This bias can obscure real signals from the study medication.

Of the five different ways to collect ePRO, many sponsors and CROs prefer the tablet, smartphone (handheld) and Internet for diabetes research. The digital pen and integrated voice response (IVR) do not integrate with glucometers to capture blood glucose levels automatically. Handheld patient data collection devices as well as Internet-based systems have been used to collect ePRO symptom and safety data, and integrate that data with glucose readings recorded directly from wireless connected glucometers.

Tablets, smartphones, and the Internet provide easy-to-read screens for electronic diaries and questionnaires, enabling clear question displays and unambiguous areas for touch-screen or stylus responses. Companies in the ePRO industry have managed many trials for diabetic neuropathy, diabetes, diabetic peripheral, and neuropathy indications. There are a number of standardized questionnaires implemented on ePRO devices:

• Insulin treatment satisfaction questionnaire

• Brief pain inventory short form

• SF-36v2™ health survey

• EuroQol-5 dimensions (EQ-5D)

• McGill pain questionnaire

• Neuropathic pain scale

• SF-36v2™ health survey

• Profile of mood states-brief

DHP-18 for ePRO data collection and analysis

The Diabetes Health Profile (DHP-18) is a ubiquitous diabetes questionnaire that has been extensively used in its paper form to capture QOL data from diabetes patients. It was developed by co-author Keith Meadows, and was adapted for electronic administration because it has been shown that electronic questionnaires are preferable to paper.

Diabetes mellitus is a prevalent chronic condition affecting approximately 55.5 million adults in Europe and 29.5 million adults in North America. The long-term health impacts of diabetes may include heart disease, kidney failure, and microvascular complications such as neuropathy and retinopathy. Alongside these health complications, diabetes can also have an adverse impact on the patient's psychological and emotional functioning as well as social activity and behavior.

The importance of PRO measures in the care and treatment of people with diabetes has gained significant prominence over the past two decades. This has resulted in the development of a variety of diabetes-specific outcome measures to assess the impact of living with diabetes and its treatment including, quality of life, health-related quality of life, health status, treatment satisfaction, and symptoms.

One such instrument is the DHP-18, which as a quality of life questionnaire was initially developed as a patient self-completion instrument for use across a range of settings including clinical trials, research, clinical practice, and population surveys.

Diabetes Health Profile-an overview

Derived from the original 32-item DHP-1, the DHP-18 was developed specifically to assess the impact of diabetes-related aspects of psychological and behavioral functioning on the quality of life of people with Type 1 and Type 2 diabetes aged 18 years and older. As such, it offers a significant benefit for example in assessing the efficacy of different treatment modalities such as changing from pre-insulin to insulin.

Comprising 18-items which are scored using a 4-point Likert type scale, items are summed to provide three subscale scores using a 0-100 metric for: psychological distress; perceived barriers to activity; and eating problems, which despite the importance of eating behavior in the management of diabetes, is largely neglected in the majority of diabetes-specific PROs. The DHP-18 can be administered through a range of modalities including paper, read to the respondent face-to-face, and administered over the telephone, with average completion times of six to eight minutes.

Globally, the DHP-18 has been completed by nearly 5,000 respondents participating in clinical trials, community surveys, academic research studies, and educational intervention programs both in the United Kingdom and across Europe. At present, the DHP-18 is being applied in the assessment of quality of life in a variety of settings including, Phase III trials across Europe and research projects in Australia, Ireland, Martinique, North America, and national funded research in the United Kingdom. The DHP-18 is also the diabetes-specific outcome measure selected by the UK Department of Health and is currently being used in their pilot study to evaluate the efficacy of patient reported outcome measures for long-term conditions in primary care. Planned international diabetes clinical trials include the use of the DHP-18 in a number of countries including the United States, Canada, China, and Southern and Northern Europe.

The DHP database comprising information on nearly 5,000 people with Type 1 and Type 2 diabetes who have completed the DHP-18 is currently being utilized to:

• Estimate the minimally important difference of the DHP-18 for both Type 1 and Type 2 diabetes using anchor and distribution based approaches.

• Develop norm referenced scores.

• Estimate the association between the DHP-18 responses and EQ-5D utility values by response mapping.

• Develop a shorter version of the DHP-18 using Rasch analysis.

Reliability, clinical utility, and user acceptability

The DHP-18 has been evaluated and described as having good evidence for reliability and internal and external construct validity and is one of the few diabetes-specific outcome assessment instruments to have involved diabetes patients in its development.

The clinical utility of the DHP-18 has been demonstrated by its ability to discriminate between patients groups treated with insulin, oral treatment and diet, presence of microvascular complications, weight gain, non-adherence to treatment, and severe hypoglycemic episodes.

To further enhance the clinical utility of the DHP-18, work is in progress by a team led by John Brazier, Professor of Health Economics at the University of Sheffield to derive utility scores for the DHP-18.

To date, the DHP-18 has been administered mainly as a patient self-completion pencil-and-paper format. It has been shown to have high user acceptability with questionnaire postal response rates of approximately 70% and total item completion rates of 90%. Results from cognitive debriefings across a wide range of language groups have also shown there to be minimal difficulties in patients understanding of the meaning of the 18 items.

The paper version of the DHP-18 has been translated to 26 languages for use across Europe including Bulgarian, Czech, French, German, Slovak, Slovenian, Spanish and Swedish. The DHP-18 has also been adapted for use in the United States (English) and Canada (English and French) and is currently being adapted for use in a further three languages including Mandarin.

Creating and validating for electronic collection

Despite evidence of patient acceptability, high-item completion rates, and the overall integrity of data collected using the paper version of the DHP-18, there is both a strong scientific and business case for the creation of a validated electronic version of the measure, eDHP-18. Electronic patient reported outcomes are increasingly being seen as essential by clinical trial sponsors for ensuring data integrity and regulatory support.¹

Validating the eDHP-18

Although paper-based PROs are an established and accepted medium which are easy to reproduce and distribute, ePROs offer a number of distinct advantages over paper. Apart from enabling administration of the eDHP-18 in a consistent, standardized, and objective manner, a key advantage is the ability to date and time stamp PRO data to avoid the recognized limitation of paper-based PROs "parking lot effect" where study participants retrospectively and prospectively enter data-often in the parking lot before their follow-up appointments.

Further benefits of the DHP-18 for electronic data capture include the ability to integrate eDHP-18 data with other electronic data capture, for instance primary clinical endpoints such as patients HbA1c levels; and, extend to web-browser data collection.

Migrating from paper to electronic data collection is a significant movement in the field of PRO measurement. There is a range of modalities available for data collection including: smartphone (handheld), tablet, IVR, and Internet. To retain the integrity of the different administration modalities of the DHP-18, an eDHP-18 can be administered using a combination of handheld devices and the Internet for self-administration, reading face-to face to the respondent and administration over the Internet. However, there is the requirement that sponsors provide evidence to support the comparability or measurement equivalence of the eDHP-18 to the paper-based version from which it has been adapted.

The level of evidence to support the comparability or measurement equivalence of the ePRO to the paper-based PRO from which it has been adapted will vary in accordance with the magnitude of the modification and its effect on the content, format, and interpretation of the PRO items and scales. Guidelines suggest that the psychometric properties of the original measure will still hold for the ePRO version if only minor modifications have been carried out, and that cognitive debriefing and usability testing will likely suffice as the level of evaluation.² In cases where substantial modifications have been made, establishing the measurement equivalence of the ePRO is likely to include full psychometric evaluation.

Let us compare one question from the DHP-18 in paper format, and the electronic version on integrated handheld devices. Questions on both paper and electronic devices are the same but the response mechanics are different. Also, the electronic version requires that answers be complete before progressing to the subsequent question, improving data quality and compliance.

When there is only a little change between paper and the electronic instrument, it will be shown, a cognitive debrief is required to demonstrate the similarities and differences between the visuals.

ePRO migration challenges

While the reuse of validated questionnaires seems intuitive and practical, there remain some obstacles to transitioning questionnaires to electronic versions. Developers do not license screen shots of validated instruments and this makes it difficult for ePRO providers to provide fully reusable versions of questionnaires based on a published specification. Sponsors often regard validated instruments as private assets and a competitive advantage, and are reluctant to publish any studies done to establish the validity of such assets. Additionally, the compensation of the authors and copyright holders of the questionnaires necessitates licensing of copyright and other logistical matters.

The virtue of providing validated instruments to the industry at large is certainly recognized, and academic societies, industry groups, and distribution channels are evolving to support this need. One such group which has recently formed is the ePRO Consortium which is working with the Critical Path Institute on this goal.

Validation required to migrate from PRO to ePRO

The level of validation depends on the degree to which the appearance, operation, and content may be changed. An ISPOR publication distinguishes small, medium, and large changes.

Small changes are exemplified by simple alterations in instruction wording, such as asking the responder to select a response option by "tapping" a screen rather than by "drawing a circle" on the paper. Another example is changing the presentation from several questions on a page to one per screen.

Medium changes include changing modes of presentation from one sensory channel to another such as reading questions and responses to listening to someone else read them, or introducing an operation such as scrolling a screen in order to view what would appear on a single page on paper.

Large changes are those that affect the meaning of the questions or the content of the measure (additions, deletions).

Simply putting a paper questionnaire on an electronic device with no other changes may need only some usability and cognitive testing to support suitability for use. The level of change dictates the type of validation work required, ranging from cognitive debriefing with target patient populations to equivalence studies between the paper and electronic versions of the instruments.

Ownership

The questionnaire author owns the content copyright. Usually the technology provider owns the code and programming that present that content on particular platforms. Sponsors who pay for validation work have typically not authorized the publication of the results, which is resulting in duplication of validation work to support conversion of a PRO instrument from paper to an electronic version.

Summary

Electronic data collection is a powerful means to ensure data integrity of patient reported outcomes within diabetes indications. The eDHP-18 remains a universal diabetes questionnaire and provides additional value now that it has been converted for presentation as an electronic instrument.

Some of the biggest advantages that ePRO provides over paper PRO for diabetes research include:

• Automated transfer of blood glucose readings from radio-enabled glucometers.

• Validated time stamps for completion of contextual information pertaining to each glucose reading.

• Clear instructions for sites and subjects to help them comply with their protocol (e.g., reminder at end of timed assessment), including automation of contingent branching to prevent logically inconsistent data.

• Episodic diaries that can be completed in real time, which prevent errors in recall.

• Review screens to prevent duplicate reports into an episodic diary.

• Remote monitoring for significant changes in health status.

• Ready access to data on remote participants, enhancing trial management and the safety of subjects.

Keith Meadows, PhD, is Founder and Director of DHP Research & Consultancy, Oxfordshire, UK. Christine Hall* is Senior Marketing Manager at PHT Corporation, 500 Rutherford Avenue, Boston, MA, e-mail: chall@phtcorp.com.

*To whom all correspondence should be addressed.

References

1. US Food and Drug Administration, Guidance for Industry, "Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims," (2009), http://1.usa.gov/7ROMCu.

2. S. J. Coons, C. J. Gwaltney, R. D. Hays, et al., "Recommendations on Evidence Needed to Support Measurement Equivalence Between Electronic and Paper-Based Patient-Reported Outcome (PRO) Measures: ISPOR ePRO Good Research Practices Task Force Report," Value in Health, 12 (4) 419-429 (2009), , http://bit.ly/MMpBIj.com.