The Future of ePRO Platforms

It is tempting to imagine the use of the patient’s own mobile computing platform for collection of patient reported outcomes. This would solve some of the problems faced when using the e-PRO devices employed today:

• Provisioning costs (purchasing or leasing the devices to be used in the trial)

• Supply issues (delivering the devices to the sites for distribution to subjects, and collection after the subject completes the trial)

• Training (handling and use of the device by subjects and site staff)

• Maintenance and Help Desk (device-related help desk questions, replacement of faulty devices)

This article evaluates the practicality of such an approach, and the issues that need to be addressed if it is to succeed.

Present state of the art

The goal of a Patient Reported Outcome (PRO) system is to collect data directly from subjects; data used to measure the benefit of treatment or the risk in medical clinical trials.¹ Initially this was done using a pen and paper, and patient responses were collected both in the form of surveys conducted once (or a few times) during a trial and/or in the form of a patient diary, containing responses collected regularly (often one or more times a day) throughout the trial.

The move towards electronic PRO (ePRO) solutions, which started in the 1990s, was fuelled by a number of considerations, primarily:

• Improved compliance through the use of alarms, reminders, and date and time stamps

• Improved data quality through the use of electronic data collection and inbuilt data checks

• Reduced trial times due to quick access to data without requiring data transcription

Interestingly enough, cost has not been one of the primary movers. Although most companies adopting ePRO have had hopes that improved compliance, data quality and reduced trial times in themselves would lead to cost savings, these are cost savings that are difficult to quantify. Indeed often the move to ePRO involved higher up-front costs, with eventual savings being realized later in the trial process.

ePRO solutions diverged early along two separate paths. The simplest and most cost effective solutions have been the Interactive Voice Response Systems (IVRS), but these have had restrictions in their functionality, the user interface and the type of data that can be collected.

In order to support the collection of more complicated data a number of vendors developed solutions that could support entry of textual and graphical data.² These solutions were based on proprietary software running on commercially available electronic platforms, or “device-based applications.” Initially these solutions were based on commercially available Personal Digital Assistants (PDA) platforms. The earliest were based on the Apple Newton PDA, followed in the late 1990s by systems using the Palm Pilot. These all used offline synchronization techniques, making it necessary to store data temporarily on the device itself, initially until the next time the patient visited the clinic. Later on solutions were developed that allowed subjects to synchronize remotely (e.g. from home). GSM-enabled PDA devices were introduced in the early 2000s, allowing continuous synchronization (as long as the subject was within reach of a GSM network).

The one thing in common for device-based applications is that they used proprietary software installed on a commercial platform. This necessitated supplying subjects with the devices to be used for the study in question, training them in the use of the devices and collecting the devices from the subjects as they complete or withdraw from the trial.

Because device-based applications store the application itself and in many cases act as a temporary store for the data then there are special requirements that need to be addressed by these solutions.^1,3 The software must prevent end users from:

• modifying the application or the data stored on the device

• installing and using other applications that may influence the device-based ePRO application or the data collected by it

• deleting the ePRO application and using the device for other purposes

The device-based applications often use hardware specific capabilities in order to fulfill the above requirements, which results in new aspects that need to be considered:

• The ePRO software can only be used on hardware platforms that support the capabilities used^1,3

• Every release of the device-based application needs to be validated with every release of the hardware it is used on to ensure that the software operates as required (e.g. the user is still blocked from deleting data on the device) ^1,3

In summary, the present state of the art revolves around the following two major solutions provided by a number of vendors at present:

• Interactive Voice Response Systems (IVRS) that support the collection of simple data from a telephone handset

• Device-based applications executing on and adapted to a dedicated commercial hardware platform

IVRS solutions do not always provide for collection of the data needed while device-based applications involve capital investment in terms of devices, training and the logistics of distribution and collection of the devices.

PRO instruments and requirements

A PRO instrument is the collection of questions and scales used to elicit information from the subject. It is not dependent on technology as such—a PRO instrument can be implemented on paper, using an ePRO solution or both. However there is a regulatory requirement that the PRO instrument be shown to measure the correct information to support later uses of the PRO data, for example in labeling claims. Typically this is shown by validating the PRO instrument.^{1,3 ,4}

One concern has been that a PRO instrument that has been validated in one implementation (usually on paper) may not produce the same results if it is transferred to a new medium (such as ePRO). The concern has been that differences in layout, the presentation of the question, the number of questions presented at the same time, and the size of scales and other similar aspects could influence patient responses.

One large study (looking at 46 trials and 278 scales) was carried out to investigate these concerns.⁵ The conclusions reached by this study were that the responses collected from the subjects were comparable even when using different media (paper, ePRO). Other similar studies^6,7 have shown that minor changes caused by changing from one media or device to another did not adversely affect the results, but larger variations in the presentation such as rewording or reordering the instrument could result in the results not being comparable.

New technology

We now have a potential pool of subjects for clinical trials to whom the use of web-based software and mobile computing platforms is commonplace.

Web-based applications are now to be found in most users’ internet histories – buying goods and services online, social media and personal banking are web-based services now used by most of us.

Connectivity and computing power are areas that have seen a dramatic development and evolution (almost revolution) in the last 5-10 years. Smartphones and tablet computers that are more powerful than the desktop computers we used just a few years ago are gaining market shares.

According to reports from Gartner⁸ and Statista,⁹ worldwide smartphone sales in 2012 amounted to a little over 722 million units, of a total 1.746 billion mobile phones sold. The prognosis for 2013 is that smartphones will make up over half of all mobile phones sold - smartphones will account for 958 million of a total of 1.8 billion mobile phones sold. On top of this it is estimated that 180 million tablet computers will be sold in 2013, compared to 300 million PCs (a decrease from earlier years). And by 2015 tablet computer sales are estimated to reach 325 million while PC sales continue to decline. See figure 1.

Figure 1 Worldwide mobile platform market shares 2012 and 2015

So more and more of the subjects that we recruit in our clinical trials will have advanced mobile computing platforms (smartphones and/or tablet computers), platforms that are more advanced than today’s ePRO devices. Indeed the estimated sales for 2013 are a total of almost 1.14 billion smartphones and tablet computers, compared to only 300 million PCs.⁹

The standardized delivery of software installed on the client platform (computer, smartphone or tablet) has also been revolutionized by the use of “apps”, which are now even used to install software on other consumer products such as Smart TVs. This enables the easy delivery (over the internet) and installation of proprietary software on the consumer’s own device.

Two new technology solutions are therefore available when compared to earlier generations of ePRO solutions – the use of web-based applications and the use of dedicated apps.

App or Web-based Application?

What are the advantages and disadvantages of the two new technology solutions that offer us the possibility of using the patient’s own device – web-based applications and apps?

Validation

A web-based application requires validation for every supported combination of operating system (i.e. iOS, Android, Windows) and browser (i.e. Safari, Chrome, Internet Explorer, etc.). There is little or no requirement for device-specific validation.

When using apps there are still some differences between platforms and devices. The user does not have to look far to find examples of apps that run on some phones and tablets but not on others.¹⁰ So the introduction of the “app” methodology has helped standardize the software environment but the basic validation requirement is still the same—the instrument must be validated on every type of mobile phone, tablet and computer used in the trial.

Offline

The greatest single disadvantage of a web-based application is that you must have internet connectivity in order to use it. The latest HTML standard (HTML 5) has introduced limited offline capabilities, but you still need an internet connection to submit and store the data once the questionnaire has been completed.

The use of a local app allows for local storage of data and synchronization with a central database at a later time when connectivity is re-established. This is a well-established method used by existing legacy solutions and accepted by the regulatory authorities. The only major risk (which is the same for existing legacy solutions) is the risk of losing data if the device is lost or if it should break down.

Installation

A web-based application has a zero footprint on the patient device and no need for local installation on the patient device.

A local app does require installation, and although modern systems (iOS, Android and Windows) have simplified the downloading and installation of apps it still must be done. And this also brings into play a range of requirements that we mentioned in the first section regarding device-based applications:

• The need to prevent patients from modifying the app or the data stored on the device

• The need to prevent patients from installing and using other apps that may influence the ePRO app or the data collected by it

• The need to prevent patients from deleting the ePRO app

As can be seen above a web-based application simplifies the use of ePRO instruments in all cases except when an offline capability is of vital importance to the trial. Although the use of an app simplifies the distribution and installation of software and can help ensure that the ePRO instrument looks the same on all supported devices it does not address the other issues facing the legacy device-based applications, as an app is after all still basically a device-based application.

The use of app technology is an improvement on the existing legacy device-based applications, but it is not a radically new idea—it is simply a standardized environment for the distribution of, installation of and the operating system for computer software. It is a step toward the future in software development in general that started with the use of Linux (which also delivers all three of those benefits, although the use of Linux is limited for mobile computing platforms).

The future of ePRO platforms can be even brighter when considering web-based applications.

The issues

We want to collect patient reported outcomes in a fashion that ensures the data collected is correct, dependable and repeatable, in terms both of:

• Producing comparable responses from the same subject over time, and

• Producing data that is comparable between subjects.

There are a number of challenges to be faced if we want to use the possibilities presented to us by the spread of smartphones and tablet computers.

One of the most important issues is that of validation of the PRO instrument. Attempts to use the subject’s own mobile phone for ePRO have often been rejected due to problems with validation of the PRO instrument. The arguments used include:

• How does the sponsor show that the data collected supports their claims, when subjects are using different devices, with different sized screens and varying graphical interfaces?

• How can they ensure that the results are comparable except through validation of the instrument on every type of mobile phone used in the trial?

The cost of such a validation effort is prohibitive.

The Solution

The studies mentioned earlier^5,6,7 give a clue to how such a situation can be handled. Their findings indicated that minor changes in appearance of the PRO instrument still produced comparable results.  This can be leveraged by ensuring that:

• Devices with comparable capabilities are used. Smart phones and mini-tablets all have similar sized screens, similar graphic resolutions and colors.

• The PRO instrument needs to utilize a common graphical denominator that appears the same on all devices (e.g. all answer choices are shown without scrolling). When using larger tablet computers and PCs then the same limited area should be used for display as on smart phones and mini-tablets, making the appearance the same across all devices.

• The use of a single application across all devices ensures the same “look-and-feel” within the PRO instrument with regards to ordering and presentation.

• The use of a web-based application would mean there was no software installation required on the subject device.

• The use of a web-based application counteracts the need for computer system validation on each possible platform.

See figure 2.

Figure 2 A real life example showing the same PRO instrument on three different devices – a mobile phone, a tablet computer and a laptop.

The study protocol and the design of the PRO instrument should take into account the need for comparability in responses across slightly different devices, and thus avoid cases that could potentially create difficulties. The use of advanced graphical scales, such as graphical body representations (e.g. point at the part of your body that is in pain) is generally considered to be more dependent on exact equivalence in the graphical representation than textual questions and answers. To ensure compliance across multiple devices the body could be divided up into different areas (head, shoulder, etc.) that are highlighted if the subject clicks on any part of that area.

How many of the prospective subjects in our clinical trials have their own smartphones? Market analysts predict^11,12 that the major pharmaceutical markets will pass 50% market penetration for smartphones from 2012 to 2014. If your subject group contains subjects that do not own a device suitable for use in the trial, then you can use a mixed model—supply devices for subjects who do not have one of their own using proven techniques, while allowing subjects who do own their own device to use those. The advantage of a “subject’s own device” model is that it implicitly allows for varying devices to be used in the same trial. One advantage is that even if a subject changes device in midtrial (e.g. purchases a new smartphone) then data compliance is still maintained.

Regulatory Aspects

It is absolutely essential that any system used to collect data for clinical research is compliant with the regulations and guidelines covering this work. So when evaluating the use of new technology it is especially important to highlight the areas that differ from existing solutions, and whether these areas require special consideration in order to ensure regulatory compliance.

Areas to be considered when using the patient’s own mobile computing platforms are not that many.

The use of a web-based software application instead of a device-based application does not alter the fact that the software used needs to be documented and validated in exactly the same way as all software in the industry is handled. But this is well understood today, and auditors know what to look for when checking whether the vendor knows what they are doing and are following all the regulations and guidelines relevant to the industry. It is also the responsibility of the investigator and the trial sponsor to formally document a risk assessment (Quality Risk Management Plan) for the continuity of data entry when the trial subject loses their device or decides to get a new one, but this already applies even when using legacy device-based applications. So there are no extra burdens added to the industry when moving to a solution based on the patient’s own mobile computing platform.

When using a legacy device-based application it is vital that the user cannot influence either the application or the data stored locally. An important functionality (and validation step) to be considered when developing device-based applications is how to disable user access to the software and data, and validating that there is no way the user can get at the software or data.

The following problems that exist when using device-based applications are all automatically solved by the use of a web-based application:

• Loss of data due to loss of device or device malfunction

• Collection of incorrect data due to the latest protocol amendment not being implemented on the device

The solution of these issues for device-based applications involves additional software, and therefore additional validation effort and additional risk.

Using the patient’s own mobile computing platform provides substantial savings from a regulatory compliance point of view. There is no software installed on the remote device, nor is any data stored on the device, not even temporarily. So the fact that we are using the patient’s own device then becomes almost unimportant – as long as the device supports the web-based application then no further validation is required. Platform support can be programmed into the web-based application itself in the form of requiring certain versions of given browsers – if they are available on the patient’s device then there is no problem. The use of the patient’s own device then becomes directly analogous to the use of a telephone in an IVRS system – there are no requirements to validate IVRS systems against all possible telephones in all countries in the world, it is enough that standard telephone functionality is available to the subject.

Summary

What are the advantages that a subject’s own device solution offers? The most important advantages were named in the opening paragraph, namely:

• Provisioning – as the subject would be using their own device there is little additional cost for the trial. In some countries the subject may need to be reimbursed for data connection fees, but in many countries subjects have mobile plans that include free data (or free up to a certain amount of data per month).

• Supply – no supply is necessary, as the subject already has their device. The site need not handle any devices at all. This is a major area of concern in trials using legacy device-based applications, with regards to cost, project setup times and overall project logistics.

• Training – the subjects are already “trained” in both their device and with the web-based applications they presently use. Only a minimal amount of training is required in the PRO instrument itself, probably even less than the training they would receive if the PRO instrument had been on paper (as they do not have to be instructed in where to return the paper afterwards).

• Maintenance – Any repairs/replacements would be handled by the subject directly with their dealer, and neither the Sponsor nor the sites need to be involved. As there is no software or data stored on the subject device, if the subject should need to return their usual device for repair they can then use an alternative or new device to enter data without dropping out of the trial.

When would the legacy IVRS and device-based applications be more suitable?

IVRS solutions do not require a mobile computing platform, they operate on any telephone. In this respect they are still applicable for all potential subjects that have access to a telephone, but not to a smart phone, tablet computer or PC. This is currently a large, but diminishing, proportion of the overall pool of subjects.

Device-based applications can still be the solution of choice for trials with specific requirements for a uniform hardware solution. One example is a requirement to connect to external equipment at the subject’s residence, such as PEF meters and blood pressure cuffs.

The future is already here

It would appear that there are few, if any, insurmountable problems with the use of the subject’s own device. If the study protocol and the PRO instrument have been designed with this in mind then the ePRO comparative studies already conducted^1,6,7 indicate that the subject’s own device can be used.

Traditionally large corporations in the clinical research sector exhibit a certain resistance to adopting new technologies, but are there any regulatory or other substantial concerns that would contraindicate adopting the patient’s own mobile computing platform for ePRO? As can be seen from our discussion above, the answer is no.

So why isn’t this already being done? Actually it is—all around the world trials are presently being run that collect e-PRO data in this fashion, including trials making regulatory submissions. Both the FDA1 and the EMA4 have issued guidelines and reflection papers, which outline their current thinking when it comes to compliant use of ePRO. Almost weekly, new announcements are made on the internet from vendors both small and large offering this type of solution.

Three studies using a web-based application on the patient’s own mobile computing platform from different parts of the world that the author is aware of are:

• A Phase II clinical trial in the United States testing the use of a new pharmaceutical designed to increase sexual desire, arousal and satisfaction in females with sexual desire disorder. The ePRO data contains primary efficacy data as the measure of success of the treatment is heavily dependent on the qualitative responses from the subjects. The trial included over 200 subjects at more than 15 sites in the United States. The data that has been collected is being prepared for submission to the FDA.

• A medical device trial in Europe to evaluate an additive for pain relief in a plastic surgery product used for cheek shaping. Again the ePRO data contains primary efficacy data as the measure of the degree of pain relief is heavily dependent on the qualitative responses from the subjects. The trial included more than 50 patients at three sites.

• An investigator initiated Phase IV study in Japan to evaluate efficacy and safety of three types of hyaluronic acid injections into patients with osteoarthritis of the knee. The ePRO data collected is a Quality of Life questionnaire containing the WOMAC scale. The trial included more than 600 patients at 30 sites.

So the final word is that if you design your study protocol and your PRO instrument with the aim of it being comparable across different devices, and choose your study population such that the subject’s own device can be used for data collection then you can run one of the new breed of ePRO trials already out there.

References

1. U.S. Department of Health and Human Services. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. December 2009.
2. Vendors of e-PRO device-based solutions:
   PHT Corporation – http://www.phtcorp.com
   eResearch Technology (incorporating Invivo Data and Clinitrac) –http://www.ert.com/
   CRF Health – http://crfhealth.com
3. European Commission Health and Consumers Directorate-General. Annex 11: Computerised Systems from EudraLex, The Rules Governing Medicinal Products in the European Union, Volume 4 Good Manufacturing Practice, Medicinal Products for Human and Veterinary Use
4. EMA Committee for Medicinal Products for Human Use. Reflection paper on the regulatory guidance for the use of health related quality of life (HRQL) measures in the evaluation of medicinal products; EMEA/CHMP/EWP/139391/2004
5. Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health 2008;11:322-33.
6. Coons SJ, Gwaltney CJ, Hays RD, et al. ISPOR ePRO Task Force recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value Health 2009; 12:419-29.
7. Juniper EF, Langlands JM, Juniper BA. Patients may respond differently to paper and electronic versions of the same questionnaires. Respir Med 2009;103:932-4.
8. Gartner: Gartner Says Worldwide Traditional PC, Tablet, Ultramobile and Mobile Phone Shipments On Pace to Grow 7.6 Percent in 2014
   http://www.gartner.com/newsroom/id/2645115
9. Statista: Global smartphone shipments from 2010 to 2017 (in million units)
   http://www.statista.com/statistics/263441/global-smartphone-shipments-forecast/
10. “Fitbit” is an example of a common App available for both iOS and Android that does not support all devices on either platform, see:
    http://www.fitbit.com/uk/devices
11. Nielsen: MOBILE MAJORITY: U.S. SMARTPHONE OWNERSHIP TOPS 60%
    http://www.nielsen.com/us/en/newswire/2013/mobile-majority--u-s--smartphone-ownership-tops-60-.html
13. eMarketer: Smartphone Adoption Tips Past 50% in Major Markets Worldwide
    http://www.emarketer.com/Article/Smartphone-Adoption-Tips-Past-50-Major-Markets-Worldwide/1009923#BDPmZj33rlE8fTVX.99