ePRO vs. Paper

Studies show that electronic patient reporting yields higher quality data than paper-based diaries and that ePRO elicits significantly greater subject compliance, sometimes as high as 97%.^1-5 Importantly for clinical research sponsors, both FDA and EMA have signaled their acceptance of ePRO study data.^6,7 The number of trials using electronic data capture (EDC) and ePRO has increased a good deal during the past 15 years. Yet many clinical researchers continue to use paper-based questionnaires.

Estimates of the overall expense of bringing a drug to market ranged from $800 million to $2 billion.⁸ Depending on the product under investigation, clinical trials can account for 50% to as much as 65% of that cost. So it's no surprise drug companies have an interest in cutting costs.^9,10

When a clinical trial protocol calls for subject reports of treatment outcomes, the sponsor's project team will determine the kinds of outcome measures it needs to collect from the participants. Health outcome specialists either choose an appropriate standard PRO instrument. If none is available, specialists begin the process of creating a new instrument. At the instrument design phase it makes little difference whether the questions will be asked on a paper diary or using ePRO. Once the questions have been defined, however, the choice of data collection device depends on a wide variety of factors. The end point, the number and frequency of responses required, and the population and indication being studied are among the many issues that can affect the cost.

Start-up costs for an ePRO-based study may be as high as 10 times the cost of printing and distributing paper diaries. But sponsors that look no further than the start-up investment can miss a critical opportunity to cut the overall cost of their study. Sponsors that look at the long-term return on investment can find that adopting ePRO technology plays a significant role in reducing the overall cost of their drug development program.

Resistance to change

Nearly every company, laboratory, school, and home in the developed world uses the Internet. Scientific research is critically dependent on electronic technology. Yet pockets of technology resistance remain in the pharmaceutical and clinical trials industries. The relatively slow growth of ePRO adoption in the pharmaceutical industry is a surprising example of that resistance.

Some possible reasons for that are the increasing number of studies conducted in countries where many people cannot read and are not technology savvy and sometimes poor communication between a sponsor and vendor. Most resistance, however, seems to be a function of corporate culture or "techno-fatigue." In many cases, the corporate culture has a legacy practice of using paper-based data collection. Some organizations have a long history of working with paper-based systems that have a record of what they consider to be good results. Such companies see no need for change.

Then there's "techno-fatigue." Changing the perceptions of busy professionals who are immersed in their own work is a challenge. The demands on their time and attention are already daunting. No one likes to feel behind the times, so they may not admit to their techno-fatigue, but their resistance is understandable. Researchers tend to be thoughtful people who take a long-term view; some can find the dizzying pace of technological change disruptive. The technology they just mastered is likely to become obsolete within months. Researchers who often spend 10 to 15 years working on a single project see no need for change.

Cost-cutting change

A major way that ePRO technology can cut the cost of a trial involving patient-reported outcomes is by improving compliance rates, which improves data quality. Another advantage is the way ePRO automates data management functions: reviewing, collating, verifying, and archiving data.

The FDA is particularly interested in the way a patient diary or other form of unsupervised data entry is used. The agency's new guidance document on using patient-reported outcomes to support labeling claims states, "[W]e plan to review the clinical trial protocol to determine what steps are taken to ensure that patients make entries according to the clinical trial design and not, for example, just before a clinic visit when their reports will be collected."⁵ Electronic diaries specify specific time windows for patient entries, and their time stamps document those entries.

A paper-based trial is more likely than one using ePRO to have incomplete, misleading, conflicting or falsified diary entries. Such inaccuracies can force the sponsor's study team, data managers, and site personnel to spend countless hours reviewing questionnaires, entering data, and seeking follow-up information. It can be weeks before errors on a paper questionnaire are discovered and, by then, it is too late to retrieve missing or corrected data. Labor costs for a paper-based study can eclipse the originally projected ePRO costs and erase the savings projected for the use of paper. An ePRO instrument itself can encourage compliance. Lightweight and portable PDAs, netbooks, and smartphones provide convenience, privacy, and confidentiality. Trial subjects can easily carry their eDiaries with them during all their everyday activities: going to work, running errands, traveling. With the device at the ready throughout their day, people respond to the alarms and, consequently, comply with protocol criteria. When an ePRO instrument sounds an alarm to remind a study subject to record data, even a distracted participant can provide accurate data. The device can be programmed to instantly question an out-of-range, contradictory or invalid response. Today's ePRO specialists—using touch screens, intuitive page designs, and sophisticated graphical interfaces—design compelling questionnaires that trial subjects find interesting, and even enjoyable. That keeps patients motivated.

Because ePRO reports are sent and reviewed in real time, the potential for lost, missing, and noncompliant data is significantly reduced. At the close of a study, ePRO data is already in the study database, speeding study close-out.

With compliance rates in ePRO trials at 90% to 97%,^1-5,11 sponsors have further evidence that ePRO can help control costs. At best, the compliance rate for paper-based trials is around 30%; it's generally lower. With documented high compliance rates and confidence in the accuracy of their data, sponsors of ePRO studies can often collect sufficient data from fewer subjects and investigative sites.

There is no doubt that the cost to purchase and program 300, 500 or 1000 electronic devices is substantial. But those dismayed at the initial investment need to consider the fact that many times standardized ePRO components can be recycled. For example, an electronic questionnaire can be designed once, stored in an electronic library, and retrieved for use in subsequent studies. That not only makes everything more standard, it also allows the sponsor to avoid some development and validation costs. As the devices and components are used for a second, third, and fourth study, the sponsor sees study set-up costs slashed.

Along with ePRO's capability to satisfy regulators' documentation standards, sponsors have other compelling reasons to embrace change: staying a step ahead of other companies in a highly competitive arena while cutting costs.

Numbers count

Rigorous proof-of-concept research is an integral part of drug development, and wise managers apply the same disciplined approach to determining what "slashing" costs means in dollars. To this end, we developed a model for demonstrating the return on investment (ROI) for ePRO studies.

The paper vs. ePRO model first collects the key cost drivers for paper trials (see Table 1). Then, using the calculated volume of data, the model looks at site activities, monitoring, and data management processes to calculate an estimated cost for completing the paper-based activities (see Table 2). These costs can then be compared to similar costs for running the study using ePRO (see Figure 1). The model reveals some paper cost items that do not come from the study budget and, consequently, shows a more accurate average break-even point.

Economists may be comfortable thinking in millions and billions, but the rest of us work better with numbers more familiar in everyday life. A clinical trial manager may ask how many subjects a study needs to enroll to make an ePRO diary cost-effective. The real cost driver for paper studies, however, is the volume of data collected. Consequently, a more logical question is, "How many data pages does it take?" Even the answer to that question—39,551—is less than fully informative without a concrete example.

Assume that a study's protocol requires that 200 subjects complete a diary once a day for six months (182 days): 200 x 182 = 36,400. In this example, cost estimates are based on industry averages, our own experience, and interviews with clinical data management professionals. The graph shows the linear regression of number of pages (x axis) and cost (y axis) for ePRO and paper data collected (see Figure 2). With paper, costs rise more steeply. So, for paper-based data collection, that makes for a stronger correlation between the volume of data collected and cost progression. We found that ePRO is on average more cost-effective after the average break-even point, 39,551 pages of data.

With increases in the number of sites and the duration of the trial, the volume of paper generated rises, and so does the potential cost savings of using ePRO. Revisiting the example above, assume that each subject is required to make two entries per day. That doubles the number of data pages to 72,800—a number well past the break-even point. That assessment of cost-effectiveness favors ePRO even when additional costs are "invisible" to the sponsor's study team. The sponsor provides resources (e.g., data management) that the sponsor's project team may see as "free." The site often adds personnel dedicated to catching up with collating the paper data, reviewing entries, and entering data into the sponsor's or CRO's electronic data capture system. Although those resources are not shown in a specific study's budget, they are unquestionably a part of the sponsor's overall costs.

The model we use includes many items that don't normally appear in the budgets for specific studies. That makes it easier for sponsors to estimate the total cost of running a paper-based study or the same study electronically.

ePRO's future

Most industry analysts already saw ePRO as a tide that cannot be turned. Even where ePRO is less than fully embraced, the concept and benefits are well-established, and market forces may have eventually forced ePRO holdouts into the 21st century even before the new return on investment model made it easier to calculate savings.

PDA sales continue to rise, and at least one source projects that smartphone shipments will number 164 million in 2009, up from 13% from 2008.^12,13 Some analysts predict that sales of netbooks—small laptops designed primarily for interfacing with the Web—will match or even surpass those numbers in 2010.¹⁴ These tools, along with other popular mobile devices, are making it easier than ever to collect clinical information from patients.

What might appear as a sudden, sweeping change in collecting PRO data, however, will simply be the sum of incremental, sometimes imperceptible, movements. The domination of ePRO is slow in coming, but now appears inevitable.

Nadina C. José, MD, is a former Clinical Advisor with CRF Health, and Kai Langel* is Senior Systems Specialist with CRF Health, Helsinki, Finland, email: [email protected].

*To whom all correspondence should be addressed.

References

1. V. Arnera, "Why Paper Diaries Should Be Banned in Clinical Trials," Pharmaceutical Executive Europe Digest, March 2009, http://www.slideshare.net/challPHT/why-paper-diaries-should-be-banned-in-clinical-trials.

2. T. Hair, "PROactive Patient Reporting," Good Clinical Practice Journal, 27-28 (February 2006).

3. M.R. Hufford, A.A. Stone, S.Shiffman, J.E. Schwartz, J.E. Broderick, "Paper vs. Electronic Diaries: Compliance and Subject Evaluations," Applied Clinical Trials, 38-43 (August 2002).

4. A.A. Stone, S. Shiffman, J.E. Schwartz, J.E. Broderick, M.R. Hufford, "Patient Noncompliance with Paper Diaries," British Medical Journal, 324, 1193-1194 (2002).

5. "Clinical Trials Compliance Triples With Electronic Diaries," Applied Clinical Trials Online, June 2008, http://appliedclinicaltrialsonline.findpharma.com/appliedclinicaltrials/EDC/Clinical-Trials-Compliance-Triples-With-Electronic/ArticleStandard/Article/detail/523162.

6. Food and Drug Administration, Guidance for Industry, Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims (FDA, Rockville, MD, December 2009).

7. European Medicines Agency, Reflection Paper on the Regulatory Guidance for the Use of Health-Related Quality of Life (HRQL) Measures in the Evaluation of Medicinal Products (European Medicines Agency, London, UK, July 2005).

8. N. Masia, "The Cost of Developing a New Drug," in Focus on Intellectual Property Rights, U.S. Department of State, Bureau of International Information Programs, 2006, pp 82-83, http://www.america.gov/st/econ-english/2008/April/20080429230904myleen0.5233981.html.

9. J.A. DiMasi and H.G. Grabowski, "The Cost of Biopharmaceutical R&D: Is Biotech Different?" Managerial and Decision Economics, 28, 469-479 (2007)

10. PhRMA, What Goes Into the Cost of Prescription Drugs?...and Other Questions About Your Medicines (PhRMA, Washington, DC, 2005) http://www.phrma.org/files/attachments/Cost_of_Prescription_Drugs.pdf.

11. K. Mussina, "The ePRO Evolution," Next Generation Pharmaceutical, March 2006, http://www.ngpharma.com/article/The-ePRO-evolution.

12. D. McGrath, "Analysis: Smartphones Throw Suppliers a Lifeline," EE Times, February 2009, http://www.eetimes.com/showArticle.jhtml;jsessionid=PTIWBB4WJOEQDQE1GHRSKH4ATMY32JVN?articleID=213300761&printable=true&printable=true.

13. "LG Projects 2010 Mobile Phone Sales at 140 Million," The Economic Times, January 2010,http://economictimes.indiatimes.com/news/international-business/LG-projects-2010-mobile-phone-sales-at-140-million/articleshow/5441372.cms.

14. P. Chubb, "Netbook 2009 Sales and 2010 Expectations," ProductReviews, December 2009, http://www.product-reviews.net/2009/12/23/netbook-2009-sales-and-2010-expectations.