Electronic Diaries, Part 2: The Role of the Clinical Protocol in Developing and Implementing Electronic Diaries

Subject diaries are used in about one-fourth of all clinical trials to collect data on primary and secondary endpoints.¹ Part 1 of this four-part series on developing and implementing subject diaries compared and contrasted traditional paper and electronic diaries.²

Teri Stokes

In that article, we examined both methods in the context of existing FDA and ICH regulations and guidance for capturing diary data. We concluded that newer, electronic methods are in fact more compliant with FDA and ICH data standards than traditional paper diaries. The rest of this series focuses solely on electronic patient diaries (EPDs), in an effort to help clinical researchers better understand the process of implementing an electronic system to be used in their clinical trials.

In our experience, clinical teams evaluating the use of an electronic diary system often express that they

• are uncertain about the regulatory compliance of electronic systems (see Part 1).

• do not know how an electronic system is specified, built, and deployed.

• are unsure about the effect of including an electronic diary in the clinical trial development process and timelines.

• wonder what changes an electronic diary will require from the participants in the clinical trial process-sponsor staff, monitors, investigators, and subjects.

To address these concerns, this and the following articles in the series present the clinical, technical, and regulatory issues involved in the development and implementation of EPD trials that meet regulatory requirements. Although clinicians have reasonable concerns about adopting technology, we plan to demonstrate that the collaboration of clinical and technology teams can result in a robust clinical trial executed in a timely fashion.

Jean Paty

Our experience is primarily with EPD systems implemented on handheld computing devices (personal digital assistants, or PDAs). Consequently, such systems are used as the example in the rest of the series. There are, however, a number of other ways to implement EPDs-interactive voice response systems (IVRSs), for example. Although such systems are not the focus of our discussion, the principles discussed here can be applied to a broad array of subject-based data collection systems, including IVRS.

Protocol development for an EPD study

The first step for any clinical trial is to develop a protocol. For studies that make use of an electronic diary component, it is important to understand the role of the protocol in developing an EPD system that delivers data that meets FDA and ICH electronic data regulations and guidance. Along with our discussion of the role of the protocol in developing an EPD system, we discuss the roles and responsibilities of the participants in a clinical trial. Finally, we mention the software development life cycle (SDLC) approach to the development and deployment of an EPD study-an approach that will get more attention later in this series.

Here, we address several questions about regulatory issues in subject diary research.

• What is the process of producing a protocol for an EPD study?

• What role does the clinical protocol play in the development of an electronic subject diary system?

• What are the sponsor, investigator, and subject roles in an EPD study?

• How do the clinical trial life cycle and the system development life cycle interact in the design, development, and deployment of electronic subject diaries?

Producing a protocol

Clinical-regulatory perspective (Paty).

Fundamentally, preparing a protocol for an EPD study is much like preparing a protocol for any traditional subject diary study. The clinical team responsible for the trial delineates the primary and secondary objectives, and then documents the trial design, treatment of subjects, and assessment of efficacy and safety in the protocol-per section 6 of the ICH Guideline for Good Clinical Practice.

The key difference between the protocol for an EPD study and one using a traditional paper diary is that EPD technology enables researchers to implement novel study designs. To better understand how EPD systems enable new study designs, it is first useful to consider the factors that influence the study designs typically used in a paper diary trial.

A slight departure

All subject diaries aim to directly capture the subject's experience at a medical moment. We use the term medical moment to describe the subject experience prescribed for capture by the clinical protocol. Longstanding clinical lore, now supported by empirical studies,^3-5 maintains that a number of subjects will not complete their paper diary, will give out-of-range answers, and/or will complete the diary long after-sometimes even before³-the relevant medical moment.

In response to these known issues with paper diaries, clinical teams often reduce the scope of study objectives or the amount and type of data to be collected in a diary protocol. Studies using paper diaries, for example, often ask subjects to complete their diaries once a day.⁶ The end-of-day diaries often ask subjects to report on several events that occurred during the day (for example, micturitions in urinary incontinence trials) or to aggregate their experience for the day (for example, average pain level in arthritic subjects). Because we ask subjects to retrospectively recall a number of brief events and/or to summarize their experience, such designs are vulnerable to recall-based bias and error.⁷ A number of studies have shown that even these brief periods of recall can be biased and inaccurate.^{8, 9}

To optimally capture brief medical moments, subjects should be able to enter data in real time, at the occurrence of the medical moment.¹⁰ This approach yields the most reliable and accurate data, and therefore allows the strongest test of the study objectives.¹¹ For example, the most accurate count of daily micturitions would be derived from having subjects make entries when the micturitions occur, rather than at the end of the day. The most accurate reflection of daily pain would be an aggregate of multiple momentary pain assessments captured across the day.¹² Such real-time protocols have been successfully implemented with electronic subject diaries; for examples, see Hufford and Shields.⁶ Thus, by facilitating real-time recording of events, electronic diaries can extend diary study designs in clinically significant ways.

Figure 1. The user requirements specification (URS) for e-diary design.

Technical-regulatory perspective (Stokes). The use of electronic subject diary technology gives the clinical research team an array of options to support the quality, integrity, and regulatory compliance of self-reported data. The use of electronic diary technology should also enhance and expand the medical horizon for a clinical protocol and not restrict it to an electronic version of past experience with paper diaries. To accomplish this, the clinical protocol must closely describe the subject's experience in the medical context and define experience reports and collection time strategies in detail.

In other words, the clinical protocol must identify what medical moments the subject should observe, how the subject is to measure or describe those medical moments, and when the subject is to report the medical moments. By doing this, the clinical protocol provides the basis for some of the key sections of a user requirements specification (URS) for an electronic diary (Figure 1). The clinical team may choose to include sufficient details in the protocol to make it act as the URS, but most often a separate URS document is developed with the aid of a technical team. The URS document contains the technical details necessary to develop the EPD for that trial. The URS is based on a final protocol, and typically developed after the protocol has been fully specified.

Protocol's role in developing an EPD system

Clinical-regulatory perspective (Paty).

The clinical protocol, as stated above, provides the basis for developing the technical specifications for the EPDs. The protocol thus acts as the first mover in the development of the EPDs, but that fact need not change the general process of protocol development for clinical researchers. The primary difference required for an EPD protocol is the addition or modification of sections that describe the computerized system to be used in the trial.

¹³

This may include the following protocol sections from section 6 of the ICH Guideline for GCP: inclusion/ exclusion criteria for subjects (6.5.1 and 6.5.2), assessment of efficacy and/or safety (6.7 and 6.8), and treatment of subjects (6.4.4).

¹⁴

(Also see

what

,

how

, and

when

components above.).

Glossary

The inclusion/exclusion criteria under subject selection should include a statement regarding subjects' ability and willingness to comply with use of the data collection device to record their medical moments (as would be the case in any trial). When considering inclusion criteria, a frequently stated concern is that elderly subjects will be unable to use EPD systems. Recent comparisons clearly show that elderly subjects perform as well as, and in some cases better than, younger subjects.¹⁵ Another inclusion criterion that can be used in an EPD trial is timely completion of the diary, which can be measured by comparing the date and time stamps on the EPD data stream against what is expected of the subject.

The assessment of efficacy and/or safety sections of any protocol will include descriptions of what medical moments are to be captured and what is to be measured at those moments. A protocol that includes an EPD component should describe how the PDA will be used to capture the medical moments, when the subjects should use the device to record those moments, and what measures will be implemented on the system. The use of technology-specific features, such as real-time edit checks of data, might also be mentioned.

In some trials, the PDA might be used to capture information about the study medication in addition to efficacy and safety data, and that would also be discussed in the protocol. When the URS is a separate document, those protocol sections can include a general description of the use of the EPD system, and leave the technical details to the other documents generated in developing the system.

Figure 2. The functional requirements specification (FRS) for e-diary design.

Technical-regulatory perspective (Stokes). The sponsor's protocol should specify the rationale for collecting data in the context of the medical problem to be solved. It should specify the type of data to be collected to describe a given medical moment, the timing and sequence of data collection, and the clinical logic and diagnostic inclusion/exclusion rules for measuring data integrity and accuracy within the context of the medical problem being studied.

The URS is a high-level specification that is developed jointly by the clinical and technical teams. It should list the requirements for the system in a manner and language understood by the clinical team. The URS should briefly identify the data flow from subject to investigator to sponsor and the types of interaction between those three. If a trusted third party is involved, their data and device roles need to be identified. The URS should also define the geographic scope of a clinical trial, because time stamps for data must include information about time zones.¹⁶ To a greater technical degree than the protocol does, the URS specifies the what, how, when, and why of data to be self-reported by the subject.

The technology team's response to the URS is a detailed functional requirements specification (FRS) that will be used by the software developers of the EPD system. An active dialogue between the protocol team and the diary technology team is essential to develop a good functional specification. Protocol teams that are used to thinking in terms of a static paper diary often need support in applying the new tools available to them when using EPDs. Support may be needed both for understanding the new options available with technology and for the discipline of plotting out the logic of diary data pathways in the protocol and URS.

The FRS should address such factors as logon security, range checks for numerical data, spell checks for text data, logic checks for data entered over time, time stamps, and audit trails (Figure 2). While no technology can prevent humans from entering deliberate misinformation, accurate time stamps can prevent humans from recording any data at an unacceptable time or in an unacceptable sequence according to the protocol. Electronic alarms can prompt people to make entries as scheduled by a protocol. Logon password security can prevent unauthorized entry of information. Electronic field checks can verify that the expected type of data has been recorded (such as numeric data within a specified range or analogue data to a particular scale or text data in defined multiple-choice or free-format fields). Such electronic checks support both data quality (attributable, legible, contemporaneous, original, accurate)¹³ and regulatory compliance to 21 CFR 11 for electronic record keeping.¹⁷ The FRS should address each of these components.

9 Steps in the Subject Diary Life Cycle (SDLC)a

Thus, the protocol is the basis of the URS and FRS documents, which in turn guide the software/system development life cycle (SDLC) shown in Figure 3.

Sponsor, investigator, and subject roles

Clinical-regulatory perspective (Paty).

The roles and responsibilities of the sponsor, investigator, and subject remain largely the same for an EPD trial as for any other clinical trial. For example, the sponsor still provides ongoing monitoring of the investigative sites (per the ICH guideline for GCP, 5.18). The procedure for monitoring the diary portion of the trial is quite different, however, because there is no need to review paper diary pages. The data, already in electronic form, is less vulnerable to common problems with handwritten diaries, such as out-of-range responses, illegible comments in margins, and blank diary pages. Thus, the clinical monitor is freed to focus on the actual quality and integrity of the data. The monitor's role can be facilitated by summary versions of the electronic diary data that show the subjects' compliance with the protocol procedures.

Figure 3. The system development life cycle (SDLC) for an EPD.

In any clinical trial, the investigator must "prepare and maintain adequate and accurate case histories that record all observations and other data pertinent to the investigation on each individual administered the investigational drug or employed as a control in the investigation" (21 CFR 312.62(b)16). In an EPD trial, the source data is the responsibility of the investigator. If the electronic diary data is transferred to a central repository away from the investigative site, it is important for the investigator to maintain access to and responsibility for such data.^19-21 The sponsor can employ a trusted third party to act on behalf of the investigator to maintain the source data.^20,21 The investigator should have 24/7 access to the data (through a secure Internet connection, for example), and should authorize any changes to the source data. Exactly how this is achieved may be different from one system to another. The investigator, like the monitor, is also freed from the review of pages of paper diaries and can attend to the treatment of the clinical subjects.

Finally, the subject in an EPD study provides the data directly-as opposed to an investigator taking measurements, such as blood samples, from the subject. With an EPD, as with a paper diary, training is necessary so that the subject clearly understands how the diary is used to collect study data. The simpler the interface on the electronic diary, the more quickly subjects can be trained and the more likely they are to comply with the protocol procedures.

Although researchers new to EPD often express concern about subject acceptance of the technology, a number of studies have shown that subjects prefer electronic over paper diaries.^22,23 As a result, their compliance with completing diary entries can be very high.⁶

Technical-regulatory perspective (Stokes). The sponsor, in collaboration with the technical team (if outside of the sponsor), has overall responsibility for ensuring that relevant technical components are in place for the trial. As stated above, the FDA Guidance for Industry: Computerized Systems Used in Clinical Trials¹³ identifies electronic subject diaries as collecting electronic source records and the ICH guideline for GCP (4.9) identifies the investigator as responsible for all subject source records.¹⁴ The sponsor, represented by a combination of the clinical and technical teams, needs to translate these regulatory directives into system functionality for data access, review, query resolution, reporting, and retention for audit and inspections. Of course, broadly, these responsibilities are the same for any subject self-report study, whether using paper or technology.

Figure 4. The clinical trial life cycle (CTLC).

The sponsor designates the roles and responsibilities for review, query resolution, analysis, and reporting of the diary data. In an EPD study the roles and responsibilities are reflected in the assignment of system roles and access to the database of diary data. Monitors, for example, should only have read-only access to the source diary data, because the data is the responsibility of the investigator, and that would be clearly specified in the documentation on system privileges (21 CFR 11).¹⁶

The format must also be defined for export of subject data from the diary database to the sponsor's clinical data management or statistical analysis systems. The physical and logical management of diary devices and data across CROs or other third parties involved in the trial needs to be defined. Procedures for data clarification, assistance with technology problems, and emergency response for medical questions also need to be defined. Finally, training of the investigators and monitors is also the responsibility of the sponsor.

The sponsor (or a technology representative) needs to train the investigator's staff in how to use the diary and how to teach its use to subjects. Once trained, the investigator's staff trains subjects in how to use the electronic diary, how to transfer the data back to the host server, if appropriate (which could be as simple as putting the PDA into a transfer device at night), and the security measures to follow when using the PDA (such as safe storage of the device and use of a subject password). The investigator's staff, with support from the sponsor, will assign and track device ID codes with subject ID codes for proper authentication and confidentiality of diary data.

9 Steps in the Clinical Trial Life Cycle (CTLC)a

The sponsor's clinical research associates or the CRO's medical monitors need to be trained in all aspects of the investigator and subject roles so that they can continue ongoing training to support investigator sites.

The subjects in an EPD trial, as mentioned above, directly provide the data. If subject data is to be sent across borders and across organizations for review and data processing by people outside the investigator site, the subject's written consent is required. This is in accordance with the EU Privacy Act of February 1995 (Directive 95/46/EC) and for the USA after April 2003, it is the medical research provision of the Hospital Insurance Portability and Accountability Act (HIPAA, June 2002).²⁴

The clinical trial life cycle and the system development life cycle interact in the design, development, and deployment of electronic diaries. The complete development and implementation of an EPD trial is illustrated in Figures 3 and 4. Figure 3 shows the life cycle-development, implementation, and wrap-up steps-for developing the EPD system. Figure 4 shows the life cycle of a clinical trial. The table summarizes in narrative form the steps depicted in Figures 3 and 4. In an EPD study, the two life cycles are interdependent.

The protocol is key to success

This discussion shows that two perspectives, technical systems and clinical science, converge on a common message: A well-defined clinical protocol is key to the success of an electronic subject diary system, and it takes close collaboration of a sponsor's clinical and technical teams to develop a suitable URS based on the protocol.

At the tactical level of building an EPD trial, the clinical team conducts its normal development tasks to prepare for a study, while working interactively with the technical team on the preparation of the EPD system. The development of the EPD system follows a system development life cycle (SDLC, Figure 3), just as any other regulated computerized system does. This SDLC occurs concurrently with the normal clinical trial life cycle (CTLC, Figure 4). Although this dual track approach can be challenging for the first EPD study for a clinical team, both software and clinical trial development cycles can and should run in parallel to yield a successful and timely study launch, execution, and completion.

It is perhaps most important that the clinical team appreciate its role in defining the medical moments and the way they are to be self-reported as the initiating step in the development of the EPD, and to give careful consideration to the new ways that technology can enhance their trial design.

The next article in this series (Part 3) will discuss in greater detail how electronic diaries are validated and the respective roles of the clinical and technical teams in working to achieve data quality and regulatory compliance.

References

1. DataEdge, 1999. Unpublished data regarding frequency of use of diaries in clinical trials.

2. T.S. Stokes and J.A. Paty, "What Is a Subject Diary and How Do Regulations Apply?" Applied Clinical Trials, September 2002, 38-43.

3. Arthur Stone, Saul Shiffman, Joseph Schwartz, Joan Broderick, and Michael Hufford, "Patient Non-compliance With Paper Diaries," British Medical Journal, 324: 1193-1194 (2002).

4. B. Tiplady, G.K. Crompton, M.H Dewar, F.G.E. Bollert, S.P. Matusiewicz, L.M. Campbell, and D. Brackenridge, "The Use of Electronic Diaries in Respiratory Studies," Drug Information Journal, 31: 759-764 (1997).

5. M.E. Hyland, C.A.P. Kenyon, R. Allen, and P. Howarth, "Diary Keeping In Asthma: Comparison of Written and Electronic Methods," British Medical Journal, 306: 487-489 (1993).

6. Michael Hufford and Alan Shields, "Electronic Diaries: Applications and What Works in the Field," Applied Clinical Trials, April 2002, 46-56.

7. S. Shiffman, "Real-Time Self-Report of Momentary States in the Natural Environment: Computerized Ecological Momentary Assessment," In A.A. Stone et al., Eds., The Science of Self-Report: Implications for Research and Practice (Lawrence Erlbaum Associates, Mahwah, New Jersey, 2000), pp. 277-296.

8. D. Redelmeier and D. Kahneman, "Patients' Memories of Painful Medical Treatments: Real-Time and Retrospective Evaluations of Two Minimally Invasive Procedures," Pain, 66: 3-8 (1996).

9. A.A. Stone, J.E. Schwartz, J.M. Neale, S. Shiffman, C.A. Marco., M. Hickox, J. Paty, L.S. Porter, and L.J. Cruise, "A Comparison of Coping Assessed by Ecological Momentary Assessment and Retrospective Recall," Journal of Personality and Social Psychology, 74: 1670-1680 (1998).

10. O. Nived, G. Sturfelt, S.A. Eckernas, and P. Singer, "A Comparison Of 6 Months' Compliance Of Patients with Rheumatoid Arthritis Treated with Tenoxicam and Naproxen. Use of Patient Computer Data to Assess Response to Treatment," Journal of Rheumatology, 21: 1537-1541 (1994).

11. Saul Shiffman, Michael Hufford, and Jean Paty, "Subject Experience Diaries in Clinical Research, Part 1: The Subject Experience Movement." Applied Clinical Trials, February 2001, 46-56.

12. E. Eich, J.L Reeves, B. Jaeger, and S.B. Graff-Radford, "Memory for Pain: Relation Between Past and Present Pain Intensity," Pain, 23: 375-379 (1985).

13. Food and Drug Administration, Guidance for Industry: Computerized Systems Used in Clinical Trials (FDA, Rockville, MD, May 1999; also available at www.fda. gov/cder/guidance/index.htm).

14. Guideline for Good Clinical Practice, International Conference on Harmonisation, Federal Register 62 (901) 25691-25709 (9 May 1997); also available from EMEA, 7 Westferry Circus, Canary Wharf, London E14 4HB, UK, or www.eudra.org/humandocs/PDFs/ICH/013595en.pdf.

15. S. Shiffman, unpublished data.

16. Food and Drug Administration, Draft Guidance for Industry: 21 CFR Part 11; Electronic Records; Electronic Signatures, Time Stamps (FDA, Rockville, MD, February 2002; also available at www.fda.gov/OHRMS/DOCKETS/98fr/00d-1542_gdl0001.pdf).

17. Code of Federal Regulations, Title 21 CFR, Part 11: Electronic Records; Electronic Signatures; Final Rule, Federal Register, March 20, 1997.

18. Food and Drug Administration, Draft Guidance for Industry: 21 CFR Part 11; Electronic Records; Electronic Signatures, Validation (FDA, Rockville, MD, August 2001; also available at www.fda.gov/OHRMS/DOCKETS/98fr/001538gd.pdf).

19. Paul Bleicher, "e-source: To e or Not to e," Applied Clinical Trials, May 2001, 36-38.

20. Paul Bleicher, "eSource Redux," Applied Clinical Trials, August 2002, 30-31.

21. S.A. Raymond and G.F. Meyer, "Interpretation of Regulatory Requirements by Technology Providers: The Case for Electronic Source Data," Applied Clinical Trials, June 2002, 50-58.

22. J.M. Rabin, J. McNett, and G.H. Badlani, "A Computerized Voiding Diary," Journal of Reproductive Medicine, 41: 801-806 (1996).

23. H.E. Drummond, S. Ghosh, A. Ferguson, D. Brackenridge, and B. Tiplady, "Electronic Quality of Life Questionnaires: A Comparison of Pen-Based Electronic Questionnaires with Convention Paper in a Gastrointestinal Study," Quality of Life Research, 4: 21-26 (1995).

24. Adam M. Kohn, "HIPAA Privacy Rule: Effect on Medical Research," Applied Clinical Trials, June 2002, 60-62.