Electronic Diaries, Part 1: What Is a Subject Diary, and How Do Regulations Apply?

Subject diaries are used in approximately 25% of all clinical trials.¹ They have been developed using one of three types of diary technologies. The traditional approach has been to use paper cards or booklets configured to help the subject follow directions from the clinical protocol. More recently, electronic means have been used, such as dial-in phone numbers with computer-driven questions to answer (interactive voice response systems) and handheld devices with alarms and menu-driven prompts to guide the subject through the protocol requirements.

Subject diaries are used to directly capture subjects experiences surrounding a medical momentthe subject experience prescribed for capture by the clinical protocol whether it is physical or time/logic-based experience. Medical moments may include symptom counts (for example, involuntary micturitions, migraine episodes), subjective symptoms (for example, discomfort surrounding an involuntary micturition, pain severity of a migraine), and quality of life (for example, activity restrictions due to involuntary micturitions or migraines).² Subjects record this information as they go through day-to-day life. The goal is that when the appropriate medical moment occurs, the subject makes an appropriate recording into the diary. Because the diary is the subjects first record of an experience, it is source data. This differentiates diary data from case report form (CRF) data, which are copied from many other sources in the subjects medical file.

Given the importance of subject diaries in clinical research, this article, the first in a series, discusses the relevant regulations and guidance that must be followed to obtain the high standards of data quality set by regulatory bodies. This series is intended to extend previous articles published by Applied Clinical Trials that compare and contrast diary methods³ and survey the diary literature.⁴ The two authors will approach the subject from different backgrounds. Teri Stokes, a global validation consultant, will take a technology/regulatory perspective. Jean Paty, a clinical researcher, will take a scientific/regulatory perspective. Both authors prefer electronic solutions; however, both traditional paper and electronic approaches for subject diaries will be discussed.

The basic concern of regulators is data integrity and data quality regardless of the method used to acquire it. This first article discusses regulators focus on ensuring that they can have confidence in the data. Future articles in this series will explore designing clinical protocols for diary data quality, system and clinical challenges for validation of electronic diaries, and standard practices and issues arising from practical experience with electronic diaries. Each article will pose four relevant questions, with the two authors responses presented for comparison.

In this article, the authors address the following questions about regulatory issues in subject diary research:

• Why are subject diaries important to clinical research?
• How does ICH GCP apply to subject diarieselectronic and paper?
• How do FDA guidance and 21 CFR 11 apply to subject diarieselectronic and paper?
• What are the audit and inspection criteria for subject diary data?

Stokes, as the validation expert, will address those parts of the regulations that place direct demands on the design and development of a diary system. Patys background in clinical research will focus his discussion on those parts of the regulations that most directly affect study design and methodological rigor for using diaries.

Why are subject diaries important to clinical research?Technical systems/regulatory perspective, Teri Stokes: The investigator has limited time with a subject to make observations and assessments of how well a study treatment is working for a particular subject. Capturing the subjects personal experience in a diary at set times between office visits can add much important information about the safety and efficacy of a treatment. Whatever the diary technology (paper, handheld device, or IVRS), the subject is the one person best able to record self-reports at prescribed time intervals.

Clinical science/regulatory perspective, Jean Paty: Understanding the subjects perspective, in addition to physiological effects, is critical in evaluating the safety and efficacy of a drug. Indeed, in a number of therapeutic categories (for example, incontinence and migraines), the subjects self-report is the primary endpoint for evaluating the efficacy of the drug. When the subjects self-report, via a diary, is not the primary endpoint, it is often a key secondary endpoint in determining drug efficacy; for example, subjects reports of day- and nighttime asthmatic symptoms are key secondary parameters in evaluating respiratory drugs. Finally, quality of life data will play a central role in drug development, because subjects want not only to be labeled better by some objective medical standard, but to feel better as well.

TS: Accurate and timely subject diaries are important because they provide a unique inside-the-subject view of what is happening in a clinical trial. While all diaries provide self-assessment data, electronic diary methods also provide audit trail records that show how well subjects comply with the protocol for recording self-assessments. Electronic systems can remind subjects about important events in a complex treatment regimen, and they can prevent out-of-sequence assessments. Between-visit uploads from electronic diaries can provide the investigator with quicker access to diary data so that medical care can be more responsive to subject needs.

JP: In practice, it is not simple to get subjects to reliably complete their diaries in the context of daily life. Unlike research staff, subjects are not accustomed to completing research questions. Moreover, diaries require subjects to take time out of their day-to-day activities to make entries about disease events that might be quite unpleasant for them (for example, migraines). Finally, the information captured from the subject must meet scientific and regulatory standards for rigor, reliability, and accuracy. Thus, there are a number of factors to balance when developing and implementing a subject diary. These issues have led to the development of electronic diaries, which can yield very high rates of compliance from subjects and meet scientific and regulatory requirements.

How does ICH GCP apply to subject diarieselectronic and paper?TS: The International Conference on Harmonization (ICH) developed a Guideline on Good Clinical Practice (GCP) to provide a unified standard for the European Union (EU), Japan, and the United States to facilitate the mutual acceptance of clinical data by the regulatory authorities in these jurisdictions. In section 1.52 of the ICH GCP, the definition of Source Documents gives as a specific example subjects diaries or evaluation checklists.⁵ This classifies both paper and electronic diaries as source documents, and as per section 8.3.13, subject diaries are to be protected and archived with the trial file at the investigator site. For a more detailed treatment, the reader is referred to an ongoing discussion in Applied Clinical Trials regarding reg-ulatory issues surrounding electronic source documents. The most recent installment is from Paul Bleicher in August 2002.⁶

JP: The ICH GCP Guideline (section 2.10) makes the following statement regarding data: All clinical information should be recorded, handled, and stored in a way that allows its accurate reporting, interpretation, and verification. The ICH GCP also requires that the investigator should ensure the accuracy, completeness, legibility, and timeliness of the data reported to the sponsor in the CRFs and all required reports. (Section 4.9.1).

TS: Being a source document differentiates the subjects diary from the subjects case report form (CRF), which is a collection of data derived from many source documents in a subjects medical file. The investigator is responsible for retaining all original source documents whereas the sponsor is responsible for keeping the original of the CRFs.

JP: Historically, diaries have taken the form of paper booklets or cards given to subjects at study visits, and completed by the subject each day, during the course of everyday life. A number of studies, including a recent one published in the British Medical Journal,⁷ have shown that subjects do not complete these paper diaries in a timely manner. In that study subjects entered only 11% of their diary entries in compliance with study procedures. This raises the question of the accuracy of subject reports with traditional paper diaries. Electronic diaries have been shown to yield high rates of compliance with timely completion.

TS: Paper diaries always go to the investigator first. When electronic diaries are used, they are often developed and deployed by a third-party service provider on behalf of the sponsor. It is important for sponsor and suppliers to remember that the diary data is source data and belongs with the investigator under ICH GCP. During the trial, the investigator must have immediate online access to diary data so that the medical moment(s) described therein can be incorporated into the subjects medical care. At the end of the trial, a certified copy of all the subjects diary data must be sent to the investigator for retention with the investigators trial source records to support future audits and inspections.

JP: Published research using electronic diaries provides evidence that this approach can yield data of high quality and integrity (see the recent review in the April 2002 issue of Applied Clinical Trials).⁴ One such study was conducted using electronic diaries with smokers trying to quit.⁸ The study required that subjects report their craving to smoke throughout the day. Traditionally, this would have been captured by asking the subjects to make a single entry at the end of the day about their smoking. However, by giving each of them a handheld computer, it was possible to ask subjects about their craving 8 to 10 times each day. Subjects recorded the relevant medical moment, in the moment, rather than later on. The results showed that subjects readily complied with these procedures, responding to over 90% of the handhelds prompts to answer questions about craving.

TS: Processing and access to electronic CRF data can and usually does occur long after the investigator has processed the medical moment (visit, assessment, test, episode) in a trial. Electronic diary data, however, is part of the medical moment itself and must reach the investigator for diagnostic processing in the shortest possible time, even though it may be first received on the server of a sponsor, service provider, or CRO.

JP: Thus, electronic methods can yield high rates of subject compliance with protocol procedures and scientific rigor. This higher compliance will in turn yield more reliable and potentially more accurate data, which makes electronic diary methods compliant with ICH data quality guidelines.

TS: As per ICH GCP section 5.5.3 on electronic data handling, an electronic diary system has to be validated, have an SOP for its use, include an audit trail, and have security that prevents unauthorized use. Only authorized individuals should be allowed to make data changes on the system, the data must have adequate backup, and any blinding must be safeguarded. Finally, section 5.1.3 states, Quality control should be applied to each stage of data handling to ensure that all data are reliable and have been processed correctly.⁹

How do FDA guidance and 21 CFR 11 apply to subject diarieselectronic and paper?TS: The Food and Drug Administration passed a two-part law in 1997 in parallel to implementing the ICH GCP. The Final Rule on Electronic Records and Electronic Signatures (21 CFR 11) applies to electronic subject diaries as electronic records, and when electronic signatures are used, it applies to the signatures as well. This law does not apply to paper diaries unless they are scanned into electronic form and archived as electronic records.

JP: Whether a paper or electronic diary is used, it must yield high data quality and integrity. The FDA makes the following statements in the April 1999 Guidance for Industry: Computerized Systems Used in Clinical Trials:
FDAs acceptance of data from clinical trials for decision-making purposes is dependent upon its ability to verify the quality and integrity of such data during onsite inspections and audits. To be acceptable the data should meet certain fundamental elements of quality whether collected or recorded electronically or on paper. Data should be attributable, original, accurate, contemporaneous, legible.¹⁰ These five components of data quality and integrity have been conveniently labeled ALCOA.

TS: 21 CFR 11 echoes all of the ICH GCP concerns mentioned above. It adds time-stamping to the audit trail so that local time can be established for any edits or changes and the use of operational system checks to enforce permitted sequencing of steps and events, as appropriate.¹¹ These two new Part 11 components are very important for subject diaries and describe the key ways that an electronic diary performs better than a paper diary. A combination of alarms, prompts, and lockouts in the electronic diary can enforce protocol requirements for subject responses to be made in a specified sequence and at prescribed times.

JP: Timely or contemporaneous recording of subjects experiences, or medical moments, is one of the key components of the ALCOA standard of data quality and integrity. If subjects wait to record the relevant information from a medical moment, be it a brief episode of urinary incontinence or an extended migraine headache, the impact of other life experiences will affect the accuracy of their recording. Again, the subject diary method employed must be shown to meet the ALCOA standards, whether it is paper or electronic.

TS: Paper diaries can be filled in at any time, and this fact leads to the questionable accuracy of their data. Part 11compliant electronic diaries structure response times to the protocol design by allowing data entry only at specific times, time-stamping subject entries, and preventing subjects from entering data out of sequence.

Other Part 11 concepts include the use of device checks to determine the validity of source data input and the use of electronic signatures. Part 11compliant electronic diaries use handheld devices with unique identifiers and have software that can check the device ID communicated during a transmission. Such a check ensures that the device has been assigned to the subject, trial, and site identified in the diary report during each upload to the database server. With paper diaries, the subjects handwriting is unique, but if the subject forgets to sign the diary card and the report is mostly check boxes and analog graph lines, this could present an authentication problem.

The use of electronic signatures brings with it a technical burden described in sections 11.50 and 11.70 and an administrative burden as per Subpart C of Part 11. From a practical perspective, it is best to limit the use of electronic signatures in clinical trials to only those places where a handwritten signature is mandated. There is no legal mandate for a handwritten signature on subject diaries. One option to ensure authentication of the source for electronic diary data is the use of one password by the subject and the device checking performed by the server software.

What are the audit and inspection criteria for subject diary data? TS: As an introduction to his paper on FDAs good laboratory practices (GLP) and computerized data acquisition systems in 1992, Paul D. Lepore of the FDA published a definition of what the FDA means by the terms data quality and data integrity. Lepore stated that the important characteristics of data quality are that the data are accurate, immediate, legible, durable, and attributable.¹² He then stated that the important characteristics of data integrity are consistency, fidelity, and honesty.⁹ These criteria for data quality and integrity can be applied to diary data irrespective of the collection methodelectronic or paper. The Lepore paper on GLP data quality provides historical context for the ALCOA standard for GCP data on clinical trial systems in FDAs 1999 Guidance.

JP: According to the Computerized Systems Used in Clinical Trials Guidance, inspectors will look to verify the quality and integrity of the data.¹⁰ From a practical perspective, this will mean evaluating the diary using the ALCOA standard. The challenges will be different for paper versus electronic diary methods, as shown in the table Technical/Regulatory Perspective on ALCOA.

TS: Auditors and inspectors will seek to examine documented evidence of these characteristics when diary data are used to support endpoints in pivotal studies. Specific application of these characteristics to electronic diaries is given in the table.

JP: From a clinical perspective, it is essential that the data reported on diaries be clear and usable for the investigator as a start, and then later for the CRA and sponsor. With paper diaries, legibility and completeness are consistent problems. If the diary is not legible, it is functionally incomplete. This missing information creates gaps that the investigator cannot fill at the site weeks, or even months, after the fact. This is, of course, problematic if the investigator needs to use this information during the course of the trial to make a medical decision.

TS: Every audit and inspection has training records on its checklist, and Part 11 section 11.10 (i) requires that persons who develop, maintain, or use electronic record/electronic signature systems have the education, training, and experience to perform their assigned tasks.⁸ It is important to have training materials and training records for both investigator staff and subjects using electronic diaries. If there are no instructions and/or users have not been trained in how to complete a diary record in either paper or electronic form, then the quality and validity of the data may be compromised.

JP: Problems with legibility and completeness directly affect data quality and the ability to use the data to evaluate the primary and secondary aims of the study. If the CRA and data management group at the sponsor cannot read the data, then it will be very difficult to capture it in a database. In turn, analysis of the data becomes difficult. The scientific importance of high data qualitythat diary data be clear, legible, and completecannot be overemphasized.

TS: In addition, it is important that the diary provider(s) have training records for their software developers and the personnel who configure and manage the trial diary process for a specific study. For paper diaries, only instruction materials and training for investigator staff and subjects are needed, but training records should still be kept.

Common message
In this discussion, two perspectivestechnical systems and clinical scienceconverge on a common message: Subject diaries are important to trials where the clinical protocol uses self-reports as primary or secondary endpoints, and the quality and integrity of subject diary data is essential to the credibility of such trials. Regulations, guidance, and published papers state requirements for data quality and data integrity that apply to both paper and electronic diary systems. This discussion suggests that electronic diaries can potentially comply with ALCOA standards of quality and integrity more readily than paper diaries. FDA Guidance and other regulations (21 CFR 11 and ICH GCP) do provide useful guidance to support development of compliant electronic diary systems. The next article in this series (Part 2) will discuss how clinical protocol design and electronic diary design must work together to achieve data quality and regulatory compliance.

References

1. DataEdge, 1999. Unpublished data regarding frequency of use of diaries in clinical trials.
2. Saul Shiffman, Michael Hufford, and Jean Paty, Subject Experience Diaries in Clinical Research, Part 1: The Subject Experience Movement. Applied Clinical Trials, February 2001, 4656.
3. Stephen Raymond and Robert Ross, Electronic Subject Diaries in Clinical Trials, Applied Clinical Trials, March 2000, 4858.
4. Michael Hufford and Alan Shields, Electronic Diaries: Applications and What Works in the Field, Applied Clinical Trials, April 2002, 4656.
5. Guideline for Good Clinical Practices, International Conference on Harmonization, Federal Register, Section 1. Glossary (www.emea.eu.int/pdfs/human/ich/013595en.pdf).
6. Paul Bleicher, eSource Redux, Applied Clinical Trials, August 2002, 3031.
7. Arthur Stone, Saul Shiffman, Joseph Schwartz, Joan Broderick, and Michael Hufford, Subject Non-compliance With Paper Diaries, British Medical Journal, 324 (2002), 1193-1194.
8. Saul Shiffman, Celeste Elash, Stephanie Paton, Chad Gwaltney, Jean Paty, Deborah Clark, Ken Liu, and Michael DiMarino, Comparative Efficacy of 24-Hour and 16-Hour Transdermal Nicotine Patches for Relief of Morning Craving, Addiction, 95: 11851195 (2000).
9. International Conference on Harmonization, ICH Guidelines, Good Clinical Practices, Section 5. Sponsor (www.emea.eu.int/pdfs/human/ich/013595en.pdf).
10. Food and Drug Administration, Guidance for Industry: Computerized Systems Used in Clinical Trials (FDA, Rockville, MD, 1999).
11. Code of Federal Regulations, Title 21 CFR, Part 11: Electronic Records; Electronic Signatures; Final Rule, Federal Register, March 20, 1997.
12. Paul D. Lepore, FDAs Good Laboratory Practice Regulations and Computerized Data Acquisition Systems, Laboratory Information Management 17: 283287 (1992).