When Should the Audit Trail Begin?

When audit trails are used in any business process, whether it be in pharmaceutical clinical research, stock market trading or banking, it creates a step-by-step record by which data can be traced to its source. An audit trail is most often utilized when the accuracy of a data element needs to be verified or when a record of changes made to original, or source data is required. Not all data have the same level of criticality. For example, in a clinical trial, the significance of tracking data changes may depend on the nature of critical variables such as those affecting randomization, as well as compliance with inclusion and exclusion criteria. For example, it would be unacceptable for a data field to be changed in order to make an ineligible subject eligible without full visibility and documentation to allow assessment by those monitoring the study.

This article addresses and discusses the regulatory history behind audit trails and its interpretation as it relates to clinical research data originating or residing in EDC or eCOA systems.

Regulatory History:

FDA: In 1997¹, the FDA published the Final Rule on Electronic Records; Electronic Signatures as related to 21 CFR Part 11. As part of the discussion, it was stated in comment 72 on page 13446 that: “§11.10(e) would require the use of time-stamped audit trails to document record changes, all write-to file operations, and to independently record the date and time of operator entries and actions.”

It was also stated that: “…record changes must not obscure previously recorded information and such audit trail documentation must be retained for a period at least as long as required for the subject electronic documents and must be available for agency review and copying.” “As many comments objected to the proposed requirement that all write-to file operations be documented in the audit trail…”, FDA noted that it is the “…agency’s intent that the audit trail provide a record of essentially who did what, wrote what, and when.”

Although FDA acknowledged that “…not every operator ‘action’, such as switching among screen displays, need be covered by audit trails, the agency was concerned that revising the rule to cover only ‘critical’ operations would result in excluding much information and actions that are necessary to document events thoroughly.” “The agency intended that the audit trail capture operator actions and operator information at the time the information is saved to the recording media (such as disk or tape), in much the same manner as such actions and information are memorialized on paper.” FDA noted that the “…audit trail need not capture every keystroke and mistake that is held in a temporary buffer before those commitments. For example, where an operator records the lot number of an ingredient by typing the lot number, followed by the ‘return key’ (where pressing the return key would cause the information to be saved to a disk file), the audit trail need not record every ‘backspace delete’ key the operator may have previously pressed to correct a typing error (our bold).” FDA added, that “…subsequent ‘saved’ corrections made after such a commitment, must be part of the audit trail.” Accordingly, the FDA revised proposed § 11.10(e) “…by removing reference to all write-to-file operations and clarifying that the audit trail is to cover operator entries and actions that create, modify, or delete electronic records.”

EMA: In June 2010², the EMA published a Reflection paper on expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials and stated that: “The audit trail starts with the initial entry of the data.”

FDA: In 2013³, the FDA Guidance for Industry, entitled: “Electronic Source Data in Clinical Investigations” defined an audit trail as: “A process that captures details such as additions, deletions, or alterations of information in an electronic record without obscuring the original record. An audit trail facilitates the reconstruction of the course of such details relating to the electronic record.” "§III.A.2.e. Transmission of Data From Patient-Reported Outcome (PRO) Instruments to the eCRF. When a PRO instrument is used by a subject to transmit data elements directly to the eCRF, the subject is the data originator and the eCRF is the source. If a process is used by which the subject uses the instrument to transmit data to a technology service provider database, the service provider database is the source.”

MHRA: In 2018⁴, the Medicines & Healthcare Products Regulatory Agency (MHRA) ‘GXP’ Data Integrity Guidance and Definitions stated that: “The audit trail is a form of metadata containing information associated with actions that relate to the creation, modification or deletion of GXP records. An audit trail provides for secure recording of life-cycle details such as creation, additions, deletions, or alterations of information in a record, either paper or electronic, without obscuring or overwriting the original record. An audit trail facilitates the reconstruction of the history of such events relating to the record regardless of its medium, including the ‘who, what, when and why’ of the action. Where computerized systems are used to capture, process, report, store or archive raw data electronically, system design should always provide for the retention of audit trails to show all changes to, or deletion of data while retaining previous and original data. It should be possible to associate all data and changes to data with the persons making those changes, and changes should be dated, and time stamped (time and time zone where applicable). The reason for any change, should also be recorded. The items included in the audit trail should be those of relevance to permit reconstruction of the process or activity. The relevance of data retained in audit trails should be considered by the organization to permit robust data review/verification.” “It is not necessary for audit trail review to include every system activity (e.g., user log on/off, keystrokes etc.).” (our bold)

Discussion:

We believe, that unless there is a clear need to collect keystroke data, any data collected prior to clicking on the SUBMIT/SAVE keys serves no purpose. Analysis of these data also would be complex and adds no value to the study, the study record, or the audit trail. 

While the definition of an audit trail is very clear, there appears to be some confusion about the definition of an original or source record in the digital world. We believe the data entered after clicking on SUBMIT or SAVE are the source, except for a very few cases: 1) where there are other original records such as data originally collected as a paper record, or 2) where the ability to type accurately (or speak clearly) is a variable to be measured according to the study protocol, or the natural characteristics of end point data. Conversely, we do not see value for clinical research, in assessing data quality in data that includes erroneous keystrokes as source, such as data that would be collected before the user has corrected unintentional errors and before the user clicks SUBMIT or SAVE. 

Edit checks can also affect audit trail retention. Edit checks on end user devices should only catch obvious user errors. These are flagged and the user has the opportunity to correct “immediately” before going forward, and no audit trail is necessary as this is simply automated mistake correction. 

However, there are situations where an immediate edit check should not be used, and instead edit checks that fire after the data have been submitted can be used to alert site staff of anomalies that should be investigated. All changes made after the data have been submitted must be captured in the audit trail.

While we do not recommend saving pre-submit keystroke data, we do recommend that the source of the data that is transmitted to the clinical research database is properly attributed. The audit trail functionality of the eCOA devices (mobile phones, tablets, Internet of Medical Things (IoMT), and web browsers) used in clinical trials should be aligned with the regulatory agency’s descriptions noted above, as well as industry standard practices. 

We recommend that at the time of data entry into an electronic system that supports a clinical trial such as EDC and/or eCOA, two additional metadata should be collected: the data source and the data originator. Examples of a data source are direct data capture, paper record, EHR etc. Examples of data originators are the clinician who did the physical exam or the study coordinator who did the interview. A data originator for an eCOA questionnaire, whether using a dedicated device or a website, is the person logging in with his/her username and password, as it is assumed that data entry will be performed by that person (regardless of if this is the study subject or authorized designee/caregiver, such as a parent in a study of pediatric patients).

For eCOA users, the data source or data originator is usually very clear in the system and in the audit trail. There is also no need for source data verification (SDV) against the eCOA device, as there is only one copy. The devices essentially acquire data from study subjects in real time, temporarily store the data within the application, and then eventually pass the data to the centralized database as a completed task. Changes made to entries prior to submitting the record being created by the data originator, can be considered draft data, held in temporary storage until that point in time when the user acknowledges their entry is complete, and then consciously saves the record and submits it for transmission. This is analogous to completing a draft paper form prior to signing the record. Thus, the audit trail for eCOA devices used in clinical trials begins at the time data is committed to permanent storage in the main eCOA database, similar to authenticating a paper form at the time of signing. 

It is important that each company is clear on when the audit trail begins for their EDC and eCOA systems. We recommend that they perform a risk assessment regarding the value of saving keystroke data prior to a save/transmit transaction. The protocol or other document (such as the Data Management Plan) should include this information.

Conclusion

All of us can hit wrong keys when entering data using a keyboard/touchscreen or when we observe voice transcription errors. Unless there is a clear need to collect keystroke data, any data collected prior to clicking on the SUBMIT/SAVE keys serves no purpose. Analysis of these data also would be complex and adds no value to the study, the study record, or the audit trail. 

To properly attribute source data, two additional metadata need to be collected at the time of data entry: the data source and the data originator.

Finally, to be clear as to when the audit trail begins, it is recommended that each company perform a risk assessment regarding the audit trail value of keystroke data and data entered prior to a save/transmit transaction. It is also recommended to put this information into the protocol or other document such as the Data Management Plan (DMP). And finally, when in doubt, share the plan with regulators and obtain their agreement.

Members of the eClinical Forum Regulatory Advisory Group:

Jules Mitchel, PhD, is the Co-Founder of Target Health, US. Yumi Sugiura is the Associate Director of Global Data Management & Centralized Monitoring for Bristol Myers Squibb, Japan. Valdo Arnera, MD, is a Scientific Advisor for ERT, Switzerland. Greg Gogates is the Vice President of Quality/Regulatory for Fasor, US. Alan Yeomans is a Quality Manager for Viedoc Technologies, Sweden. Devry Spreitzer is a Senior Director, Data Integrity Quality Assurance for Astellas Pharma Global Development, US. Suzanne Bishop is the Americas Administrator for eClinical Forum.

References

1. FDA: 21 CFR Electronic Records; Electronic Signatures – Final Rule (1997)
2. EMA: Reflection Paper On Expectations For Electronic Source Data and Data Transcribed to Electronic Data Collection Tools in Clinical Trials (2010)
3. FDA: Electronic Source Data in Clinical Investigations (2013)
4. MHRA: ‘GXP’ Data Integrity Guidance and Definitions (2018)