Collaborating for CT Systems Validation

As the pharmaceutical industry faces the continuing challenge of validating the software used to collect and store data during clinical trials, its best source of help should be the vendors who create the applications.

This article reviews the need for software validation from both a pragmatic and regulatory standpoint, describes the crucial role vendors must play in the software validation process, and provides guidelines to use when working with vendors. The goal: Help make certain that an application meets FDA requirements to ensure the accuracy, reliability, integrity, availability, and authenticity of data.

From a pragmatic standpoint, sponsors and contract research organizations (CROs) have always needed software validation. Software validation encompasses the absolutely critical process of defining what functions an electronic data capture (EDC) or other application needs to provide in order to support the needs of the sponsor, CRO, contract research associates, and other stakeholders. Additionally, software validation requires verifying that the stated needs have been met by the application; that the application has been thoroughly tested, properly deployed to users who have been provided training; and that there is a well-documented plan for deploying updates and otherwise managing the complete lifecycle of the application.

Without software validation, sponsors, CROs, and other stakeholders face the bleak prospect of having bad data creep into a study, good data corrupted, digital signatures lost, security breached, and a nightmare of other possibilities—any of which could jeopardize a clinical trial. So, from a pragmatic standpoint, software validation is needed because it would be unconscionable to do otherwise.

From a regulatory standpoint, sponsors and CROs need to validate software and document their validation process to meet guidelines set forth by the FDA's 21 CFR Part 1,^1,2 and by the FDA's General Principles of Software Validation.³

Breaking through the confusion

Nearly 10 years after its introduction, 21 CFR 11 still causes confusion within the industry. The FDA published 21 CFR 11 to facilitate the use of electronic records and electronic signatures in clinical trials as well as to establish requirements for validation. It describes the technical and procedural requirements that must be met when electronic records and/or signatures are used in lieu of paper records and signatures. The FDA resisted industry calls for a more prescriptive set of requirements for 21 CFR 11, which was general on content and tone. This left the door open for rapid innovation in development of software applications for clinical trials, but also engendered a degree of angst and confusion for sponsors and CROs faced with the need to validate EDC and other eSolutions.

Confusion erupted again in February 2003 when the FDA announced that it was embarking on a re-examination of Part 11 and issued "Guidance for Industry, Part 11 Electronic Records; Electronic Signatures—Scope and Application,"² which provides the agency's current thinking and narrows the interpretation and applicability of some areas of the regulation. The clarification also makes clear that Part 11 is still in effect for predicate records.

However, the same document, in the Discussion section, makes the case for adopting Part 11 requirements, with passages such as:

"FDA will enforce predicate rule requirements for records that are subject to Part 11...including, but not limited to, certain controls for closed systems...use of operational system checks; use of authority checks; use of device checks; determination that persons who develop, maintain, or use electronic systems have the education, training, and experience to perform their assigned tasks; establishment of and adherence to written policies that hold individuals accountable for actions initiated under their electronic signatures; and appropriate controls over systems documentation."

This section of the discussion concludes with:

"Furthermore, persons must comply with applicable predicate rules, and records that are required to be maintained or submitted must remain secure and reliable in accordance with the predicate rules."

The above passages are quoted simply to show that despite the announced narrowing of interpretation, 21 CFR 11 remains relevant to clinical trials, for both pragmatic and regulatory reasons.

The vendor's role

The vendor has a crucial role in helping sponsors and CROs validate that software is compliant with 21 CFR 11. This role will become only more important as the industry increasingly moves to EDC solutions. The benefits of electronic data capture in clinical trials have been well documented, including in "The Virtual Revolution (The Time is Now)," a July 2005 article in Applied Clinical Trials that notes, "A well-developed EDC process will allow real-time access to data, online trial monitoring that saves travel time and increases trial efficiency, better trial management, and overall improvement in data quality."⁴

Vendors creating innovative EDC solutions for the industry need to play a leading role in helping sponsors and CROs to validate that their software is 21 CFR 11 compliant. The vendor's role is critical for two basic reasons: First, 21 CFR 11 compliance cannot simply be added into an EDC solution after the fact—it must be designed and built into the product from the very beginning—and second, the vendor, as creator of the code, is in the best position to help sponsors and CROs validate the compliance of the code.

This is not to say that the organizations purchasing and deploying EDC and related solutions should abdicate responsibility for software validation to the vendor, but rather that the vendor should work closely with its customers to document how 21 CFR 11 compliance is built into the full product lifecycle—from design to coding, testing, deployment, release of product upgrades, and ongoing maintenance. This cooperation should include the vendor providing sponsor and CRO validation teams with guidance on running their own test cases against the software to ensure compliance.

Guidelines for validating software

1. Define business and functional requirements. The sponsor or CRO should carefully define the functionality required of a software product, so that it can confirm how a product will meet the needs of its clinical trial process. This is an essential step in the validation process to prove the system functions as expected once delivered by the vendor.

2. Determine that a system meets the established requirements. With product needs specified, the sponsor or CRO can work with the vendor to map out the product function needs and ensure applicability of the system.

3. Validate the software development lifecycle methodology. Once a product is identified that meets the specified requirements, the sponsor or CRO needs to validate the vendor's software development lifecycle. This means developing an understanding of how the vendor designs, tests, supports, and controls change management of its product. This also includes learning how the vendor builds the relevant 21 CFR 11 requirements such as support for electronic signatures, security, and audit trails. Exploration of the software development lifecycle can be a key element in gaining confidence in the vendor—which is essential to successful deployments.

4. Validate vendor testing. The sponsor or CRO should work closely with the vendor test team to understand the testing process, including test cases used to demonstrate key functions such as record auditability. Defined and expected by the FDA, the validation of the system and/or processes must meet predetermined requirements before the trial can begin. It is imperative that the data not be jeopardized and remain safe and unchanged to ensure the successful outcome of the clinical trial. The need for software testing is addressed in the FDA's "General Principles of Software Validation," which was written for medical devices but is relevant to clinical trials solutions.

The requirement for validation of vendor testing for clinical systems is identified in "Guidance for Industry Computerized Systems Used in Clinical Trials," where FDA clearly states: "For software purchased off-the-shelf, most of the validation should have been done by the company that wrote the software. The sponsor or contract research organization should have documentation (either original validation documents or on-site vendor audit documents) of this design level validation by the vendor." Software validation has also been addressed by the International Conference on Harmonization (ICH).^5,6

The FDA acknowledges that software testing cannot be absolute. "Software verification and validation are difficult because a developer cannot test forever, and it is hard to know how much evidence is enough. In large measure, software validation is a matter of developing a 'level of confidence' that the [product] meets all requirements and user expectations."³

5. Verify auditability. You must be able to validate that an EDC or related solution has efficient audit trails that can trace data from its entry into the system (including who entered it) through all transformations and storage. At every stage the audit trail should include computer-generated time stamps. Your best guide to what the FDA wants is 21 CFR 11, which speaks to the need for being able to generate accurate and complete copies in human-readable and electronic form, suitable for review by the agency, and being able to ensure data is protected and retrievable throughout the record retention period.

6. Verify security. The FDA, under 21 CFR 11, is keenly interested in data security. FDA concerns include: limiting access to authorized individuals trained in the use of the system; providing written policies to ensure user accountability for actions initiated under the user's electronic signature; controlling access to system operation and maintenance documentation; encrypting records, or otherwise protecting record authenticity, integrity, and confidentiality; and meeting specific requirements for electronic signature components and controls.

7. Perform your own testing. Sponsors and CROs should perform their own testing on the product to verify that the software meets established requirements and is compliant with 21 CFR 11. High-value areas for testing include demonstrating audit trail functionality and system security. Testing should also verify the established requirements for the system and study, with a carefully chosen set of test cases. The results of this testing need to be documented in a report that describes what functionality was tested, how the testing was performed, and what the results were.

8. Avoid overkill in reproducing vendor testing. Validating the software development lifecycle and vendor testing should help develop confidence that a vendor can save sponsors and CROs from the burdensome overkill of recreating all of the vendor's testing. It is a needless expense for the customer, who is drawn away from core competencies to set up their own software testing shop. Rather than recreating all product testing, customers should focus their testing on verifying that the requirements of the study have been addressed, including 21 CFR 11 compliance.

9. Validate change management. The FDA has noted the need for change management strategies to ensure that software updates do not create unintended problems. Sponsors and CROs should ask for a detailed description of the vendor's change management procedures and include these in the audit report.

10. Qualify the computing environment and product installation. The sponsor or CRO needs to validate that the software and hardware environment, including servers, desktop computers, operating systems, and other solution elements meet vendor specifications. Correct installation of the software per the vendor's specifications must also be verified.

11. Test against initial requirements. Once the software has been deployed, the sponsor or CRO should do a final round of testing to confirm that the solution supports the day-to-day needs of the clinical trial process, as originally set forth in the definition of requirements.

12. Verify documentation and training. Finally, the sponsor or CRO needs to validate that documentation is readily available and that training is implemented to ensure proper use of the solution for clinical trials.

In conclusion, the pharmaceutical industry should benefit from deployment of EDC and related solutions that streamline the clinical trial process. Software validation is an essential first step on the path toward enjoying the benefits of EDC. Because of their deep knowledge of the underlying code and architecture, vendors are uniquely positioned to lead the joint effort of software validation.

References

1. U.S. Food and Drug Administration, "21 CFR Part 11 Electronic Records; Electronic Signatures," Final Rule, Federal Register, March 20, 1997.

2. U.S. Food and Drug Administration, "Guidance for Industry, Part 11, Electronic Records; Electronic Signatures—Scope and Application," Federal Register, February 4, 2003.

3. U.S. Food and Drug Administration, "General Principles of Software Validation; Final Guidance for Industry and FDA Staff" (FDA, Rockville, MD, January 2002).

4. L. Scheible and M. Pozsgai, "The Virtual Revolution (The Time is Now)," Applied Clinical Trials, July 2005.

5. International Conference on Harmonization (ICH), "Good Clinical Practices E6," May 1996.

6. ICH, "Statistical Principles for Clinical Trials E9," February 1998.

Gaby Trespalacios is vice president, quality systems & compliance, with DataLabs Inc., 101 Academy Drive, Suite 250, Irvine, CA 92617, (949) 851-2030, email: gtrespalacios@datalabs.com