Meeting the Challenges of Internet-based Clinical Trials

In the mid to late '80s, a tool called remote data entry was available which replaced double key data entry and paper case report forms (CRFs) at the clinical trial study site. When remote data entry (RDE) was used, the drug sponsor would provide a portable computer to the investigational site. The coordinator would collect study-related patient data, and then enter the data directly into the computer via the specially designed user interface of data entry screens. The electronic data would then be monitored. After data cleanup, a floppy disk with the site's data would be sent to the sponsor via an overnight courier service. These tasks would occur periodically during the course of the clinical trial.

More recently, Internet-based clinical trials (a prime aspect of electronic data capture called IBCTs for simplicity) have been introduced. However, there are major challenges for companies who choose to implement IBCTs. Some clinical research professionals question the advantages of moving away from "tried-and-true" tools such as paper CRFs. These individuals point out that such tools have worked well over time, and "if it ain't broke, don't fix it." In addition, clinical research study sites may be hesitant to embrace IBCTs as they may have to shift resources and retrain staff.

The reality is that paper-based systems do not work very well and are not efficient. Paper-based systems, by definition, generate enormous amounts of paper, involve redundant and inefficient processes, and introduce the potential for major delays from the time of data capture to the time the data can be reviewed and analyzed.

Kelly and Oldham¹ wrote one of the first articles regarding the Internet and randomized controlled trials. These authors addressed the factors which could constrain the implementation of IBCTs, as well as the advantages of IBCTs in obtaining large sample sizes with reduced unit costs through global access, fast interaction and automation. Kuchenbecker et al.² described the advantages of using Internet technologies for data acquisition in clinical trials and supported the premise that IBCTs provide a fast and easy avenue for the acquisition of scientific data. The article predicted that this method of data acquisition and data processing would become more commonplace in multicenter clinical trials. Richardson³ provided details on planning and running an IBCT.

Figure 1. Sample data entry screen.

A recently published summary revealed practical considerations and steps in setting up IBCT.⁴ Another paper⁵ demonstrated the areas of cost saving in IBCTs and how the query rate is markedly reduced compared with a paper-based trial. In terms of data quality, it was also shown⁶ that data capture in a IBCT had a very low error rate, and that the errors observed were related to human activities and not related to the underlying technology. Banik⁷ showed that queries in electronic data capture-based studies can be reduced by at least 80%, compared to paper-based queries. Banik also showed that compared to paper-based studies, IBCTs can reduce clinical trial duration by at least 30%, time to database lock by 43%, and queries by 86%.

The changing role of data management

Data entry.

In a paper CRF trial, the study site coordinator completes a paper CRF from data documented in the patient record. When the monitor arrives at the study site, the contents of the CRFs are reviewed for accuracy against the patient record. Once the data transcription is verified, the monitor "pulls" the CRF pages and either overnights them or physically brings them to the office upon return from the monitoring trip. These paper CRFs have been known to get lost and/or damaged every once in a while.

Once in-house, the pages are sent to the data entry group, which logs them in and enters the data into each of two parallel databases. Each "double entry" is then compared, usually using a SAS-generated data-compare program. If the two entries correspond to each other, the data are accepted. However, if the two entries do not correspond, the correct entry is determined by referring to the original CRF and the incorrect entry is changed. If the data entry clerk cannot read a field, or the data seem illogical or obviously incorrect, a notification is sent to the clinical research associate (CRA) and a query is sent to the site to clarify or correct the data entry field in question. Finally, data listings are generated in SAS, and quality control (QC) procedures are put in place to verify that the data listings are consistent with the original paper CRF. At the time of the final QC, all paper queries must be filed with each CRF to ensure that the final database reflects both the data captured on the CRFs and the resolved queries.

In an IBCT, there is no double-key data entry, and all tasks, from "pulling" CRF pages to data listing review against paper CRFs, are completely eliminated. In an IBCT, the study site coordinator now completes the CRF via an electronic user interface from data collected in the patient record. A sample data entry form is illustrated in Figure 1. It should be noted how similar the user-interface can be to a traditional paper CRF.

Figure 2. Sample data entry screen with identification of missing data.

Upon electronic submission of the data to the database, the system automatically checks for inconsistent, illogical or missing data, and when appropriate, sends back an error message notifying the site of the possible error. Figure 2 illustrates a "soft" edit check in which missing data are flagged; in this case "Race" was missing. The system identifies the missing data element and prompts the coordinator to explain why Race is missing.

Data manager. While data management responsibilities vary between companies, in general the data manager acts as a bridge between clinical research and biostatistics. In paper CRF clinical trials, the CRFs are usually prepared by the clinical group. The data manager may or may not see the final CRF. This gap may be obviated, at least in part, if a company has worldwide standards for CRF and database design. However, in our experience, this is not usually the case. As a result, at some point in time the data managers receive a copy of a CRF, for which they may have had no impact on design. The CRF is then annotated to describe the variable names, their definitions, any characteristics of the variable such as numeric versus text, the length of the variable, and any specific decodes such as a yes/no drop down box. After CRF annotation, the data manager may be involved in database design, and also the generation of SAS edit checks in order to look for data entry errors and discrepancies. In some circumstances, the data manager can even generate queries to resolve data inconsistencies in addition to creating the data listings. The data manager states when the database is clean and is directly involved in the tasks surrounding database "lock."

In IBCTs, the annotated CRF, database design, and the vast majority of edit and logic checks (some SAS edit checks are still needed) are finalized prior to enrolling the first subject. In other words, most of data management planning and implementation are completed even before data entry is initiated. Operationally, the data manager now reviews the CRF prior to its finalization and provides input to the clinical group in order to allow for CRF and database optimization. While the data manger still generates the edit checks for implementation by the programming group, this task must be done immediately after CRF design is completed. The reason is that edit check specifications must be given to the programmers for incorporation into the data entry forms prior to initiation of the clinical trial. Each form can now be "initially reviewed" by the CRA at the time of source document verification. The data manager can now look at the data in real-time, lock the form after final review, have rapid access to SAS datasets, and begin the generation of data listings and identification of potential problems early during the course of a study.

Query management. Query management is one of the main tasks in data "cleanup." Queries are usually generated by CRAs and/or Data Management. Medical monitors may also request clarifications from a CRA during medical review of the data. Currently, query management is a paper-intensive and labor-intensive process. Queries are usually generated manually by completing a two-part no-carbon-required paper form. The form is sent to the clinical study site for resolution by the coordinator and approval by the investigator, and finally returned to the sponsor. The query is then filed with the appropriate paper CRF.

This process can be lengthy and may require several iterations. Query management is subject to errors and it is not uncommon for a paper query to be misplaced or misinterpreted. Reconciling "queries sent" versus "queries resolved" can be a very labor-intensive process. Even if a query system is Internet-based, if it is not fully integrated into an electronic data capture system there is still a need to manually transfer information to the main database from the query database.

Figure 3. Query alert message.

In IBCTs, the entire query management process is handled electronically, with no paper. Queries can now be managed via a computer user interface rather than paper forms in ring binders. What used to take hours now takes minutes in a seamless operation. After the monitor logs on to the IBCT, there is a message that indicates whether there are outstanding queries. A sample query alert message is illustrated below. Figure 3 indicates that in this demonstration there are six outstanding queries.

There is no need to check a book, look up a log, or make a phone call. The monitor clicks on the query button and all outstanding queries are at her/his fingertips. Query turnaround time is minimized and forms can be locked effectively, thereby shortening the time to database lock. Figure 4 illustrates how an electronic query management system may work. Basically, there are four states in this particular query process: 1) "New Edit Check"-where there are missing, illogical, and/or out of range data at the time of data entry for which the monitor needs to respond to the site's explanation of the event; 2) "Pending Site Reply"-where the monitor has queried the site and is waiting for a reply; 3) "Pending Approval By Monitor"-where the site has replied to a query and is waiting for a response from the monitor; and 4) "Resolved Data Modified/Query Ap-proved"-where there is full query resolution.

Queries must be managed via a secure communication tool with a permanent record of each generated edit check and response. Query management requires no more effort than the equivalent of an email communication. Now, the efforts of the CRA and study coordinator are more appropriately focused on resolving potentially important clinical issues rather than tracking and accounting for paper forms.

The changing roles of study monitoring

Case report form (CRFs).

In paper-based clinical trials, the CRFs are generated based on the protocol, printed on three-part paper, and mechanically bound or placed in binders. A sufficient number of CRFs are shipped to the sites, and the rest are stored for future shipments. When the sites receive the CRFs, they are placed on shelves until used. If there are amendments during the study that require a change to the CRF, new pages are shipped to the study sites. This can be a very labor-intensive and costly process, and can have a large impact on quality assurance procedures. During the course of a study, the monitor removes the original CRF pages. At the end of the study, a paper copy of the CRF is maintained in a binder at the site. Sites have to allocate large areas for CRF storage, and often charge storage fees to maintain CRFs during the record retention period.

Figure 4. A list of outstanding queries.

In IBCTs, a draft paper CRF is initially generated based on the protocol. However, the paper CRF only needs to contain the content and format of the body of the CRF. The programming department then generates the data entry screens based on the design of the paper CRF. However, the CRF is no longer just a repository of data, but is designed to allow the backend computer systems to perform efficiently. The forms also have built-in logic and edit checks, and no longer passively accept entered data. The navigation (the visit at which each form appears), the form elements and edit checks are provided to the programming group with instructions such as: 1) which forms appear only once (e.g., medical history), 2) which forms repeat (e.g., adverse events), 3) which forms occur at more than one visit, 4) unique calculations, and 5) any study-specific instructions. Prior to system release, the CRA and the data manager review the layout, form content, and edit checks for accuracy and completeness. The use of drop-down boxes and radio buttons help to reduce spelling errors and can standardize the way certain questions must be answered (e.g., a text field cannot be entered into a numeric field). There is no printing, shipping, or storage of paper CRFs. If there are protocol amendments during the study, the CRA submits a change request to the programming group, the system is reprogrammed, and the change is automatically displayed at each study site once the sponsor accepts the change.

Monitoring. Currently, site monitoring is one of the most time-consuming and labor-intensive tasks in clinical trial management. In paper-based clinical trials, when the CRA visits a clinical study site, it is the first time she/he sees the data. At that time, the CRA verifies that the CRF data matches the data entered in the patient record and all missing, out of range, or inconsistent data have been verified. As a result, much of the interaction of the CRA and study coordinator involves resolution of missing data, out of range data, and careless transcription errors, not protocol compliance issues. The CRA will attempt to resolve the issues identified during the monitoring process with the study coordinator. However, if the study coordinator is not available, there may be a need to wait until the next visit for problem resolution. It is not uncommon for site monitors to visit a study site and find out that the site is not fully prepared for the visit (e.g., the CRFs have not been updated or filled out).

While the end-product of monitoring, i.e., patient record verification, is no different with the introduction of IBCTs, the process is more efficient. The main difference is that the monitors are much more knowledgeable about the status of the trial prior to the monitoring visit. Missing data, logic checks, spellings, and incorrect terminologies/acronyms can now be initially monitored from the home office, and the study coordinators can make corrections based on the monitor's initial review of the data even prior to the monitoring visit. If there are missing or out-of-range data, the CRA has the explanation in electronic format prior to the visit. There is no need for the CRA to make inquiries at the time of the monitoring visits as to why data are missing, out of range, or inconsistent, since these issues have already been resolved via the query process. The monitor can also have the authorization to "lock" the form to prevent the site, prior to authorization, from changing data after the monitoring visit. All of these tasks are reversible until final data lock at the end of the trial.

Figure 5. Sample status report of data entry and data review.

Project management

Study status.

Some of the most challenging tasks in the management of a clinical trial are the determination of the status of enrollment, the status of data entry, and what CRFs have been monitored. When using a paper CRF, currently each site sends a study enrollment status form to the CRA on a weekly basis, usually via fax. The CRA then transfers these data into a functional spreadsheet for review by the project team. This task is labor-intensive for both the site and the sponsor, and the data are not always complete and accurate. In IBCTs, online management reports can include, but are not limited to: enrollment, data entry, monitoring status, and adverse events. Enrollment data are current and are based on actual patient data, and not re-entered into project management software. There is no longer a need to request information from the sites or to reformat faxed information. Both project managers and CRAs can assess when to visit the site based on the number of CRF pages that have been entered into the system, but not yet reviewed in the field. Importantly, the entire study team can have access to real-time status reports. A sample status report from an IBCT is illustrated in Figure 5, where there is a clear indication of how many forms and pages were entered as well as the number of pages reviewed by both the in-field and in-house monitors.

Figure 6. Status report of safety outcomes.

IBCTs also allow for online reports that can include study-specific outcome parameters, such as safety summaries. A sample safety report is illustrated in Figure 6. This report can be made available to the medical monitor, Data Safety Monitoring Board (DSMB), and even the IRB, if appropriate.

Discussion

One of the main advantages of IBCTs is that those who know the data the best (the clinical study site) enter the data directly into the database. There are no judgments of intent, there are no illegible fields, and there are no symbols that require interpretation. In addition, all missing and out of range data are flagged and require explanations at the time of data entry. Either before or during monitoring, data discrepancies can be confirmed. The CRA now assumes some audit functions and perhaps even some of the roles of data management. Thus, sponsors must be aware that training must be adjusted as workloads are redefined.

IBCTs, when properly implemented, can streamline the monitoring and data management processes, as well as improve overall clinical trial efficiencies. Also, when managed properly, there are significant time and cost savings by eliminating double key data entry, automating the query system, reducing the time the monitor has to spend at the study site and interact with the site coordinator, and reducing the time from last-patient last-visit to database lock. IBCTs are here to stay, and eventually all companies will abandon paper CRFs and move into the electronic world.

The first step is to decide "what" the company actually wants to achieve in order to capitalize on the advantages of IBCTs and associated technologies, and ways to turn a concept into a requirement. The second major challenge is "how" to do it, which involves system design, employee training, and possibly employee redeployment. For example, companies can choose a commercially available product that may or may not completely satisfy their needs, or they may choose to develop their own system. Nevertheless, at some point in time a system will be chosen, after perhaps some frustrations, failed attempts, and minor or major system redesigns. The third challenge, and one of the most difficult parts of implementing IBCTs, has nothing to do with the fundamental technology, but rather with making the necessary changes within the sponsoring company of structure, mindset, and culture (if necessary), as well as at the clinical study sites. Choosing the right people to manage and execute the process is key to the success of any program.

Conclusion

The pharmaceutical industry has now reached a fork in the road. There will be those who will choose to stay where they are, at least in the short run, and there will be those who will embrace IBCTs. It is clear that the overall benefits of IBCTs allow for more efficient data and study management. When IBCTs are designed and managed properly, time is decreased for the following: database lock, statistical analyses, final study reports, regulatory submissions, and ultimately, market launch. In addition, time is saved, man-hours and costs are reduced, and the processes of clinical research, data management, biostatistics, and project management are streamlined. However, in order to accomplish this, companies must be willing to take the necessary steps to re-evaluate their workflow and resource allocations as they move to implementing and executing IBCTs.

References

1. M.A. Kelly and J. Oldham, "The Internet and Randomised Controlled Trials,"

International Journal of Medical Informatics

, 47, 91-99 (1997).

2. J. Kuchenbecker, H.B. Dick, K. Schmitz et al., "Use of Internet Technologies for Data Acquisition in Large Clinical Trials," Telemedicine Journal and e-Health, 73-76 (2001).

3. A. Richardson, "Planning and Running the e-Clinical Trial," Applied Clinical Trials, 28-34 (January 2003).

4. J. Mitchel, J. You, Y.J. Kim, A. Lau et al., "Internet-Based Clinical Trials-Practical Considerations," Pharmaceutical Development and Regulations, 1, 29-39 (2003).

5. J. Mitchel, J. You, A. Lau et al., "Paper Versus Web; A Tale of Three Trials," Applied Clinical Trials, 34-35 (August 2000).

6. J. Mitchel, J. You, A. Lau et al., "Clinical Trial Data Integrity. Using Internet-Based Remote Data Entry to Collect Reliable Data," Applied Clinical Trials, March, Suppl., 6-8 (March 2003).

7. N. Banik, "Evaluation of EDC versus Paper in a Multinational Asthma Trial," Presented at the DIA European Data Management Meeting. Berlin, October 1998.

Jules T. Mitchel,* MBA, PhD, is president of Target Health Inc., 261 Madison Avenue, 24th Floor, New York, NY 10016, (212) 681-2100, fax (212) 681-2105, email:julesmitchel@targethealth.com. Cynthia Ernst, BS, is with Target Health. Silvana Cappi, MSc, is with Ferring Pharmaceuticals AS, Ferring International Center, Kay Fiskers Plads 11, DK-2300 Copenhagen S, Denmark, +45 3815 0394, silvana.cappi@ferring.com. William Beasley, BS, CCRP, is with Biomimetic Pharmaceuticals Inc, 330 Mallory Station, Suite A-1, Franklin, TN 37067, (615) 844-1280, email:bb@biomimetics.com. Amy Lau, MPH, Yong Joong Kim, MS, and Joon You, BS, are with Target Health.

*To whom all correspondence should be addressed.

The authors would like to thank Joyce Hays, MS, for reviewing the manuscript.