Elements of a Successful Platform for Image Analysis

Over the past decade, the use of medical imaging in clinical trials has become increasingly important in the drive to produce quantifiable and reproducible results that establish efficacy of a given compound. Imaging is a highly specialized arena, and most drug companies have focused on outsourcing to CROs as a way to plan and execute the imaging components of a trial. The goal in choosing a CRO is to concentrate on finding a partner that can deliver reasonably priced services and the desired level of quality within what is often an aggressive time frame.

Putting aside the differences in scientific approaches to analysis, it may seem that all partners are basically the same—images are acquired, read, and results come back. However, when you start digging more deeply into the scope of a CRO's operations, real differences emerge that may determine how your partner really performs. One key area of differentiation for CROs is how these companies make use of electronic systems to streamline their operations.

A detailed look at three of the key attributes of an effective platform for image analysis follows, with hard questions for you to ask potential partners.

Security and compliance

A company's approach to the quality system regulations imposed by the FDA is an important indicator of its relative maturity within the CRO community.

Paper and ink approaches to quality records can be effective in extremely small companies but will certainly show signs of strain during times of growth. If your provider relies on screenshots of radiologists' findings stored in a binder on somebody's desk, that company is likely to lose regulatory paperwork that may be critical to your trial. The need to protect such data leads inevitably to the use of an electronic system, but what are the critical features of such a system?

Usable audit logging support. Each action that reflects a decision made during a trial should be recorded along with the name of the user who made the action, the date the decision was made, and an electronic signature that validates his or her identity. The audit log should be stored in a protected environment and accessible only to authorized personnel. Tools to view and process the information in the log should be provided. In audit situations, it is necessary to be able to find this information.

Future data availability. If the CRO goes out of business, how will you successfully reproduce the results on which your drug relies? It is not enough to simply send system backup tapes to a data storage facility and then hope you can actually do something with the data in the future. At a minimum, some kind of escrow should be in place whereby the code and operations procedures for the CRO's system are stored by a third party with a provision for your company to have full access to these assets in critical situations. Remember, you need the method as well as the results to survive in an audit.

Current security practices. The correct use of passwords, biometrics, and other authentication technologies is critical to the protection of data. Systems that store user passwords in plain text, do not encrypt communications or rely on homegrown authentication are simply not secure. In a similar vein, minimum access should be granted by role. If a user has the right to perform image inspection, this does not infer the right to perform RECIST measurements. Access should be granted on the basis of certification by an experienced user.

Beyond the paper trail. 21 CFR Part 11 lays out stringent rules for electronic signatures. Documents that will be managed electronically must comply with these regulations. Note that even if a vendor claims to be keeping paper records, substantive reliance on the electronic system means that they are subject to the provisions of 21 CFR Part 11.

Efficiency

Efficiency in CRO operations can be measured in cost, speed, and the ability to reduce the size of the subject population through the application of quantitative methods that decrease statistical variation. An effective platform brings an IT systems approach to operational efficiency through the use of integrated software components. So, what are some approaches a CRO uses to maximize its results and minimize its costs?

Concurrent multisite management. One of the key benefits of a CRO is the management of sites by experts in the emerging domain. An integrated platform should manage a database of sites and contacts that allows for tracking the site relationship throughout the trial. As the number of sites in a trial increases, tracking this information in Excel spreadsheets becomes extremely inefficient and an automated system is necessary.

Data management. Imaging trials typically involve huge amounts of data that require tasks such as quality inspection, manipulation, and analysis. Managing the quantity of data that can arrive from several hundred sites requires an advanced electronic system. It is critical that a CRO knows the state of data analysis at all times, understands how the entire trial is progressing, and can provide meaningful management reports when asked. How many subjects have been scanned? Which sites are performing best? These are important questions.

Scaling tasks. There are typically many independent tasks that must be performed on each study under examination. Some of these tasks can be performed concurrently, and each requires a certain level of expertise (or certification). Efficient processing of these tasks requires a workflow-based approach, where the tasks are allocated to the most cost-effective resource to be performed. Some tasks such as registration do not require a radiologist, and using expensive resources in these situations results in inflated costs and longer trial times. In the ideal situation, many of these tasks will be handled by automated computer algorithms. In short, if your CRO relies completely on radiologists to perform all tasks, they will not be able to scale to larger studies in a cost-effective manner.

Communication

Perhaps the most important differentiator in CRO relationships is the ability to communicate meaningful data in real time.

A successful partner can provide insights into data that you did not expect and will look for ways to provide you with that information in the most effective fashion. This can range from portals into production systems that show real-time data to emails from a project manager containing that report you need right now in the format you need it. Customized reporting that provides meaningful answers to your questions is a critical element of a successful platform. How is your CRO going to provide you with this kind of information on demand?

Information presentation. Web-based reporting tools vary from full data mining applications to simple repositories where reports are placed for you to access easily. Since it is unusual for customers to want to spend a significant amount of time building sophisticated queries, a successful CRO will provide the capability to do this for you and then use the electronic delivery capabilities to speed up your access to data.

Information protection. When sending information over the Internet or through email systems, it is critical that your CRO has separate areas for storing information for each of their customers and provides a secure repository for that information (complete with user names and strong passwords).

Conclusion

Choosing a CRO requires a detailed analysis of the scientific, clinical, and operations capabilities of each vendor. While the majority of focus is placed on the first two elements, the electronic systems used by the CRO should not be ignored.

If the CRO makes substantive use of an electronic system yet relies on paper records, they may well be in violation of CFR 21 regulations. If they do not protect both the data and method (software) used to derive results, then how will you show reproducibility in the event of a regulatory finding several years down the track? Can they scale to handle your larger trials if they rely only on radiologists with no automation of analysis? What if they change radiologists midway through a trial, and the lack of precision that is inherent in systems without automation leads to differences in analysis?

These are some of the typical issues that a platform for image-based trials can help you address.

Colin Rhodes is director of software development at VirtualScopics Inc, email: Colin_Rhodes@virtualscopics.com

References

1. 21 CFR Part 11 "Electronic Records, Electronic Signatures," http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/cfrsearch.cfm

2. Good Clinical Practices, http://www.fda.gov/oc/gcp/.

3. J.K. Shim, editor, The International Handbook of Computer Security (Lesson's Professional Publishing, London, 2000).

4. P.I. Good, A Manager's Guide to the Design and Conduct of Clinical Trials, 2nd Ed. (Wiley-Liss, Wilmington, DE, 2006).

5. S. Fisherman, Web & Software Development: A Legal Guide (NOLO, Berkeley, 2004).