Benefits of Image Analysts

Clinical trials that assess the efficacy of cancer therapies are becoming increasingly dependent on imaging techniques as surrogate endpoints for the evaluation of treatment response. Patients with malignancy typically present with lesions that require serial magnetic resonance imaging (MRI), computed tomography (CT), and/or positron emission tomography (PET) scans. Treatment decisions, particularly in reference to longitudinal assessments, are based on the information in these scans.

John Foxx, Getty Images

In evaluating response to cancer therapy, it is important to determine whether a lesion is stable, responding to treatment or progressing (growing), and if so, at what rate. However, the current state of practice at many institutions does not typically include accurate quantitative, longitudinal measurements provided in a centralized, reliable, organized, and reproducible manner. Frequently, the existing workflows are inefficient and cumbersome, resulting in an unproductive use of both the oncologists' and the radiologists' time and their staff.

A centralized image analysis service can provide increased reliability and consistency, thereby attaining greater confidence levels in generated data. However, the added cost of a central review can potentially limit trial investigators from utilizing this service. By employing a team of highly trained Imaging Analysts (IAs) to provide preliminary measurements for clinical trial assessment, centralized image services can improve cost efficiencies to reduce cost and turnaround time.

Imaging assessment issues

Response assessment criteria were established to add uniformity to radiologic evaluation.¹ While these guidelines have significantly improved the quality of clinical trial data, they have also presented a number of challenges to those responsible for the evaluation of cancer therapies.

One of the main challenges is obtaining consistent data. Numerous issues can arise when applying response assessment guidelines. For example, the interpretation of radiologic scans can be subjective, with reviewers varying in their methods for performing lesion measurements.

Each lesion measurement is dependent on which diameter a particular reviewer chooses, a task that is often completed without direct reference to what was measured on prior scans for that patient. Certain circumstances, such as the merging of two lesions into one, can further confound the results. Therefore, inter-rater reliability across different reviewers is generally low. Additionally, measurement tools are not always consistent between reading platforms, which can increase the degree of variability across imaging studies.

The lack of standardized training for radiologists and trial staff can also adversely impact data quality. Although many institutions have begun to offer seminars on such issues as data compliance and study management, training is not generally required by the clinical research industry.

Additionally, very few investigational sites and sponsors offer response assessment criteria training for the imaging reviewers. Instead, the skills that are needed to conduct a well-organized and controlled clinical research study are usually learned through trial and error, with many problems going unnoticed until data collection has concluded.

Further compounding the issue is the frequent turnover of study staff, which is a common issue among investigational sites. Without sufficient training programs, the influx of new personnel can be detrimental to the integrity of the data and the reliability of the study outcome.

Another challenge to clinical trial investigators is the absence of an image-based longitudinal record. Typically, the documentation of the radiological measurements is limited to paper records. As a result, sites frequently lack a readily available measurement assessment record for clinical trial audits. Therefore, when a regulatory auditor reviews the study progress, additional time and effort is required to collect the records of the imaging data at each investigational site, which can be costly and time-consuming for both sponsors and trial staff.

Solution

A centralized imaging core can provide many advantages to sponsors and sites. Imaging cores are open and accessible to experienced staff in all major imaging modalities and therapeutic areas, which may not be readily available at each investigational site. Advanced imaging and data reporting technologies, such as 3D advanced visualization workstations and electronic data capture (EDC) systems, can also provide flexible alternatives for data analysis and reporting. Another asset to investigators is the regulatory affairs expertise available at many imaging cores. Investigators can utilize this knowledge to strengthen the study protocol design and ensure the integrity of the data.

Additionally, imaging cores can greatly improve the efficiency of radiologic assessment by increasing the measurement consistency across reviewers and investigational sites. A central review by a limited number of trained reviewers serves to lessen measurement variability. Bias may also be reduced because the imaging core does not have information regarding treatment assignment. For this reason, a centralized imaging core can provide greater confidence, increase statistical power, and improve the ability to detect treatment response to therapy for clinical trials.

Centralized imaging challenges

Centralized imaging cores also face a myriad of challenges throughout a trial. Off-site centralized radiology review is usually done retrospectively, prohibiting the replacement of missing or discrepant scans. Because of this delay, the on-site radiologist and oncologist are responsible for study enrollment and treatment decisions. Discrepancies between the on-site and off-site review on issues such as eligibility may potentially alter the study outcome.

Furthermore, the added cost of a central review can potentially deter trial investigators from utilizing this valuable service. For this reason, imaging cores must find creative ways to improve cost-efficiencies to reduce the cost and turnaround time, thereby, making the service more affordable to smaller pharmaceutical companies and national clinical research groups.

Analyst role & training

By utilizing IAs to perform preliminary measurements of lesions on MRI, CT, and PET scans, radiologist's assessment time can be more efficiently utilized. However, to optimize the effectiveness of IAs, team members must have a working knowledge of anatomy and experience in cross-sectional imaging as well as undergo extensive instruction in clinical research principles.

In addition to training in International Conference on Harmonization (ICH), Good Clinical Practice (GCP), Health Insurance Portability and Accountability Act (HIPAA), and FDA guidelines, IAs receive training in each protocol's imaging response assessment criteria, such as Response Evaluation Criteria in Solid Tumors (RECIST),² World Health Organization (WHO),³ International Working Group Response Criteria (IWC)⁴ for Non-Hodgkin's Lymphomas, Standardized Uptake Values (SUV)^5-7 for PET assessment and volumetric measurement.⁸

With this knowledge, IAs are able to promote compliance issues and assist radiologists and investigators to ensure the integrity of data analysis and results reporting. Additionally, IAs can potentially address administrative issues, which have the potential to delay the progress of the study.

Figure 1. Comparison of measurements between two Imaging Analysts.

With expertise in image analysis applications and radiology information systems, IAs can help trial staff resolve data collection issues such as discrepant exam types or incompatible data formats prior to radiology review.

IAs can also serve as a valuable resource to radiologists by helping to alleviate common issues that might inhibit them from effectively balancing the demands of their clinical and research workloads. By helping to manage the radiologist's clinical trial workload and review schedule and functioning as a point-of-contact between the radiologists and trial staff, a well-trained and knowledgeable IA team can help facilitate the clinical trial process.

Inter-rater reliability

In clinical trials, the validity of the study conclusion is limited by the reliability of the outcome that is measured. For this reason, it is essential for reviewers to undergo comprehensive reliability assessment measures, such as inter-rater reliability, prior to study commencement to verify measurement consistency and ensure a high level of agreement across reviewers.

Table 1. Interclass correlation coefficients between each of the raters.

The IAs, blinded to clinical assessments, independently selected and measured the longest diameter of 186 lesions from 20 CT scans (see Figure 1). The CT scans included a variety of diseases, including gastrointestinal stromal tumor, gynecological, breast, and lung malignancies.

After completion by all four IAs, the measurements were aggregated and analyzed with the statistical analysis software package, SPSS 16.0. The results of this study show high agreement between the four IAs (see Table 1).

Figure 2. Correlation of measurements (in mm) between Rater 1 and 4.

Figure 2 shows the correlation of measurements between rater 1 and 4. Furthermore, only 5.4% of the measurements needed to be revised following radiologist review, reducing radiologist assessment time by greater than 40%.

Conclusion

Centralized imaging cores provide an essential service to multi-institutional clinical trials.

Clinical trials involve substantial amounts of data that are subject to intensive monitoring. By providing regulatory, medical, and technical expertise, imaging cores can assist investigational sites in the effective management of many aspects of research, enabling investigators to advance protocol planning, standardize data collection and assessment, and improve study compliance.

Ghislain & Marie David de Lossy, Getty Images

Furthermore, utilizing multimodality-trained IAs to perform preliminary radiological measurements for oncology clinical trials and establishing quality control protocols in imaging core laboratories produces:

• Decresed variability across reviewers and reduction of time points

• Increased consistency and accuracy of measurements according to the clinical trial protocol

• Improved cost-efficiencies and turnaround time.

These benefits result in improved quality for radiological assessments of tumor measurements and a more focused use of imaging modalities as predictive and prognostic biomarkers in clinical research.

Authors note

The Dana-Farber/Harvard Cancer Center Tumor Imaging Metrics Core (www.tumormetrics.org) has offices at Brigham and Women's Hospital, Dana-Farber Cancer Institute, and Massachusetts General Hospital in Boston, MA.

Trinity Urban is lead imaging analyst for the Dana-Farber/Harvard Cancer Center (DF/HCC) Tumor Imaging Metrics Core (TIMC). Gordon J. Harris, PhD, is the director of the Massachusetts General Hospital 3D Imaging Laboratory and codirector of the DF/HCC TIMC. Matthew A Barish, MD, is assistant professor of clinical radiology at Stony Brook University Medical Center and the codirector of the DF/HCC TIMC. George R. Oliveira, MD, is a clinical fellow at the University of Washington. Robert L. Zondervan is an imaging analyst for the DF/HCC TIMC. William B. Hanlon, MS, is the program manager for the DF/HCC TIMC. Annick D. Van den Abbeele,* MD, is the chief of the Department of Imaging at Dana-Farber Cancer Institute and the codirector of the DF/HCC TIMC, email: Abbeele@dfci.harvard.edu.

*To whom all correspondence should be addressed.

References

1. A.B. Miller, B. Hoogstraten, M. Staquet et al., "Reporting Results of Cancer Treatment," Cancer, 41, 207-214 (1981).

2. P. Therasse, S.G. Arbuck, E.A. Eisenhauer et al., "New Guidelines to Evaluate the Response to Treatment in Solid Tumors," Journal of the National Cancer Institute, 92, 205-216 (2000).

3. World Health Organization, WHO Handbook for Reporting Results of Cancer Treatment, Offset Publication No. 48 (Geneva, 1979).

4. B.D. Cheson, S.J. Horning, B. Coiffier et al., "IWC: Report of an International Workshop to Standardize Response Criteria for Non-Hodgkin's Lymphomas," Journal of Clinical Oncology, 17, 1244-1253 (1999).

5. L. Shankar, "Standardization of PET Imaging in Clinical Trials," PET Clinics, 3, 1-4 (2008).

6. H. Young, R. Baum, U. Cremerius et al., "Measurement of Clinical and Subclinical Tumour Response Using [18F]-fluorodeoxyglucose and Positron Emission Tomography: Review and 1999 EORTC Recommendations. European Organization for Research and Treatment of Cancer (EORTC) PET Study Group." European Journal of Cancer, 35 (13) 1773-1782 (Dec 1999).

7. L.K. Shankar, J.M. Hoffman, S. Bacharach et al., "Consensus Recommendations for the Use of 18F-FDG PET as an Indicator of Therapeutic Response in Patients in National Cancer Institute Trials," Journal of Nuclear Medicine, 47, 1059-1066 (2006).

8. A.G. Sorensen, S. Patel, C. Harmath et al., "Comparison of Diameter and Perimeter Methods for Tumor Volume Calculation" Journal of Clinical Oncology, 19, 551-557.