Independent Image Review as a Form of Risk-Based Monitoring

One of the fundamental risks to a clinical research study is the selection of study endpoints. Other risks may include data quality issues, missing data collection, inadequate site selection or training, or differences between clinical interpretations performed at the clinical site and at the independent core laboratory. This last risk may lead to ineligible subjects being enrolled into the study or censoring information. In oncology trials, censoring occurs when there is a difference between the investigator’s determination of the time of disease progression versus the core laboratory’s interpretation, resulting in a form of missing data. For example, the clinical site determines the subject’s tumor has progressed, removing them from study treatment and subsequent imaging assessments; whereas, the expert reader at the core laboratory determines the subject’s tumor did not progress and the subject should have remained on study treatment. One method to reduce this risk is to employ an independent image review of clinical trial data as presented below. It is useful to emphasize that these issues and approaches are analogous to methods employed in other aspects of clinical research as highlighted in Dr. Kosov’s analysis of the difference between site evaluation and independent expert assessment of adverse events in three lung oncology trials and concluded that overall agreement is low.¹

Core labs use expert physicians who specialize in a specific therapeutic area and have extensive clinical research experience to serve as independent reviewers. Typically, the core lab uses the smallest number of independent readers possible to reduce variability, but there are many reader models that can be employed to best meet the study requirements. The independent reviewer can confirm a patient’s eligibility for the study to determine if the site investigators are correctly enrolling patients that meet the study’s inclusion and exclusion criteria. The core lab collects and checks subjects’ data from sites and one reviewer will assess each subject’s case for all sites, or several reviewers may divide the caseload. Given the advanced technologies employed at core laboratories today, confirmation reports can be provided to the clinical sites within very short turnaround times of one to two days. Independently checking patients’ eligibility can alter the study’s screen failure rates and enrollment timelines. An independent reviewer can also confirm the presence or absence of disease, which directly impacts therapeutic response, the assessment of a potential safety finding or appearance of new disease or lesions. For example, during the bevacizumab study, it was reported that approximately 10% of subjects’ baseline radiographic scans were missing. Due to this missing data and a very high rate of discordance (50%) between the site investigator and independent reviewer’s assessment of disease response, the FDA reviewers did not have confidence in the Progression Free Survival (PFS) results, as presented within the FDA Advisory Board’s Briefing Document.² An independent review to confirm patient eligibility would have identified site enrollment or data collection issues--which may represent a potential site selection bias (e.g., enrolling subjects without required baseline image sets) or this review would have indicated that the site investigators were not interpreting the study requirements appropriately (i.e., under- or over-reporting the presence or absence of disease to meet eligibility requirements). Confirmatory reviews are typically employed to strengthen the study endpoints that can have a direct impact on study’s success and may focus on interpretation of complex clinical laboratory or blood-based biomarker tests, verifying safety requirements, reporting pathological specimens, etc. to improve the quality of data and reporting from clinical trials. 

Clinical protocol designs may include an independent review to confirm if a clinical endpoint has been met (e.g., safety finding for cardiac ejection fraction falling 20% from the baseline value) or to confirm disease status as determined by the site investigator (e.g., confirming tumor progression during the trial). The protocol design would require an independent confirmation of the site investigator’s diagnoses before the investigator is permitted to alter or discontinue patients’ study treatment. Clinical trial sponsors frequently apply independent confirmation as a risk-minimization strategy in oncology programs. Confirmation of response is extremely important in new immuno-oncology therapeutic trials where desirable treatment results may be obscured by tumor growth or flare effect before showing a reduction in the overall tumor size. The independent review would help to eliminate data censoring by the clinical sites.

An expert independent review of data can reduce differences often observed in clinical trials involving large numbers of international clinical sites. Clinical sites utilize a number of clinicians who serve as investigators and sub-investigators who perform patient evaluations. These clinicians have varied training and clinical experiences which impacts their clinical decisions and diagnoses. The sheer number of clinical assessors influencing study data may cause greater variability in clinical interpretations across a wide range of clinical sites. Whereas, an independent review involves a small, select group of experienced specialty physicians capable of providing higher-quality data with less variability in interpretation. This expert review is very beneficial to clinical trials as regulatory bodies often focus on data discrepancies (e.g., differing imaging interpretations between site investigators and independent reviewers) when evaluating a drug or medical device for market.

Clinical trial sponsors can take advantage of a panel of independent reviewers who are not associated with any of the clinical sites, to monitor site compliance with the study protocol and check for consistency across sites to ensure patient assessments are thorough and complete, as well as provide a real-time data review to improve overall data quality. The core lab checks the quality of data as it’s received from sites and works with sites to improve image quality, if necessary, to ensure the images are readable for the independent reviewers to assess. From a regulatory point-of-view, sponsors should take great care in describing the data review processes and how the data will be used and reported during the clinical trial in the study protocol

Another example of operational study risk is the failure to have experienced evaluators at sites using the appropriate assessment criteria correctly during the course of the study. This can result in misinterpretations or misclassification of subjects (e.g., incorrectly deciding if a subject has an intervention related safety issue or treatment effect) which fundamentally leads to errors, data variability from site to site or discrepancies between site and central read results. The implementation of an independent review can minimize these issues and eliminate potential bias in sites’ interpretation, which may be based on clinical information, and may influence the disease assessment instead of the assessment being solely based on tests or imaging evaluations.

Prospectively identifying and performing a risk-based assessment for the trial and implementing a plan to reduce those risks are important in a well-designed study protocol to produce high-quality data and improve overall study compliance. Clinical teams should focus on reducing study design complexity and crafting a well-designed protocol that is precise and carefully worded so that all clinical sites have a consistent understanding of the requirements and are able to generate high-quality data capable of standing up to the rigors of regulatory review for approval. When protocols require the use of novel clinical procedures, new response assessment methods, or employ clinical tests/scoring methods that are not routinely performed by investigators (i.e., investigators who lack experience in correctly applying these new methods), the study team should consider implementing an independent review methodology in the study.

The added value of an independent review is that it will identify potential issues with clinical trial data early in the study and would allow the study team to monitor, intervene and reduce study risks. Study sponsors should select a risk-minimizing approach based on the importance of study endpoints and disease assessments for eligibility, safety and efficacy in reaching critical decisions for the clinical development program and understanding the level of data accuracy required to make those decisions. The independent review of data provides a rigorous, standardized process to reduce risks for the clinical trial. 

Joseph Pierro, MD, is Chief Medical Officer of Biomedical Systems, and Kelley Atherton, MA, is a medical writer for Biomedical Systems.

References

1. Maxin Kosov MD, PhD, et al. “Interexpert Agreement on Adverse Events Evaluation.” Applied Clinical Trials. Apr 21, 2016.  http://www.appliedclinicaltrialsonline.com/interexpert-agreement-adverse-events-evaluation.
2. FDA Briefing Document Oncology Drug Advisory Committee Meeting, July 20, 2010. http://www.fda.gov/downloads/Adviso...rugsAdvisoryCommittee/UCM219224.pdf.