eClinical Technologies Penetrate Vendor Performance Oversight

Senior level clinical operations executives and clinical outsourcing functions are hunting for new ways to enhance clinical trial vendor performance measurement on a program level, and have made some progress with the increase in eClinical technology adoption, however, much of vendor performance measurement analytics remain in the dark.   This article will elaborate on the challenges affecting the ability for senior level executives and outsourcing functions to measure vendor performance, explore new ways of measuring performance, and investigate how novel eClinical technologies enlarge the quantification of vendor performance measurement metrics.   Why is Vendor Performance Measurement Still a Challenge?  

Moe Alsumidaie

Program-level vendor performance measurement is a challenge because many factors in clinical operations involve human intervention, (i.e., is subjective in  nature and variable). For example, although the implementation of eClinical technologies enable sponsors to measure performance analytically (i.e.,  impact of trial master file quality on site startup, impact of query resolution on study timelines, effect of protocol amendments on enrollment, etc.), many factors impacting study timelines, are not easily quantifiable; to elaborate, when looking at trials in aggregate or historically, it is arduous to measure whether an improvement in enrollment performance was because of a protocol amendment, whether a recruitment campaign was implemented, or whether a sponsor intervened with CRO behavior because of an enforced performance plan.   How Can Sponsors Quantify ‘Human Factors’ in Clinical Operations?   Study teams currently oversee vendor performance through pre-established analytical indicators (i.e., enrollment, query rates, etc.), however, this approach is oftentimes reactive rather than predictive because the source of this data comes primarily from the vendor and clinical operational metrics. How can sponsors measure effective trial performance indicators, such as CRA relationships with sites, whether there are issues in operational factors and study design that are causing timeline delays, or whether the site has sufficient budgetary and staffing resources to run the trial?   Study site staff are always the first who experience issues with clinical trial design and operations, and expanding measurement analytics to obtain the site’s perspective (via surveying) can not only significantly bolster vendor performance measurement indicators, but, also offer quantifiable analytics that can be visualized on a program level, and enables study teams to react predictively and proactively. Moreover, incorporating perspectives from CROs, vendors, and sponsor study teams allows for a comprehensive trial performance evaluation, as each different perspective offers more angles on study performance issues.   eClinical Technologies Penetrate Vendor Performance Oversight   A few eClinical technology enterprises, including Clinical SCORE and CRO Analytics, have emerged to address this challenge by enhancing program-level oversight through surveying sites on vendor performance, and aggregating survey response data with analytical dashboards. The latest site surveying and benchmarking technology solution was released by Clinical SCORE (CRO Report Card), which focuses on enhancing oversight on a program level, and CRO Analytics has developed a Quality Performance System that also measures vendor performance.   Figures 1 and 2 delineate how Clinical SCORE’s CRO Report Card enables clinical operations executives to comparatively analyze CRO performance through a variety of benchmarking criteria and are able to categorize CRO performance by geographic region. Figure 3 shows how clinical operations personnel can assess critical study performance indicators via CRO Analytics’ Quality Performance Indicators.  

Impact of Benchmarking Technology on Vendor Oversight and Study Performance   “The introduction of these technologies will likely transform the way study teams evaluate vendor performance, and react to clinical trial issues proactively,”said Gary Kaplan VP of Research at Clinical SCORE. “Surveying technologies enable both study managers and executives to better understand the human factor dynamics impacting clinical trial budgets and timing. Executives can finally understand CRO performance issues; which CROs might have major difficulties across therapeutic areas, regions of the world and patient demographic classification.” Kaplan added, “Being able to compare results of clinical trials, vendor performance and sponsor personnel opinions to a normative database of successful clinical trials conducted throughout the world and in over 20 distinct therapeutic areas allows for targeted research and rapid results generation.”   Peter Malamis, CEO of CRO Analytics, said, “When we evaluate the human factor in clinical trials, it is important to attain a 360 degree perspective from all stakeholders including patients, sites, CROs, and sponsors. Using statistically validated assessments built through predictive modeling and deploying them via a cloud-based survey technology allows study teams to comprehensively diagnose and address clinical trial performance issues. This methodology also prioritizes actions according to their meaningfulness to trial performance and enables the identification of leading indicators of performance for future studies,” added Malamis.   Summary   Accessing vendor performance metrics through eClinical technologies allows clinical trial executives and vendor oversight personnel to not only better evaluate vendor performance during study execution, but also select vendors during feasibility based on historical comparative data from a normative database. Every vendor has their strengths and weaknesses, and knowing this information upfront grants study teams the authority to select the most optimal vendor based on their study’s design, and risk profile. Moreover, vendor oversight eClinical technologies empower study teams to engage in proactive and predictive decision-making, and allows sponsors to develop more comprehensive prediction models by quantifying human factors and evaluating impact against study risk and timelines.