Endpoint Adjudication Survey Reveals Cultural Challenges and Lacking Regulatory Guidance

The field of clinical trial endpoints adjudication is relatively unexplored; several experts argue that the field of endpoint adjudication lacks clarity, standardization, and that there are no existing definitive regulatory policies on the process.

The Clinical Endpoints Independent Review & Adjudication Group and Ethical Clinical designed and executed an industry survey, which was publicized on LinkedIn and through Applied Clinical Trials, that evaluates the impact of technology utilization in clinical endpoint adjudication.  This article will delve into some of the data to uncover opinions about traditional endpoint adjudication use (i.e., Excel, paper, emails, etc.) compared to those who used eClinical technologies to adjudicate events.  Click Here to Register for CBI’s Endpoint Adjudication Event, May 5-6 in Philadelphia.

Respondent Demographics

140 eligible respondents completed the survey from a variety of organizations and functions (Figure 1).  Most respondents are affiliated with Academic Research Organizations (AROs), CROs, and Pharmaceutical/Biotech enterprises with many respondents working in Clinical Operations, and Endpoint Adjudication.   Other respondents comprised of consulting, computing industry, and non-profit organizations.  Figure 2 breaks down the seniority of each respondent in specific organizations.

eClinical Solutions Adoption in Endpoint Adjudication

Figure 3 illustrates that 62% of respondents used an eClinical solution to implement endpoint adjudication, whereas 38% used traditional models, such as Excel, paper, and email exchange.

Figure 3 demonstrates that eClinical solution utilization for endpoint adjudication is most dominant in Endpoint Adjudication functions (77% utilization), and Data Management (83%), whereas Clinical Operations (43%), Quality Assurance (33%), and Safety (0%) exhibited the lowest eClinical solution utilization.

Figure 4 shows responses from those who used an eClinical solution (from an organizational standpoint) and the impact of the type of eClinical solution used to conduct endpoint adjudication.  Most used endpoint adjudication specific software, however, many are still using existing eClinical solutions (i.e., EDC, imaging, etc.) to conduct endpoint adjudication.   Pharmaceutical/biotech enterprises seem to exhibit the highest adoption with endpoint adjudication specific software (92%), AROs followed at 73%, CROs at 53%, and then medical device companies (0%).

Challenges with Endpoint Adjudication Software Use

Figure 5 demonstrates that respondents who used traditional methods and tools (i.e., Excel, emails, paper, etc.) for endpoint adjudication realized challenges with process inefficiencies and manual work; to elaborate, 61% of those who realized inefficiencies/manual work also use traditional methods and tools for endpoint adjudication.  Moreover, 70% of those who ranked ‘all of the above’ also were statistically associated with traditional endpoint adjudication method utilization.

Alternatively, most who used a software solution to adjudicate endpoints exhibited ‘Other’ as their biggest challenge.  Upon evaluating legible responses, we discovered two categories including Cultural Adoption (N=12, 52%) and Process (N=11, 48%). 

Cultural Adoption Challenges

Concerns with cultural adoption include difficulty with accepting/convincing others of new methods, processes and technologies to conduct endpoint adjudication, and lacking regulatory guidelines on the endpoint adjudication process.  For example, one respondent revealed, “no best practices followed in the industry.  Too much variation in processes. Could benefit from FDA or ICH guidance specific to endpoint processing.”  Another suggested, “the biggest challenge is to adopt the Event Adjudication process into a normal clinical trial with standard processes.  It takes time to implement the process and requires a change of mindset of how to use data and compare data.”  Others indicated, “acceptance and integration of algorithmic adjudication by regulators,” and, “lack of flexibility of adjudicators.”

Process Challenges

Concerns with process involve operational complexity, inefficiencies and lacking structure/consistency/guidelines on endpoint adjudication as a business operation.  For instance, one respondent indicated, “somewhat complicated in resolving disagreements. Unclear what an optimal QC result should be.”  Another explained, “obtaining source data and ensuring competent authorities, companies, EAC, and CROs are defining adjudication in the same manner.”  Another respondent said, “the need to track progress through automated reports, the ability to generate different types of metrics etc., and transfer of data into and out of the endpoint adjudication process.”

Other aspects with respect to process include poor protocol design.  To demonstrate, one respondent indicated, “lack of understanding that protocols need to be written with clear endpoints and definitions,” whereas another suggested, “it is not only endpoint adjudication, but also more subtle aspects of inclusion/exclusion and protocol compliance decisions with respect to complex subject populations that I feel will benefit from an open, traceable decision format.”

Expert Opinion

Endpoint adjudication experts are in accord with these findings.  Dr. Beat Widler of Widler and Schiemann, Ltd., said, “these findings make sense, and I would not wait for regulators or even ICH to provide guidance on the adjudication process.  I would invite interested stakeholders to convene and develop best practices through non-profit organizations such as ACRES. There is successful precedent to this approach: a DIA group convened to develop a TMF template, which is now widely used across the industry.”

Dr. Mimmo Garibbo of Ethical Clinical added, “We can all agree that using a dedicated Software Solution can improve efficiency.  However, it is also important to note that the Software Solution must be validated to meet federal and quality assurance requirements.  Additionally, we need to consider that Software Solutions must be user-friendly and flexible in order for Endpoint Adjudication staff to realize improvements in efficiency without adding new software-related assignments and subsequent work burden."

Summary

This analysis focused on uncovering trends with endpoint adjudication software utilization, and we have discovered that while eClinical technologies are increasing in adoption, endpoint adjudication specific software still has yet to penetrate some parts of the clinical trials industry. 

We have identified that traditional endpoint adjudication methods are inefficient, and those who are using eClinical solutions are not concerned with the solutions themselves, but, rather with facing cultural adoption and challenges with the adjudication process.  Furthermore, lacking regulatory guidelines on the endpoint adjudication process is contributing towards these challenges.  Stay tuned for future analyses on this industry survey.

CBI’s Endpoint Adjudication Conference

CBI’s Endpoint Adjudication Conferenceis back for May 2016! Regardless of your therapeutic area, this in-depth conference provides attendees with strategies to manage the complex adjudication process by offering practical insights for independent review committee selection and charter development as well as compiling information from various sources to provident consistent, reliable and accurate data to be submitted for adjudication.