Helping Rescued Adjudication Committees Get Back on Track

There are times when problems arise, and a sponsor needs to determine if its adjudication committee needs to be rescued. This article examines early indicators of risk for adjudication committees, common reasons why adjudications are rescued, mitigation steps, and indications when it is time to seek additional expertise. It shares insights regarding additional measures that can be taken to set a committee up for success and put a trial back on track.

Introduction to adjudication

Endpoint adjudication committees (EACs) are independent bodies of experts that provide a consistent, objective, and unbiased assessment of safety and efficacy events during a clinical trial. As the EAC needs to be independent from sites, participants, and the sponsor, it can prove challenging to determine whether the EAC is on track or at risk, and thus placing the trial at risk.

Adjudication processes often add a level of complexity into trial conduct and without expert management, they may impact sites, vendors, and the study analysis. Many companies do not have experts in adjudication on staff, since not enough trials need adjudication, and as a result, sponsors often look to outside experts to manage their EACs.

Impacts of improper committee management

Challenges arise if an EAC is not managed properly, or the workflows required to provide complete data to an EAC are incorrectly managed. Uncovering those challenges is critically important. Sometimes for instance, a charter submission deadline may be missed, and adjudication never begins, resulting in a growing backlog of events that will require adjudication. The EAC provider may experience ongoing and significant changes to the charter, high rates of member discordance, and missed deliverables to the sponsor or vendors that require access to the adjudication results. If delays continue, timelines will shift, and costs will rise. It is essential that an EAC provider has the requisite expertise, access to an expert network, and the tools and infrastructure necessary to deliver adjudication documents and technologies simultaneously.

Examining rescues: Taking a deeper look

Poor management of EACs

The most common reason why an EAC needs to be rescued is improper management of the committee members, adjudicated data, and overall EAC process and workflow. This may result from a vendor managing multiple trial activities, including the EAC, without adequate staff or experience or more complex projects, which require expertise in that space, taking precedent. Regardless of the reason, improper management of the EAC may impact multiple workflows, vendors, sites, and the sponsor. Common signs and symptoms of poor EAC management include:

• Lack of EAC-specific standard operating procedures, templates, and communication tools
  • Inability to drive EAC activities, data, and communication across multiple vendors
• EAC support staff concerns
  • High rate of turnover
  • Loss of knowledge between EAC support staff
  • Delays – not receiving responses within 24 hours
  • Lack of performance metrics and insights
  • EAC member surveys report unsatisfactory ratings
  • Lack of ownership and oversight of core EAC processes
• Quality concerns
  • Absence of EAC templates, procedures, or tools for sponsor review
  • Inadequate record keeping for EAC activities (i.e., manuals, charter, system training, site guidance, meeting minutes, spend reporting)
  • Inadequate EAC documentation
  • Poor metrics or EAC performance data
  • High volume of backlog events that require adjudication
  • Average adjudication time to reach decision exceeds 10 working days
  • Average time for EAC vendor to process EAC records exceeds 2.5 working days
  • No automation of EAC workflow or lack of EAC technology
  • Sponsor lacks access to EAC performance dashboard
  • EAC member turnover
  • High rate of discordance among EAC members
  • High rate of variability between EAC members uncovered during quality control re-read workflow
• Event identification obstacles
  • Challenges with event reporting process
  • Inadequate event identification workflows
  • Manual event identification processes
  • Discovery of missed events
  • Inability to programmatically detect events or detect events that require re-adjudication
  • Duplicate events in adjudication output
  • Lack of clinical and medical oversight of event identification workflow

Adjudication technology challenges

The adjudication system plays a critical role in the entire adjudication management process and workflow and should be assessed from the perspective of each user on the platform. The technology should:

• Be permissions-based
• Utilize video training modules and produce training certificates
• Track progress of all events for adjudication
• Have real-time user-generated reports and a real-time performance dashboard
• Allow users to access other EAC relevant documents within the platform
• Include built-in redaction tools for site or clinical research associate (CRA) users
• Include seamless translation workstreams for local language source record reporting
• Be easy to navigate, with global users in mind
• Be built for adjudication with EAC members in mind; it should be easy to navigate the case packet, and the platform should offer a split-screen voting option with DICOM viewer.
• Receive high customer satisfaction ratings from key users including the EAC, sites, and the CRA and sponsor teams
  • Easy to use systems result in faster turnaround times for the EAC and higher rates of on-time database locks. In one EAC rescue study, the adjudication turnaround time (time from when the event was reported to reach a decision) was reduced by an average of 23 working days.

Checklist to determine if the technology is meeting the EAC’s needs:

• Review the adjudication metrics and data
  • Review the rate of adjudication – how many events are outstanding for adjudication and what is the average time to reach a decision?
  • Review the member agreement metrics – do more than 20% of events require full EAC panel review?
• Connect with the group performing the statistical analysis to ensure they are receiving regular access to adjudication results in accordance with the transfer agreement.
• Request system access – login to the EAC technology, access reports, or seek a demonstration from the EAC view.
• Request information from the EAC vendor regarding how it seeks input from users of its EAC technology. System feedback or user pain points should be made available to the sponsor.
• Planning for the end – review adjudication results and metrics on a regular basis with the vendor (being mindful of any blinding considerations).
  • Obtain access to submission ready archive case packets throughout the adjudication either directly in the technology or by requesting them from the EAC vendor

Seek stability

Many sponsors face challenges with their EAC but are unable to pinpoint their exact cause or struggle to determine whether they should try to salvage the existing committee and vendor relationship or move their EAC to a new provider. There are many factors to consider, which usually require outside expertise, to determine the right steps for the trial.

It is recommended that sponsors seek additional expertise to assess the state of their EAC, identify risks, and form a plan to ameliorate the pain points. This may be achieved by requesting that a separate team from the current vendor performs a risk assessment, or more commonly, seeking a different provider with adjudication expertise to help uncover risks and form a plan to put the EAC and trial back on track.