Enhancing Trial Master File Quality With Pareto Charts and Funnel Charts

In the world of clinical trials, maintaining the quality of trial master files (TMF) is crucial. However, the root cause of TMF quality issues is not often addressed. Due to the daily volume of TMF documents received, the people performing TMF document quality reviews cannot take time to analyze the root cause. TMF Managers also have their daily work, such as system administration and inspections, that can make initiating a root cause analysis less of a priority. This leads to hidden costs that can negatively impact the trial's progress, compliance, and integrity. We spend a great deal of time generating TMF data and equally large amounts of time QC’ing data with armies of expensive knowledge workers focused on identifying quality issues. We need to spend more time trying to eliminate the quality issues or reduce the frequency of their occurrence. This article will discuss how Pareto and Funnel charts can be used to identify and address TMF quality issues efficiently.

Hidden costs of TMF quality issues

Quality issues in TMFs have direct costs but also have a ripple effect on the entire clinical trial process. Remediation efforts are often spread across different players, such as CROs, vendors, and trial sites. The process of identifying, prioritizing, and addressing TMF issues is time-consuming and resource intensive. If a team can’t assess and prioritize the issues efficiently, they will spend precious time addressing a large volume of non-critical issues instead of focusing on those that can potentially have implications for patient safety. For example, there may be one issue with an adverse event document alongside 50 issues related to correspondence. The volume of correspondence issues may suggest that they should be addressed first. However, the singular adverse event issue could profoundly impact patient safety and should be prioritized over any administrative reporting issues.

The Pareto principle can be employed to focus on the most critical issues and maximize the impact of remediation efforts. According to this principle, addressing 20% of the issues can lead to an 80% increase in quality.

Using Pareto charts

Pareto charts can help identify the "vital few" issues that should be prioritized. Using Excel to create Pareto charts from TMF issue data, users can standardize the data representation and quickly identify trends. Nuances in the data become clearer as it is presented in this context.

Some common quality issues include misclassifications, incorrect metadata, and incorrect document titles. The visualization helps teams see the types and severity of issues in the context of the larger TMF. They can then more quickly communicate the issues and collaborate to address them.

A Pareto chart for quality issues.

Source: Michael Agard

Remediation strategies for these issues will vary. Technology can be employed to support remediation, such as leveraging AI for classification, implementing rules-based cross-checks, and automating document titles based on context rules.

Addressing long-cycle quality issues

Some TMF quality issues take longer to resolve and may be overlooked or forgotten. An expired document in a TMF refers to a document that has surpassed its validity period or has become outdated. In the context of clinical trials, certain documents have specific timeframes during which they remain valid or up-to-date. Regulatory requirements, institutional guidelines, or study-specific protocols can dictate these timeframes. Expired documents can lead to quality issues, delays, and potential regulatory non-compliance, making it essential to monitor document expiration dates and act proactively to avoid complications.

Missing and invalid signatures are also common issues. Many organizations have been moving towards adopting electronic and digital signatures to streamline the process and make it easier with which to comply. Electronic and digital signatures offer several advantages over traditional wet-ink signatures, including improved traceability, enhanced security, and easier access within the electronic trial master file (eTMF).

Quality issues that require a long time to resolve.

Source: Michael Agard

To reduce the cycle time for resolving these issues, proactive steps can be taken, such as sending reminders in advance to address expired documents or restructuring forms to clearly indicate required signatures where they are typically missing or invalid. Additionally, using artificial intelligence and natural language processing technologies to identify documents requiring translation, redacting patient identifiers, and checking legibility before uploading can help address these issues more efficiently.

Funnel charts to analyze quality issues

Funnel charts can be used to sort TMF quality issues by classification based on the TMF reference model zones. Critical documents can be prioritized by identifying the zones with the highest error rates. This approach enables a more effective allocation of resources to maintain the integrity of the clinical trial.

The funnel chart below illustrates an example of issues found in zone 5 of the TMF Reference Model pertaining to site-level documents. Due to the volume of site documentation collected in the TMF, this section deserves special attention.

Frequently, relevant communications gets top honors for the most quality issues. This is often due to the sheer volume of communications between various parties, including sponsors, investigators, site personnel, vendors, and regulatory authorities. It is also due to the need for more standardization across communications, inconsistent documentation practices, and human error.

A number of issues are related to training, CVs, the delegation of duties, and licenses. Obviously, patient safety, monitoring, and IP documents should be checked to ensure patient safety, GCP compliance, and trial integrity.

Using funnel charts to drill down into site documents.

Source: Michael Agard

Hierarchy of document criticality

In clinical trials, the hierarchy of document criticality should prioritize patient safety, followed by efficacy, training, and monitoring documents. Pareto and funnel charts can provide insights into different data cuts. Another example is looking at study country and site-specific quality issues. By drilling down into the country and trial sites, one can begin to attack the root cause of the problem areas and identify a remediation strategy, whether it is a need for additional monitoring, training, or other support. This targeted approach can lead to a reduction in quality issues in the future, but it requires an ongoing commitment to continuous improvement.

Focusing on document criticality.

Source: Michael Agard

TMF quality issues carry a significant cost in terms of time, resources, and risks to patient safety and compliance. Utilizing Pareto and funnel charts can help identify and prioritize these issues. These tools help teams focus efforts on the root causes of issues and implement effective remediation strategies. Embracing technology like AI and automation can further streamline the process and reduce workloads and errors. Combined with a commitment to continuous improvement, using these tools can improve the overall quality of the TMF while reducing the time and resources required to maintain quality in the future.

Michael Agard, team leader, clinical consulting, NNIT Inc.