Defining Quality Tolerance Limits and Key Risk Indicators that Detect Risks in a Timely Manner: Reflections from Early Adopters on Emerging Best Practices (Part 2)

In Part 1 of our guidance for quality tolerance limit (QTL) use in clinical research, we summarized the WCG Metrics Champion Consortium (MCC), now part of the WCG Avoca Quality Consortium, QTL Working Group discussions that explored the relationship between QTL and key risk indicators (KRIs). In this second part of the series, we will delve into the process of defining QTLs.

Steps for QTL definition

TransCelerate Biopharma has published how and when to define QTLs¹ and the link to critical to quality (CtQ) factors described in ICH E8 R1.² That approach is endorsed here with some additional detail.

The recommended steps for developing QTLs are shown in Figure 1 below. The process should begin in the study design phase when robust statistical considerations are formulated. These considerations are foundational when defining QTL parameters and limits. ICH E8 R1 sets expectations that CtQ factors be established. ICH E6 R2³ then focuses on identifying the associated critical data and processes in the study. Risks associated with these critical data and processes should be minimized where possible as part of the study design. It may be attainable, at this stage, to begin identifying possible QTL parameters to monitor the remaining critical risks related to the CtQ factors.

As the operational design of the study is developed, the CtQ factors, critical processes and data—plus any proposed QTL parameters—should be reviewed and revised. At this stage, there may be additional parameters identified that can assist with providing an early signal that the QTL threshold might be deviated from—termed “companion KRIs” by the QTL Working Group and described in more detail in Part 3 of this series.

Figure 1. Recommended steps to define QTLs

Defining QTL thresholds can be challenging. A recent paper⁴ provides some historical data and approaches to this challenge. Where QTL expectations are defined in the statistical analysis plan, it may be possible to derive a threshold from this information, e.g., the number of participant withdrawals that likely lead to the statistical power being too low for the study.

Where QTLs are typically considered

The selection of meaningful QTLs should be specific to each trial, but there are common areas that usually warrant careful consideration to apply a QTL:

• Missing or non-evaluable data for primary and secondary endpoints, e.g., data not collected at the visit, data collected outside of visit window, participant discontinuation, etc.
• Other data that could impact the number of participants eligible for key endpoint analyses, such as protocol deviations, e.g., a high rate of prohibited concomitant medications, early treatment or study discontinuation

Table 1 below provides areas where QTLs are typically considered, along with the rationale and examples.

Table 1. Typical areas that are used for QTLs. Note: there are additional examples in Table 2 of Ref 1

Table 2 below highlights typical areas where, for most trials, teams should carefully consider alternatives before applying a QTL. It may be difficult to obtain data to monitor these risks effectively via QTLs. Teams should consider whether there is a more effective way of monitoring risks in these areas.

Table 2. Areas typically not used for QTLs

Please see Part 1 of this series and watch for Part 3 highlighting methods for early detection of risk. If you would like more information on the Consortium, please visit our website.

Authors

Keith Dorricott, MBB, Senior Consultant, WCG Avoca, Steve Young, Chief Scientific Officer, CluePoints, Linda B. Sullivan, MBA (corresponding author), Senior Advisor, Metrics & Performance Management, WCG, Founder and former Executive Director, Metrics Champion Consortium

Contributors to this article include: Maureen Cunningham, United Therapeutics, Adam Czernik, Janssen, Kevin Douglass, Daiichi Sankyo, Todd Johnson, Lokavant, Olgica Klindworth, Medidata (previously associated with PPD), Crupa Kurien, Pfizer, Kim Lombardo, Moderna, Gillian Pilbrow, ICON PLC, Leslie M. Sam, Wool Consulting Group, Kristin Stallcup, Castor, Yumi Sugiura, BMS, Macarena Sahores, Senior RBQM Operations Consultant, TRI

Legal note: Contributions by the co-authors and contributors are solely their own and are not intended to express the views of their organizations.

References

1. Bhagat, R et al.Quality Tolerance Limits: Framework for Successful Implementation in Clinical Development. Therapeutic Innovation & Regulatory Science. 2020. https://doi.org/10.1007/s43441-020-00209-0
2. ICH E8 R1 https://www.ich.org/page/efficacy-guidelines#8-1
3. ICH E6 R2 https://database.ich.org/sites/default/files/E6_R2_Addendum.pdf
4. Makowski M et al. Historical Benchmarks for Quality Tolerance Limits Parameters in Clinical Trials. Therapeutic Innovation & Regulatory Science. 2021. https://link.springer.com/article/10.1007/s43441-021-00335-3
5. CTTI Quality by Design Project – Critical to Quality Factors Principles Document. Version 19 May 2015. https://ctti-clinicaltrials.org/topics/quality/quality-by-design/critical-to-quality-ctq-factors-principles-document/
6. TransCelerate Interpretation of Clinical Guidances & Regulations: Quality Tolerance Limits Initiative Frequently Asked Questions. https://www.transceleratebiopharmainc.com/wp-content/uploads/2020/09/TransCelerate_Interpretations-of-Clinical-Guidances-and-Regulations_QualityToleranceLimits-FAQs_September-2020.pdf