The PUEKS Project: Process Innovation in Clinical Trial Monitoring

The monitoring of clinical trials is responsible for 22%-30% of overall clinical study costs.¹ The practice involves several stakeholders and clinical monitors with specialized skills. However, many researchers have traditionally hesitated over its efficiency and claimed that a more centralized and adaptive risk-based approach to monitoring can be a better alternative.^2,3 And now, due to regulatory changes in clinical trials, such as the introduction of the addendum to the ICH GCP E6(R2), new FDA guidelines for industry on risk-based monitoring (RBM), and the European Medicines Agency’s (EMA) reflection paper on risk-based quality management, the urgency of the RBM process implementation has increased in most biopharma establishments. This has sparked a whole range of internal projects industry-wide, despite the lack of standardization and guidance concerning many RBM procedural aspects. Thus, neither academia nor industry are able to offer a clear description of a holistic, unified, and standardized process for RBM, comprising centralized monitoring procedures. Therefore, such RBM process definition, together with derived roles and function descriptions, as well as the required tools and documents, are more relevant than ever.

As a result, two pharma-related companies and two academic institutions initiated in 2014 a research project (code name: PUEKS), the aim of which was the development of a holistic system via process integration to effectively control clinical trial risks beyond isolated RBM strategies.

The PUEKS project started in December 2014 as a winner from the research and development project applications in the framework of the Hessen ModellProjekte (Germany), and was completed in November 2016. The Hessen ModellProjekte program is coordinated by Hessen Agentur, a fully state-owned organization. The objective of this program is the promotion of innovative R&D projects. The project selection criteria are the degree of innovation, exploitation potential, and technology transfer of the project as well as the competencies of the involved partners.

PUEKS complements other cross-stakeholder collaborations such as the Clinical Trial Transformation Initiative (CTTI),⁴ TransCelerate Biopharma Inc.,⁵ and the Metrics Champions Consortium (MCC)⁶ to innovate and improve clinical trial quality management. The initiatives of these three non-profit organizations depict the need of a common understanding and standardization of risk-based quality management approaches in clinical research.

This project depicts the importance of the collaborative efforts between industry, research institutes, and academia. Thus, the project initiative of the consortium manager, Cyntegrity, a company specialized in cloud solutions for RBM, has been joined by PPH plus, a contract research organization (CRO) offering clinical development project and medical leadership and consultancy services; the Institute of Biostatistics and Mathematical Modeling at the Goethe University; and the Fraunhofer IME Project Group TMP, sited at the University Hospital, Goethe-University; all located in Frankfurt am Main, Germany.

The integration of risk-based clinical trial management processes entails a collaborative effort between company departments, academics, and across multiple clinical operations functions. Well-organized teamwork has been streamlined and boosted with fluent risk communication systems supported by online cloud platforms and shared visualizations, to capture, monitor, control, and report not only on clinical trial risks but also on clinical trial overall performance beyond the clinical operations department.

International regulations, standards (ISO) and guidance concerning clinical trials (ICH,⁷ EMA,⁸ and FDA⁹), previous and contemporary projects on RBM procedures (ADAMON,¹⁰ OPTIMON¹¹), and international risk assessment scoring systems (e.g., TransCelerate Biopharma RBM initiative,¹² MRC/DH/MHRA Joint Project for the development of risk-adapted approaches to the management of clinical trials of investigational medicinal products¹³) have been reviewed and used for ensuring compliance with clinical research regulations and guidelines and adjustment to the variable clinical study requirements. RBM benchmarking sources have been consulted for detecting the pros and cons of the currently available RBM solutions.¹⁴

The PUEKS project has been developed in observance of the recent addendum to the ICH GCP guideline, E6(R2),¹⁵ the EMA reflection paper on risk-based quality management,¹⁶ and the FDA guidance for industry concerning RBM.¹⁷ The outcomes of the project relate to each of the main risk-based quality management system (QMS) processes mentioned in the ICH Good Clinical Practice (GCP) addendum E6(R2):

• Critical process and data identification

• Risk identification

• Risk evaluation

• Risk control

• Risk communication

• Risk review

• Risk reporting

Thus, the PUEKS project has aimed at filling the gaps of the-up-till-now available methods or procedures implemented in each of the above-mentioned processes.

Project methodology

The PUEKS project has devoted two years to the evaluation and optimization of RBM. For this purpose, PUEKS used data already available from ongoing clinical studies to select substantiated key performance and risk indicators (KPIs, KRIs). Thus, the obtained data-driven KRIs have been tested with real clinical data. The power of the selected KRIs (see Table 1) in terms of risk identification and mitigation has been tested, comprising not only an evaluation of KRIs, but also site risk profiles and performance indicators.

In the frame of the PUEKS project, data from two investigator-initiated clinical trials (IIT) on autoimmune diseases (studies A and B) have been analyzed with an RBM cloud solution created by Cyntegrity:

• Study A (Phase III, randomized, parallel, multicenter clinical trial, four-year duration) involved the placebo-controlled use of a new combination of two licensed drugs, with the initiation of 44 sites and inclusion of 180 patients.

• Study B (medical device study meeting the German Medical Devices Act §23b requirements, multicenter clinical trial, 18-month duration) was comparing the sensitivity of a diagnostic method utilizing an approved device against the standard of care in a given indication, with participation of 12 sites and inclusion of 31 patients.

• Additionally, Cyntegrity’s database, comprising data from publicly available clinical trial records, has been utilized for the testing and adjustment of RBM procedures and KRIs.

For study A, PPH plus performed an analysis of the clinical study protocol, the monitoring plan, and other essential documents (e.g., informed consent form) in order to identify and select the most suitable risk indicators, as well as to develop sound standard operating procedures (SOPs) on risk-based quality management and RBM.

Based on the type of data to be captured and the related data capture system, Cyntegrity, the Institute of Biostatistics and Mathematical Modeling at the Goethe University, and PPH plus identified and developed further KRIs.

Study B has been used to detect further needs concerning SOP clarification and to enhance risk-reporting procedures.

The main list of KRIs that have been developed, tested, and implemented is shown in Table 1.

These KRIs have been complemented by centralized statistical monitoring (CSM) tools such as case report form (CRF) completeness reports, to inform about the ratio of missing critical data, and further CRF data inconsistency analysis (e.g., concerning inconsistent serious adverse event [SAE] reporting, unexpected patient visit schedules, or suspected protocol deviations).

Results and discussion

1.  Critical process and data identification

According to the ICH GCP addendum, EMA and FDA guidance documents, critical data, and critical processes are those which may threaten patient safety and rights, as well as data integrity or reliability.

Analyzing Cyntegrity’s database, as well as studies A and B, PUEKS has identified critical processes that may be inefficiently supervised by traditional monitoring methodology. Thus, quality management gaps were analyzed by a retrospective analysis of essential documents, including captured data, and clinical study procedures (i.e., data collection and informed consent). Thus, as an example, the requirements of paper and electronic data capture (EDC) systems have been checked to identify what aspects of the design of data capture systems may enhance monitoring.

Critical processes and data have been evaluated at two levels:

• System level: Adequate data capture systems, document updating systems, and investigational medicinal product (IMP)-handling procedures (i.e., GCP-compliant systems and procedures).

• Study level: Study-specific terminology clearly defined in the clinical study protocol, adequate EDC adjustment (configuration) to clinical trial requirements.

To improve critical processes and data identification, PUEKS methodology advises biopharma to implement the following tools:

• Register of audit findings, corrective action preventive action (CAPA) and lessons learned, comprising the timely communication of known audit findings and quality assurance outcomes to entire study teams, as well as preventive and corrective actions.

• Expert and experienced personnel input: Analysis of previous pitfalls and the impact of each of them on providing evidence of what processes and data are critical.

Likewise, the outcomes of continuous reviews of clinical monitoring outcomes, during clinical trial conduct, should be shared in a timely manner with all operative clinical research teams and relevant functions, beyond the particular study team. An asset for biopharma will be a list of critical processes and data provided by experts from the involved functions, similar to the Critical-to-Quality Factors Principles document created by the CTTI.¹⁸ Before identifying any risks, it is essential to understand which clinical trial design factors are enhancing or diminishing trial success.

2.  Risk identification and evaluation

Team members involved in risk identification should, above all, focus on identification of risks threatening:

• Human subject safety

• Human subject rights

• Critical study processes

• Critical data of the pertinent clinical study,¹⁷ but also risks to:

• quality requirements

• scope

• schedule

• budget

• team

• deliverables

• outcome

• IT systems (e. g., communication tools, electronic files, data capture systems)

TransCelerate,¹⁹ a non-profit organization comprising the world’s leading biopharmaceutical companies, has created an RBM knowledge center freely available online. This knowledge center includes the risk assessment and categorization Tool (RACT) used mainly for risk identification and evaluation based on the failure mode effects analysis (FMEA) method.²⁰

Although the RACT is a sufficient tool for systematically reviewing all study-critical aspects for risk evaluation and identification, it has its drawbacks.²¹ The major one is that the risk levels are assessed with high subjectivity and in disagreement with other study team members’ evaluations. This generates a bias in the risk prioritization process.

PUEKS found the need of a consensus for risk level assignments that the RACT has not been covering. Thus, PUEKS developed a risk assessment tool based on an enhanced version of the RACT originally created by TransCelerate, with the incorporation of objective definitions for low, medium, and high risk levels for each risk concerning three risk attributes:

• Severity of risk impact (I)

• Probability of risk occurrence (P)

• Risk detectability (D)

The enhanced risk assessment tool is a list of questions that facilitate the identification as well as the qualitative assessment of clinical trial risks, organized in 13 risk categories (as per TransCelerate’s RACT): safety, study phase, study complexity, subject population, technology, data collection procedures, study endpoints, organizational experience, IMP, IMP supply chain, blinding, operational complexity, and geography. The enhanced RACT enables the study team to identify risks and to evaluate the severity of risk impact (I), probability of occurrence (P), and detectability (D) by assigning a score (from 1 to 3, higher scores meaning higher risk) to each of these three attributes (same as the RACT). I, P, and D scores are multiplied to obtain a single score for each risk, the so-called risk priority number (RPN). The assigned risk scores permit single and grouped risk prioritization and the calculation of a total risk score for each clinical study.

Importantly, the enhanced RACT enables the objective assignment of risk scores regardless of the individual (team member) who is deploying it. As an example, an excerpt from the enhanced RACT is provided in Table 2 and Table 3.

In the frame of the PUEKS project, Cyntegrity has developed a publicly available cloud version of the RACT. This electronic tool offers different risk identification and assessment catalogs to adjust to the need of different clinical trials and diverse requirements of biopharma, CROs, and clinical research institutions. The cloud tool comprises as well a catalog with standardized risk level definitions developed by PPH plus (as per the enhanced RACT) to streamline risk score assignments via a more objective evaluation of risks.

Full risk identification and analysis have been performed in PUEKS for study B, which has served both the development and optimization of the enhanced RACT. The objectivity of the definitions has been verified by getting different project leaders to complete the risk assessment separately. In those cases for which the selected risk scores were in disagreement, the definitions had been revised and updated until all project managers opted for exactly the same scores. Thus, the standardization of the responses enables a more objective risk prioritization. During study conduct, the risk priorities may be automatically adjusted according to risk occurrence or the frequency of alerts triggered by the risk monitoring system or risk monitor.

3.  Risk control and communication

Controlling risks is the process of overseeing risks to timely identify and prevent foreseeable and new risks, control risk occurrence, and validate risk response efficacy during the study conduct phase.

Risk monitoring and control is not a novelty in clinical trials. However, risk control was lacking a systematic procedure for a sound deployment in clinical research.

In order to apply a systematic approach for risk-based quality management, clinical trials rely nowadays on the so-called risk indicators. A risk indicator could be defined as a metric used to measure a specific risk, with its relevant control limits (thresholds) and alerts. These alerts are signals sent to the risk owner and relevant team members that are triggered when a pre-defined risk level or threshold has been reached. PUEKS detected that identified risks were not clearly linked to the risk indicators used by the monitor and that relationship should have been established already in the clinical study planning phase.

In order to fill this gap, a coherent procedure has been developed in which each risk identified and assessed by means of the enhanced RACT has been listed in a risk register with the assigned KRIs and their respective risk priority number (RPN). This procedure enhances monitoring efficiency and medical data review by clearly highlighting those data of highest risk. This risk register provides clear instructions on risk monitoring and risk control measures, comprising the required responses to risks (mitigating and corrective actions).

Likewise, a KRI register has been generated to enable the choice of the most suitable KRIs for each risk, so that a selection of KRIs based on the quality requirements of each clinical study is streamlined for the ongoing and future clinical studies.

The timely implementation of actions to ensure risk control requires an unambiguous Risk Communication Plan in which all involved team members are informed about their roles, responsibilities, and respective communication channels.

Each study team member must be aware of what group of risks is assigned to his or her area of competence. Each risk should be clearly assigned to an accountable risk owner. In order to facilitate these assignments, risks might be categorized (as per the enhanced RACT or a similar tool) in groups according to knowledge areas or functions. The risk owner is the utmost responsible person for the control of a risk or group of risk in his/her area of knowledge and will undertake the implementation and coordination of risk monitoring activities and follow-up of corrective and preventive actions, as well as the oversight of those risk-related activities that are not directly performed by him or herself.

Each risk must be measurable, via a KRI register or equivalent tool, and have clearly defined risk tolerance or control limits, as reference thresholds. These initial thresholds will be progressively and automatically optimized with the increasing number of collected data from participating subjects according to defined mathematical models. Once a KRI is close to reaching a threshold, an alert should be triggered, and that should happen in a timely manner for the effective prevention of threats. Additionally, the timelines for regular risk monitoring and reporting should be clarified.

The type of alerts, IT systems, and the communication method (i.e., phone call, automated e-mail notifications, online platforms, etc.) should be well-known. Likewise, the clinical study team must have available a description of actions and procedures to follow (i.e., root cause analysis, investigation of serious issues or CAPA).

Effective communication requires a rigorous QMS within and across the involved organizations. A Responsible-Accountable-Consulted-Informed (RACI) matrix or similar document, together with a list of clear responsibilities, will be of great assistance to get the right reaction after risk communications from the relevant communication partners.

PUEKS fine-tuned an online RBM cloud solution developed by Cyntegrity, an automated system that continuously scans the clinical study database (comprising all type of data captured by means of EDC, comprising the eCRF, ePRO or even eTMF, if available), checks risk control thresholds for each data type, and triggers alerts to the relevant study team members (or risk owner) if any threshold has been reached or exceeded. Thus, the RBM cloud solution deploys CSM that detects data anomalies of statistical nature. This differs from traditional risk detection systems that have utilized fixed control limits. The deviations from risk and quality control limits, typically depicted in summary tables, are now enhanced by automated risk alerts and streamlined visualizations that prioritize risks according to their respective RPN. The RPN is optimized according to automatic statistical adjustment of risk thresholds as more clinical data is collected. In some cases, multiple KRIs have been used to monitor a single risk, based on the risk priority, to enable closer monitoring. As an example, SAEs have been monitored with a set of four different risk indicators (two of them to address SAE overreporting, and two additional ones to monitor SAE underreporting at sites), all of them with thresholds based on a statistical comparison of SAE reporting rates across sites. Both, overreporting and underreporting of SAEs have been controlled with an extra KRI that informs about the SAE causality relation to the IMP, which is of high priority as an endpoint directly related with patient safety. The same applied to protocol compliance concerning patient visits. Two indicators have been used, one to address the number of patients that have not complied with the visit schedule, and a second one to address the number of visits missed per patient. Both patient visit indicators have been enhanced with statistical analyses methods for a comparative evaluation of site performances, based on automatically adjusted risk thresholds, to enhance traditional indicators (supported by fixed control limit alerts).

PUEKS developed an issue management system based on KRIs assisted by CSM. Each time that the RBM cloud solution detects that a KRI reaches the defined risk threshold, the system triggers an automated alert, an e-mail communication to the person that has been assigned as the risk owner. Thus, immediate risk alerts are distributed to the relevant study team members avoiding delays caused by manual checks and manual communications. The issue management system is designed to document, track, and share with the relevant team all preventive and corrective actions, root cause analyses, and contingency plans, which are related to an issue. The cloud solution allows a team to timely and efficiently deal with expected and unexpected risk events. Moreover, it provides the essential audit trail, enabling to present a plausible story to auditors: when the issue was triggered, who was involved into its mitigation and what actions were taken.

For PUEKS, two different alert threshold levels have been defined for most KRIs:

• Medium risk alert threshold – The risk is still under control (risk lower/upper control limits not yet reached), however, there is a relevant increase on the probability of the risk to materialize. This alert informs about the need to deploy preventive actions.

• High risk alert threshold – The risk has reached the lower or upper risk control limit and corrective actions should be implemented without undue delay.

As an example, assuming that for a specific study the KRI named “rate of screening failures too high,” the risk control limit is 15% subject screening failures at a site:

• Medium risk - KRI threshold is within the risk control limits (e.g., the warning signal will be displayed if 10% screening failures has been observed).

• High risk - KRI threshold is exactly on or above the risk control limit (e.g., the warning signal will be triggered if 15% screening failures has been reached or exceeded).

The selected KRI thresholds must be documented in a KRI register. The underlying procedure for KRI threshold definitions and adjustments will be determined prior to planning the actions to be implemented, given that risk tolerability will influence the selection of the most suitable preventive and corrective actions (risk response).

An appropriate frequency and method, system or (IT) tool for the monitoring of each KRI is defined in the KRI register. KRIs related to risks of high risk level must be closely monitored, as a minimum, on a weekly basis, to ensure effective risk mitigation (e.g., critical risks to be immediately reported to the data safety monitoring board) and prompt implementation of actions if necessary.

4.   Risk review

PUEKS has developed clear SOPs concerning risk review, which was not officially conducted or completely missed in all PUEKS clinical trials.

Thus, a schedule or frequency for team meetings to regularly review and update the enhanced RACT or an equivalent tool, risk register and KRI register should be established in the risk management plan (RMP) or similar document. These reviews may be supported by the implemented IT systems, so that updates can be performed automatically. The RMP should define topics to be regularly discussed between the project or risk manager and team members during these meetings, including, but not limited to:

• Whether new risks were identified since the last meeting.

• Whether risk impact (I), probability (P), and detectability (D) levels are still accurate (based on observed risk trends or occurrence).

• Whether planned risk responses and actions are still appropriate and effective to avoid or mitigate risks.

• Whether deployed risk responses and actions were effective in avoiding or mitigating risks.

• Whether any of the initially accepted risks have occurred; and if so, mitigation actions should be discussed.

• Study deliverables and general study performance (achieved milestones, met timelines, data quality checks and data reviews, IT checks, etc.).

• Risk management process improvement.

If a deficiency in the risk-based QMS (e.g., inadequate risk control) is detected, the project manager and team members should meet, discuss solutions, and improve the affected processes, checking the efficacy of the agreed corrective actions after their implementation. In such a case, the project manager or designee must organize and direct an on-demand risk review and evaluate the need of conducting a formal CAPA.

5.   Risk reporting

How quality assurance (QA) and quality control (QC) system deviations need to be reported depends on the complexity and duration of the trial. Thus, the ICH E3 guideline states that the regulatory authorities should be consulted for agreeing with a format that will enable the overview of quality management compliance in situations where detailed reports are not practical due to their extent. In such cases, risk reporting would need to be simplified. Therefore, it is not possible to standardize the formats of risk reports.

After PUEKS, an RBM cloud solution enables the export of summary tables with the most relevant deviations from the quality control limits based on risk relevance. Thus, it is possible to select deviations of low, medium, or high risk, to enable risk report adjustment and simplification. Thus, the RBM cloud solution improves data visualization as well as communications with and reporting to the regulatory authorities.

Efficacy and safety data should be briefly summarized and depicted in the relevant tables and figures, focusing on any new or unexpected findings. Thus, the KRIs selected by the PUEKS consortium are based on key safety and study performance parameters directly related to critical issues such as SAEs, events resulting in withdrawal and deaths, and lack of compliance (i.e., ratio of missing visits).²²

The risk and quality management approach (risk-based QA and QC system) implemented in the trial should always be briefly described in the final clinical study report (CSR). Any important deviation from the predefined quality tolerance limits must be reported together with the corrective actions taken for mitigating risk recurrence (ICH E3, Section 9.6 Data Quality Assurance).²²

Efforts toward QA and standardization of the clinical study team and investigator performance have great relevance (i.e., audit procedures should be summarized). PUEKS has detected the inadequate or incomplete standard terminology definitions for the collection of accurate, consistent, complete, and reliable data, such as training sessions, monitoring of investigators by sponsor personnel, instruction manuals, data verification, cross-checking, use of a central laboratory for certain tests, centralized clinical data reading and evaluations, or data audits.

Conclusions

Briefly, the PUEKS project has principally achieved three objectives:

• It optimized the overall risk-based QMS and clinical monitoring process with a well-defined description of the involved procedures.

• It created proper risk evaluation tools and KRIs supported by CSM.

• It developed and applied intelligent technologies (IT systems) for the implementation and integration of the created risk evaluation tools and KRIs.

The key outcomes of the PUEKs project are the following:

       1. Concerning the optimization of the overall clinical monitoring process:

• A comprehensive SOP on risk-based clinical trial management (PPH plus) comprising issue management (e.g., CAPA).

• A general RBM SOP as monitoring framework followed by type-specific SOPs concerning site selection, initiation, interim monitoring, and close-out visits, respectively (PPH plus).

       2. Concerning the creation of proper risk evaluation tools and KRIs:

• A catalog for risk identification, assessment, and categorization (enhanced RACT) with new specific risk level definitions (PPH plus) based on and incorporated into the RACT template developed by TransCelerate.

• Risk indicators, with their respective metrics and tolerance limits, as well as the procedure to identify and manage (including KRI and KPI setup and adjustments prior to and during clinical trial conduct) the most effective KRIs and KPIs based on specific clinical study protocols (Cyntegrity, Institute of Biostatistics and Mathematical Modeling at the Goethe University, PPH plus and Fraunhofer IME TMP).

       3. Concerning intelligent technologies (IT systems):

• A platform with a cloud version of the RACT: a creation of a publicly available web-based platform (Cyntegrity), comprising electronic catalogs for easy and fast clinical trial risk identification and evaluation. The cloud version of the RACT enables information sharing and audit trail.

• Mathematical methods for CSM of data quality and integrity as well as patient safety issues based on clinical trial requirements (mathematical modeling at the Goethe University, PPH plus and Fraunhofer IME TMP).

• The RBM cloud solution:¹⁴ optimization of the cloud RBM system, which enables real-time risk monitoring based on pre-defined KRIs and KPIs, supported by CSM, and delivers in-time alerts according to the defined tolerance limits and upper and lower risk level control limits. The RBM cloud solution enables risk communication (e.g., risk alerts or ticketing, and risk ownership or responsibility assignments), continuous risk monitoring, risk reporting (e.g., self-explanatory risk visualizations with customized charts and tables), and audit trails.

Remarkably, the accurate description of standard terminology is an example of a critical process that is overlooked. Appropriate term and process definitions will decrease the number of deviations from the desired data collection and evaluation approach (to be summarized in the final CSR), reduce negative impact on data reliability as well as safety and quality issues to be flagged up by regulatory agencies and inspectors.

PUEKS’ objective has not been the simple development of KRIs or testing RBM. It has been far more ambitious, aiming at the development of a sound risk-based quality management framework for enhancing patient safety and data integrity.

With this project, a systematic approach to de-risk clinical trial operations has been developed, from the clinical trial design phase to CSR submission. Thus, the PUEKS consortium has created a holistic approach for a successful implementation of the risk-based QMS after ICH GCP gap analysis boosting the prevention of well-known risks to clinical trials. Additionally, a proper QMS enables the team to identify and react in a timely manner to unknown risks that may arise during study conduct, too, by means of regular risk reviews. On top of this, the continuous improvement of the QMS based on previous lessons learned, during and after each clinical trial, became the best asset of the risk-based approach. These lessons learned facilitate the early recognition of known risks and rapid implementation of the most effective preventive actions. Previously blurred or unstructured information can now be quickly utilized for risk mitigation and enhanced clinical trial outcomes.

The PUEKS project (HA project no. 448/14-38) was funded in the framework of Hessen ModellProjekte, financed with funds of LOEWE – Landes-Offensive zur Entwicklung Wissenschaftlich-ökonomischer Exzellenz, Förderlinie 3: KMU-Verbundvorhaben (State Offensive for the Development of Scientific and Economic Excellence).

María Proupín-Pérez, PhD, is Senior Project Leader at PPH plus GmbH & Co. KG. Matthias Fehlinger, PhD, is Senior Project Leader & Head of Quality Management at PPH plus. Artem Andrianov, PhD, MBA, is Managing Director & CEO of Cyntegrity Germany GmbH. Johanna Schenk, MD, FFPM, is Managing Director & Chief Operating Officer of PPH plus. Martin Koch is Chief Software Architect of Cyntegrity Germany GmbH

* Acknowledgements: The authors would like to acknowledge the guidance provided by Mr. Manuel Sturm, Project Manager at the Hessen Agentur (Wiesbaden), the essential contribution to the risk indicators optimization by Professor Eva Herrmann, Director of the Institute of Biostatistics and Mathematical Modeling, Goethe University, Dr. Natalie Filmann, and Dr. Yusuke Asai, Researchers at the Institute of Biostatistics and Mathematical Modeling, Goethe University (Frankfurt am Main), and the support provided by Dr. Frank Behrens, Head of Clinical Research at the Project Group Translational Medicine and Pharmacology (TMP) of the Fraunhofer Institute for Molecular Biology and Applied Ecology (Fraunhofer IME).

References

1. Risk-based Monitoring: Industry Guidance on Adoption, Use, and Outsourcing. Industry Standard Research, 6 May 2013.

2.  N. Sheetz et al., “Evaluating Source Data Verification as a Quality Control Measure in Clinical Trials,” Therapeutic Innovation & Regulatory Science, November 2014, Vol. 48 (6), 671–680.

3. R. Olsen, A. R. Bihlet, F. Kalakou, and J. R. Andersen, “The impact of clinical trial monitoring approaches on data integrity and cost-a review of current literature,” Eur. J. Clin. Pharmacol., April 2016, Vol. 72 (4), 399–412.

4. Alberto Grignolo. The Clinical Trials Transformation Initiative (CTTI). Ann Ist Super Sanità 2011,  Vol. 47 (1), 14-18. CTTI website. Consulted online on 20 July 2017: www.ctti-clinicaltrials.org

5. Ten Pharmaceutical Companies Unite to Accelerate Development of New Medicines. PR Newswire online news of 19 September 2012. Consulted online on 20 July 2017: www.prnewswire.co.uk/news-releases/ten-pharmaceutical-companies-unite-to-accelerate-development-of-new-medicines-170329496.html TransCelerate Biopharma Inc. website. Consulted online on 20 July 2017: www.transceleratebiopharmainc.com

6. Metrics Champion Consortium website. Consulted online on 20 July 2017: www.metricschampion.org

7. Guidelines of the International Council for Harmonization (ICH). Consulted online on 16 January 2017: http://www.ich.org/products/guidelines.html

8.  European Medicines Agency (EMA) regulations on clinical Trials. Consulted online on 16 January 2017: http://www.ema.europa.eu/ema/index.jsp?curl=pages/special_topics/general/general_content_000489.jsp&mid=WC0b01ac058060676f

9. U.S. Food and Drug Administration (FDA) regulations on clinical trials. Accessed on 16 January 2017: http://www.fda.gov/ScienceResearch/SpecialTopics/RunningClinicalTrials/ucm155713.htm

10. Consulted online on 16 January 2017: http://www.adamon.de/ADAMON_EN/Home.aspx.

11. Consulted online on 16 January 2017: https://ssl2.isped.u-bordeaux2.fr/OPTIMON/Documents.aspx. OPTIMON project plan consulted online on 16 January 2017: https://ssl2.isped.u-bordeaux2.fr/OPTIMON/DOCS/OPTIMON%20-%20Protocol%20v12.0%20EN%202008-04-21.pdf

12. Consulted online on 16 January 2017: http://www.transceleratebiopharmainc.com/assets/risk-based-monitoring/

13. Risk-adapted Approaches to the Management of Clinical Trials of Investigational Medicinal Products. MRC/DH/MHRA Joint Project, 20 October 2011. Consulted online on 16 January 2017: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/343677/Risk-adapted_approaches_to_the_management_of_clinical_trials_of_investigational_medicinal_products.pdf.

14. C. Hurley, F. Shiely, J. Power, M. Clarke, J. A. Eustace, E. Flanagan, P. M. Kearney. Risk based monitoring (RBM) tools for clinical trials: A systematic review. Contemporary Clinical Trials 2016, Vol 51, 15–27.

15. ICH Harmonized Guideline - Integrated Addendum to ICH E6(R1): Guideline for Good Clinical Practice E6(R2). International Council for Harmonisation (ICH), 9 November 2016.

16. Reflection paper on risk based quality management in clinical trials. European Medicines Agency (EMA), 18 November 2013.

17. Guidance for Industry Oversight of Clinical Investigations - A Risk-Based Approach to Monitoring. U.S. Department of Health and Human Services, Food and Drug Administration (FDA), August 2013.

18. Clinical Trials Transformation Initiative Quality by Design project – Critical to quality factors principles document, 2015. Visit: www.ctti-clinicaltrials.org/sites/www.ctti-clinicaltrials.org/files/QbD_ toolkit/Principles%20Document_finaldraft_19MAY15.pdf

19. See TransCelerate’s RACT tool available at: http://www.transceleratebiopharmainc.com/assets/rbm-assets/ (Consulted online on 7 March 2017).

20. ICH Harmonized Tripartite Guideline: Quality Risk Management Q9. International Council for Harmonization (ICH), 9 November 2005, 11. 

21. Moe Alsumidaie et al. “Data from Global RACT Analysis Reveals Subjectivity.” Applied Clinical Trials, 11 May 2016. Consulted online on 03-Mar-2017: http://www.appliedclinicaltrialsonline.com/data-global-ract-analysis-reveals-subjectivity.

22.  ICH Harmonized Tripartite Guideline: Structure and Content of Clinical Study Reports E3. International Council for Harmonization (ICH), 30 November 1995.