Norman M. Goldfarb, Executive Director of the Site Council and Executive Director of the Clinical Research Interoperability Standards Initiative (CRISI)

Twenty years ago, the practice of clinical research at sites was, shall we say, largely artisanal. Study coordinators used black pens to enter data on multi-part paper forms. One-hundred-percent source data verification (SDV) was standard. CROs boasted of their low transcription error rates. Since then, electronic data capture (EDC), eSource, centralized monitoring, remote monitoring, and risk-based monitoring have emerged, with full support from the FDA.

Study sponsors talk about the importance of quality, but if pressed, many sponsors (and CROs) will tell sites, “If you can enroll the patients, we can work with you on the rest.” Given this attitude, many sites deemphasize their investment in quality, instead leaning on site monitors to point out defects that can then be corrected to achieve minimally acceptable quality. At some sites, internal “GCP inspections” may be limited to verifying informed consent signatures, dates and versions.

If a site is getting the volume of studies it wants, study sponsors are not making threats about unacceptable quality, and regulatory compliance is adequate, why should a site invest scarce time and financial resources in quality? A semi-functional, codependent relationship seems to work just fine.

Due to these dynamics, clinical research may be the only industry in the world where the customer is responsible for the quality of the vendor’s product. Leaving aside the consequences of any deficiencies in quality, study sponsors often spend twenty percent or more of a study budget on site monitoring. Even at that astronomical cost, site monitoring is far from flawless: “On average, even experienced CRAs find only about half of the important issues presented in our monitoring visit simulation,” according to Gerald DeWolfe, CEO and founder of CRA Assessments. This problematic level of performance cannot be attributed to site monitor incompetence. Rather, it must be a systemic problem caused by the difficulty of the task that no amount of training or process improvement can ameliorate.

Site networks champion quality

The typical site management organizations (SMOs) of 20 years ago failed to realize the synergies and deliver the economies of scale needed to support overhead and pay investors a competitive return on capital.

Today, private equity is pouring into site networks because they can realize the synergies and deliver economies of scale through investments in executive leadership, technology, centralized services (e.g., business development, regulatory compliance, data entry, human resources, training and purchasing), and quality management systems (QMSs).

Site networks have found that excellent execution is a vital ingredient for success, perhaps because it is very difficult to scale a decentralized organization with problematic quality. Large-scale, high-speed COVID vaccine studies boosted sites that could deliver quality under pressure.

“Supersites” also have the scale to develop systems and processes to deliver excellence. In addition. many smaller sites deliver excellence, relying on experienced personnel and simpler processes. Some institutional sites, especially those that host embedded site networks, also give quality a high priority.

Quality concepts

Quality can be thought of as a conceptual hierarchy:

• Quality management is a system for overseeing all activities required to maintain a desired level of excellence. Quality management includes SOPs, standards and metrics, quality assurance, and quality improvement. Quality management at investigative sites involves both the site and the sponsor/CRO.
• Quality assurance is a system for ensuring a desired level of excellence. Components include qualified personnel, training, audits and reporting. Site auditing is a form of quality assurance.
• Quality control is a system for verifying and maintaining a desired level of excellence through inspections, error detection, causation analysis, and corrective actions (e.g., CAPAs). Site monitoring is a form of quality control.

Two additional quality concepts are essential:

• Quality by design (QbD) means building quality into the design of a product or service to minimize possible defects. The alternative is to “test out” defects during production and then deal with the consequences. Site monitoring tests for defects during or after production. Investigative site networks should employ QbD in their QMS.
• Fitness for purpose means delivering a level of quality that the users of a product or service consider acceptable. For example, Toyota transformed the culture of the automotive industry with six-sigma-level quality (3.4 defects in a million). Clinical research sites would benefit from clarity on acceptable defect rates. Safety-related activities require a very high level of quality, while quality standards for other activities are lower, in some cases, much lower. Site monitoring tends to lower the pre-monitored fitness-for-purpose standard for site deliverables.

Regulations and guidances

The entire US site monitoring enterprise is built on two sentences in the US Code of Federal Regulations:

• “The sponsor shall monitor the progress of all clinical investigations being conducted under its IND.” (21 CFR 212.56(b))¹
• “Sponsors are responsible for … ensuring proper monitoring of the investigation(s), ensuring that the investigation(s) is conducted in accordance with the general investigational plan and protocols contained in the IND, …” (312.50)²

FDA guidances elaborate on these regulations:

• FDA’s “Guidance for Industry Oversight of Clinical Investigations — A Risk-Based Approach to Monitoring,” introduced risk-based site monitoring to clinical research over 11 years ago.³
• ICH “E8(R1) General Considerations for Clinical Studies” (adopted as an FDA guidance) introduced the concept of QbD to clinical research. Section VII of the guidance discusses data and safety monitoring but not site monitoring.⁴
• ICH “E6(R3) Good Clinical Practice” (adopted by the FDA as a guidance) introduced the concept of quality assurance to clinical research and applied risk-based quality management to site monitoring in the ICH context.⁵
• FDA’s “Guidance for Industry, A Risk-Based Approach to Monitoring of Clinical Investigations, Questions and Answers” further elaborates on risk-based monitoring.⁶

FDA guidances do not have the force of law. Rather, they set forth the FDA’s thinking as to how regulations should be applied, so any significant deviation should have a sound rationale and be discussed with the FDA prior to implementation. For example, the most recent guidance cited above includes the following disclaimer:

• This guidance represents the current thinking of the Food and Drug Administration … on this topic. It does not establish any rights for any person and is not binding on FDA or the public. You can use an alternative approach if it satisfies the requirements of the applicable statutes and regulations. To discuss an alternative approach, contact the FDA office responsible for this guidance as listed on the title page.⁶

The above guidances demonstrate that the FDA strongly supports risk-based site monitoring and is flexible as to how it is conducted, provided safety and study-quality goals are achieved. The FDA “encourage(s) sponsors to consider a change in approach to monitoring.”⁶

• “There is a growing consensus that risk-based approaches to monitoring, focused on risks to the most critical data elements and processes necessary to achieve study objectives, are more likely than routine visits to all clinical sites and 100% data verification to ensure subject protection and overall study quality.”⁶

In other words, the FDA believes that risk-based monitoring can improve quality, regardless of the potential cost savings that appeal to study sponsors.

In 2009, the Clinical Trials Transformation Initiative (CTTI) established an ongoing program on QbD in clinical research. Numerous resources are available on their website.^7,8

The FDA’s position on QMS-based site monitoring

To determine the FDA’s position on site QMS-based site monitoring, the author asked CDER the following questions in an email:

• If a sponsor determines that an investigative site:
• (a) has a record of performing above the sponsor's fitness-for-purpose level of quality on its own, i.e., without relying on inspections by the sponsor (or its CRO),
• (b) has an excellent quality assurance program that addresses personnel qualifications, training, SOPs, inspections, audits, metrics, etc., and
• (c) receives sufficient instruction from the sponsor on the requirements of a study, would it then be acceptable to the FDA for the sponsor to inspect the site's quality assurance system rather than perform quality control on the site's study documents, data, etc.?

On September 26, 2024, CDER’s reply email did not answer this question directly. Rather, after a brief discussion of the regulations and guidances mentioned above, it concluded, “Therefore, the monitoring strategy and monitoring plan should be tailored to the needs of the trial and the specific human subject and data integrity risks of the trial.” In other words, the FDA did not say, “No.” Nevertheless, study sponsors should proceed with caution and, as per the guidance disclaimer above, consult with the FDA, as appropriate.

Implementation

Before implementing QMS-based site monitoring, study sponsors should take the following steps:

1. Assess the maturity of its own QMS to implement site QMS-based site monitoring.^9,10
2. Identify a relatively low-risk study in terms of safety, execution and exposure.
3. Obtain buy-in from relevant personnel. (CROs responsible for site monitoring may not be enthusiastic)
4. Identify a few sites with a consistent record of excellence.
5. Determine whether each site’s QMS has the necessary maturity and reporting capabilities, perhaps with an audit.
6. Prepare a plan that includes metrics for determining success.
7. Prepare a risk-management plan.

During the study, the sponsor should take the following steps at each site:

• Conduct a site monitoring visit after first-patient-first-visit, to ensure that the site understands the protocol and any study challenges or quirks. If appropriate, conduct another site monitoring visit after a new study coordinator joins the study.
• Conduct site monitoring visits, when appropriate, for quality verification and benchmarking.
• Conduct site monitoring visits, when appropriate, for training and mentoring.
• Read and analyze periodic quality reports from sites.
• Give sites feedback on their quality performance.

Conclusion

Remote monitoring, hybrid monitoring, and centralized monitoring are steps in the right direction, but they still put too much responsibility on the study sponsor and not enough on the investigative site. Relying on inspections to identify and correct errors is a fundamentally limited approach to quality. If we want quality, we need to build it in with comprehensive quality management systems at investigative sites.

It’s time to explore the next stage in the evolution of risk-based monitoring: site QMS-based site monitoring. It’s time to give sites that have invested the time and money to develop mature quality management systems the chance to prove their capabilities. It’s time to motivate more sites to develop capable quality management systems. It’s time to motivate technology providers to optimize their products for quality. It’s time to reallocate money now wasted on unnecessary site monitoring to reduce study costs, compensate sites for their QMS investments, and support mutually beneficial programs, such as sustained community outreach. It’s time to step forward into the QMS future.

Norman M. Goldfarb is executive director of the Site Council and executive director of the Clinical Research Interoperability Standards Initiative (CRISI). Previously, he was chief collaboration officer of WCG Clinical, founded and led the MAGI conferences, and published the Journal of Clinical Research Best Practices.

References

1. 21 CFR 312.50, https://www.ecfr.gov/current/title-21/chapter-I/subchapter-D/part-312/subpart-D/section-312.50

2. 21 CFR 312.56, https://www.ecfr.gov/current/title-21/chapter-I/subchapter-D/part-312/subpart-D

3. FDA “Guidance for Industry Oversight of Clinical Investigations — A Risk-Based Approach to Monitoring,” August 2013, https://www.fda.gov/media/116754/download

4. “ICH E8(R1) General Considerations for Clinical Studies,” April 2022, https://www.fda.gov/media/157560/download

5. “ICH E6(R3) Good Clinical Practice,” Draft endorsed 19 May 2023, https://www.fda.gov/media/169090/download

6. FDA “Guidance for Industry, A Risk-Based Approach to Monitoring of Clinical Investigations, Questions and Answers,” April 2023, https://www.fda.gov/media/121479/download

7. “Building Quality into the Design and Conduct of Clinical Studies: Integrating Quality by Design (QbD) and Risk-Based Monitoring (RBM) Approaches: Meeting Summary,” Duke Margolis Institute for Health Policy, January 31, 2024, https://healthpolicy.duke.edu/sites/default/files/2024-08/Meeting%20Summary_Building%20Quality%20into%20the%20Design%20and%20Conduct%20of%20Clinical%20Studies.pdf

8. “Quality by Design,” Clinical Trials Transformation Initiative (CTTI), https://ctti-clinicaltrials.org/our-work/quality/quality-by-design/

9. “Quality by Design (QbD) Maturity Model, Clinical Trials Transformation Initiative (CTTI), https://ctti-clinicaltrials.org/wp-content/uploads/2023/05/CTTI_QbD_Maturity_Model.pdf

10. “QbD Maturity Model Walkthrough and Scoring Examples,” Clinical Trials Transformation Initiative (CITTI), https://ctti-clinicaltrials.org/wp-content/uploads/2021/06/CTTI_QbD_Maturity_Model_Walkthrough_and_Scoring_Examples.pdf