SOPs in Clinical Research

Clinical project managers, monitors and assistants in the pharmaceutical industry involved in conducting clinical trials are experiencing increasing regulation. This seems to be more of a hindrance than a help in their daily work. As well as being expected to be aware of the manifold legal requirements and international regulations such as the German Drug Law (AMG), GCP Regulation, ICH GCP, various European directives, and FDA guidelines to boot, in the case of global trials, employees are also confronted with an array of sponsor-generated quality management documents which frequently seem to consist of a tangled web of rules and regulations. The result is employee dissatisfaction and demands for improvement of the content, a reduction in the number of SOPs, and improvement in the way the content is presented. This situation is due in no small measure to the fact that there has been little or no debate to date on what constitutes "adequate quality" of an SOP system.

The literature has comparatively little to say about what an optimal SOP system can and should look like. Most authors limit themselves to very basic suggestions (e.g., short, meaningful sentences, clarity, use of illustrations)¹ or alternatively very detailed instructions,² which are ignoring basic elements of instructional design. GCP requires "the setting up and maintenance of quality control systems supported by written SOPs,³ but provides virtually no guidance on SOP system design.

Only a few articles looked into the complex regulatory environment of the pharmaceutical industry⁴ and the resulting principles of writing effective SOPs.^{5, 6}

SOP benchmarking project

The German Association of Research-Based Pharmaceutical Manufacturers—Verband Forschender Arzneimit-telhersteller (VFA)—appointed a taskforce from the Clinical Research and Quality Assurance subcommittee to look into the topic. The VFA is an organization of 45 research-based pharmaceutical companies whose studies, between them, make up a significant portion of clinical trial activity in Germany. Responses in this survey were obtained from several companies who belong to the leading pharmaceutical companies in Europe. The aim of the SOP Benchmarking project was to evaluate existing SOP systems on the basis of existing best practices and benchmarking, and to develop proposals for potential improvements. Existing SOP systems including the associated training were to be analyzed and improvement proposals were to be designed as appropriate on that basis. Another important issue was to determine whether specific SOP concepts were associated with higher "satisfaction" on the part of the parties concerned in their respective functions. The survey was conducted in late autumn of 2009 to determine the current state of affairs among VFA member companies, identify the mood/opinion of their employees with respect to the current situation, and present proposals as to what sponsors could do to change their SOP systems or remedy any deficiencies and inadequacies.

The questions related only to SOP systems used to conduct Phase I to Phase IV clinical trials and did not cover health outcome studies or non-interventional studies.

Description of project/survey conduct

Two questionnaires were designed for the survey: Questionnaire 1 (Table 1) about the cost and benefit of SOPs looked at the existing structure and size/scope of the company's existing SOP system in terms of GCP, SOP management, and presentation of SOP content. The addressees were management and organizational employees (e.g., medical directors, directors of clinical investigation, and quality assurance directors).

Questionnaire 2 was addressed at clinical research employees (i.e., monitors and project managers of the sponsor and freelancers/employees of CROs involved/in sourced by the sponsor for study projects). Questions were asked about handling and satisfaction with the existing SOP system/training methods, and respondents were asked to make proposals for possible future systems and methods.

Replies from the two surveys were encrypted by VFA via company code so that it was possible to match management responses to employee responses, but without revealing the identity of individual companies and persons in the evaluation.

Results

A total of 18 companies took part in the survey, the majority of which is represented by a member in the already mentioned subcommittee. The results hence reflect relevant experience of SOP systems in the pharmaceutical industry. In nine cases the company was the local affiliate of a global corporation. Three to four responses per company were received from the employees invited to take part in the survey in the operative line departments (67 total); the majority were company project managers and/or clinical monitors with an average of 10 years of job experience.

Estimation/user satisfaction with the currently established SOP system. Irrespective of their function or role in clinical research and time with the company, 19% of respondents were fully, 57% partly, 16% less, and 8% not at all satisfied with the current SOP system in their company. The main criticisms of existing SOP systems concerned the complexity and lack of clarity of individual documents/SOP systems, which made it more difficult for users to rapidly seek and find the relevant sections/instructions required for day-to-day work or in a specific on-site situation. In many cases, instructions concerning parts of processes (e.g., study initiation) are spread out among different sections of an SOP or even among a number of different documents (SOPs, instructions, appendices, etc.).

SOPs with large numbers of pages did not necessarily receive poorer ratings. Instead, SOPs were criticized for being unclear and/or overloaded with abbreviations and jargon. Documents intended to be globally applicable were frequently mentioned in this context; respondents said global documents tend not to contain clear information/delimitations for each area (global/local functions). The associated need for additional material on local processes was seen to result in unreasonable multiplication of process descriptions. Users saw clear potential for improvement in implementing a level of document detail suited to the specific tasks and responsibilities, combined with a rapid and accurate search feature. Almost half of the respondents reported little or no personal involvement in the creation of SOPs. A mere 15% of those involved in document compilation said they were actual authors; most were involved only in the review/feedback process. Involvement in SOP compilation did however not correlate with level of satisfaction.

SOP length/level of detail. Only 27% of respondents were happy with system size/scope and less than half (48%) were happy with the level of detail. Detailed evaluation showed that monitors tended to be more satisfied with lengthy SOPs than project managers. The same applied for satisfaction in relation to time spent working in the company. The SOP system was increasingly felt to be too lengthy with increasing time spent working in study management. In contrast to size/scope, there were no major differences in satisfaction between monitors and project managers in respect of level of detail. Here again, however, there was a trend toward dissatisfaction with overly detailed descriptions as a function of time spent working for the company (> 10 years of job experience).

Comprehensibility/clarity of SOPs. Twenty five percent of respondents said existing SOPs were not coherent and comprehensive. This opinion was more commonly expressed by project managers than by monitors. There was no correlation with time spent with the company. Thirty five percent of respondents said the SOPs were not clearly structured, monitors more so than project managers. Systems with fewer pages (in reference to monitoring) also tended to be rated more favorably.

Are SOPs useful in your daily work? SOPs were primarily found to be useful (approximately 85%), irrespective of duration of job experience, even more so by monitors than by project managers.

Are SOPs more restrictive than the legislative framework? About two-thirds of respondents said internal company SOP systems were more restrictive than applicable law. Project managers tended to find them even more restrictive than monitors.

There was a trend for a negative rating of all the sub-elements of scope/size, level of detail, comprehensibility, clarity, usefulness, and restrictiveness from employees who were dissatisfied with the SOP system overall.

Existing SOP systems in clinical research. The global and/or local SOP systems for conducting trials that exist alongside legislative requirements differ greatly between companies. The survey shows that there is neither a uniform hierarchy/structure in respect to SOP documents, nor in terms of nomenclature of document types. To name just a few, the terms employed include SOP, policy, operating instruction, and working instruction. There is also a plethora of different annexes such as flow charts, tables, job aids, completion guides, etc. The reported estimated length of SOP systems ranged from a few pages to complex SOP constructions with an estimated number of up to 8,500 pages.

Comparison of job areas and processes with a largely pre-dictated scope of regulation (e.g., monitoring, SAE reporting, audit conduct, and collaboration with third parties/CROs) gives a similar picture. Again, there are huge differences between SOP systems. For monitoring (site selection/pre study visits, study conduct, study completion, and close out visit), the scale ranged from one to 35 SOPs (Figure 1) and from 10 to 700 pages. The mean number of SOPs a study manager/project manager in clinical research had to be familiar with was approximately 50 SOPs/480 pages.

Increase and overregulation of processes—potential for reductions. Whether and how subjective dissatisfaction in recent years correlates with SOP proliferation is evident from the following statements from management staff. About 40% said there had been no change regarding the estimated number of global or local SOPs. Thirty percent had an increase of less than a 10%; 30% of more than 10% increase per year. None had a reduction in the past 12 months. Respondents said processes were overregulated (not including those relating to clinical trials but not originating in clinical research) in about 5% (median) of all processes described.

Few ways of streamlining processes or eliminating SOPs showed up. Half of the companies' management staff surveyed saw little or no potential for reductions (two SOPs on average). The vast majority was unable to identify SOPs that could be done without. In contrast, clinical operations staff felt that a median of 10% of SOPs they have to comply with in their day-to-day clinical trial performance were superfluous.

Conversely, gap analysis revealed little or no need to extend the SOP system. Only two companies reported a specific need for regulation in particular areas.

SOP compilation and SOP quality. According to the respondents, corporate headquarters are responsible for SOP authorship in 60% of the surveyed companies. More than 70% of the surveyed companies involve the line departments and about 50% involve quality assurance in SOP compilation. One-quarter of the companies involve SOP writer specialists in drawing up SOPs. Affiliates and subsidiaries are involved in the generation of global SOPs in addition to corporate headquarters. Sixty five percent said they were included in discussions about new SOPs, and their feedback is adopted/integrated in documents in more than 70% of cases. The expertise of SOP authors was largely rated favorably; only 10% of respondents said SOP authors tended to be far removed from reality.

In about 70% of companies, additional local processes and SOPs are generated to supplement the corporate regulations in order to reflect and implement local/regional needs and legislative requirements.

SOP deviations and violations. Three-quarters of the surveyed monitors and project managers said they sometimes had to deviate from an SOP in their day-to-day work. The main reasons cited for necessary deviations were "unfeasibility," as well as (lack of) time, effort, and cost. About 45% of employees said they never deviated from an SOP. Almost the same number (about 42%) said they deviated one to two times per year, and only a few said they had to deviate more than three times or five times per year.

Deviations resulting from a discrepancy between global SOPs and local legislation are handled in different ways. About 50% of companies draw up local modifications. About 30% put national laws ahead of company instructions and implement this policy without any formal modification/amendment of the global SOP.

SOP cycle and change management. Respondents said one reason for violations and dissatisfaction among those working with SOP systems was the compilation process itself or the generation period/revision cycle. About 40% of management employees said that new or revised SOPs were often implemented unexpectedly and without "prior warning" by corporate headquarters. The SOP generation period from first draft through feedback periods to approval and release lasts 10 weeks on average. Lead times from announcement to enforcement of an SOP were four weeks on average.

In addition to regular review of SOPs at intervals of two to three years, there are other important reasons for revision. In more than 80% of reported cases, a change in legislation was the reason for an SOP revision or creation of a new SOP. A similar situation applied (almost 90%) with respect to structural changes within companies that entailed adaptation of processes and responsibilities.

SOP training. Two-thirds of the companies surveyed said learning SOP content was primarily the job of the individuals concerned. Nonetheless, formal learning was deemed the best method of learning (80% of companies that said this was mostly or occasionally the case). Two-thirds of companies rejected the prospect of ad-hoc training provided only to meet an acute need and if a particular process was specifically required.

Seventy five percent of employees also said self-study was one of the most important ways of learning SOPs. Even more respondents (88%), however, said face-to-face training was the best learning method. Less than half of respondents (43%) said e-learning was the best method. However, acceptance of this method increased if it was linked with review of learning outcomes (58%).

Despite significant variation among the surveyed clinical research management, self-study accounted for the major part of training effort, accounting for a median of 70% of total time required, followed by e-learning (median of 40%) and face-to-face/classroom training (median of 15%). Training with formal review of learning outcomes accounted only for a median of 7.5% of training events.

Regardless of training events, the surveyed companies said the SOP competency of employees is not reviewed systematically. One-third do not regularly check competency, 15% only in isolated cases, if the manager notices deficits or an audit identifies any deficiency. Fifteen percent of companies check their employees' knowledge of the main SOPs at least once a year. Training units are repeated every one to two years in 56% of companies, and at intervals longer than two years in another third.

The open-end answers clearly show that learning modalities cannot be viewed in isolation from the complexity or novelty of content, from the overall structure of the training system, or from the individual employee's level of experience: "The best setting to choose depends greatly on the content of the SOPs." Respondents also explicitly suggested combination with other learning modalities: self-study and online training for induction, face-to-face training to deepen understanding and for discussion.

An average of 11 days per year is devoted to training, but this figure varied significantly among the respondents (Figure 2). The three-peak distribution shows peaks at five, 10, and 20 days. Monitors and project managers were asked which quantity of training they would consider to be the most appropriate. The answer was approximately nine days, again with significant variation. The breakdown here again shows systematic differences between the relevant user groups.

Cooperation with CROs. In respect of cooperation with CROs, it emerged that sponsor SOPs were used as a basis for study conduct in approximately two-thirds of cases. CRO SOPs were used in less than one-third of cases.

Discussion of results

The baseline hypothesis (i.e., that SOP systems in GCP as they now exist would be rated inadequate by corporations and users) is not strongly supported. Although analysis of survey outcomes reveals deficiencies in the system and a need for improvement and change in some areas, the benefit of SOPs is nonetheless accepted. Nor did the survey results confirm the often cited suspected widespread overregulation of SOP systems.

Not every SOP system is the same, and not every SOP is equally suitable for every organization. What works for one company need not be the best solution for another.

Authors of SOPs tread a thin line in general, because user expectations and needs are as diverse as the spectrum of SOPs in existence. The needs of career starters differ from those of employees who have worked in the company or in clinical research for longer. Differences are also apparent in terms of the roles and responsibilities of respondents, for instance between monitors and project managers. Globally operating companies face the challenge of implementing global policies into local practice in a way that accommodates local legal requirements and best practice but at the same time does not violate global SOPs.

The various aspects are looked at in greater depth in the following:

Acceptance. SOPs are described overall as too lengthy. The subjective feeling of many respondents seems to be that "less is more."

Many respondents cited the difficulty in finding individual process steps within an SOP system as being a key problem. "Details of a single process need to be searched for in multiple SOPs." "Search features in the electronic SOP repository are ineffective." "Sometimes it is not easy to find the right SOP you need for a particular process."

The size/scope of SOP systems evidently differs greatly between companies indicating that SOP systems can display significant differences and may not always be directly comparable. The total number of SOPs and in particular the estimated number of pages—8,500 for one company—showed a wide range of variation. The only explanation is that SOPs may be written as no-frills "instructions" (who does what and how), as well as guidance (e.g., every single document, be it a simple letter to an investigating site, trial protocol template, or CRO communication plan). The latter policy may lead to a situation where important information and instructions for employees are hidden in a multiplicity of annexes/working instructions/templates to SOPs, which employees have to read and wade through even though they may not be immediately relevant to the specific (sub) task in hand. The former bears the risk of heterogeneous implementations.

As already mentioned at the outset, existing regulations do not dictate the depth of regulation in SOPs. ICH GCP section 5.1.1 merely states that sponsors are obliged to implement an effective quality management system with written SOPs. Thus, the decision on depth of regulation is a task of the company.

There may be several conditions under which companies increase the depth of regulation. For various reasons, in the case of inspection requirements, companies tend to set down processes in the greatest of detail and meticulously define every single step. This extends to detailed specification of formats for templates/forms. The aim is to provide employees with every possible aid in order to prevent errors.

It is therefore no surprise, as the survey shows, that the level of detail is seen to be either helpful or restrictive depending on the individual employee's level of experience. A high level of regulation also carries the risk of making more mistakes and unintentionally violating an SOP. This increases the need for deviations either on the basis of pre-approved waivers or post-hoc documented violations.

Globally operating companies have an additional problem: global SOPs and procedures may need to be expressed in very general terms to ensure that they cover all responsibilities and areas throughout the organization and should not interfere with country-specific regulations. The result is that local employees may find the global SOP to be inadequate and may encounter local situations that are not sufficiently addressed in global SOPs. To get around this problem, 70% of companies draw up local supplements. In case global SOPs do interfere with local regulatory requirements, the latter prevail.

Size and structure. Analysis of the survey shows that the quantity and variability of process descriptions does not seem to correlate with employee satisfaction with the SOP system. Although the subjective impression is that the amount of regulation in companies has increased in recent years and is continuing to increase, it is felt that there is little leeway for reducing SOP systems. Very few respondents made specific proposals for SOPs that could actually be done without.

In most companies, the existing SOP systems fully cover regulatory needs. Only a few marginal areas/exceptions were mentioned which still have to be covered by SOPs in individual companies.

SOP management. The survey showed that some SOPs are implemented with no lead time and may be implemented even in normal situations within as short a period as four weeks. In the authors' experience, this would generally be an insufficient period of time to give employees the chance to familiarize themselves with the new procedure and complete the necessary training.

There may be good reasons why SOPs are modified at very short notice, for instance, an imminent regulatory requirement or the need to avoid damage to study participants.

Apart from the need to scrutinize the wish to change SOPs, sufficient time should be left between finalization and the date of coming into effect of an SOP. At least eight weeks is suggested.

Training

Training of all parties involved in the process is indisputably a key responsibility of sponsors. It derives from the requirement that sponsors must ensure that all involved parties are properly qualified.⁷ However, there are no specific requirements as regards the methods used to qualify staff. Inspectors tend to attribute non-compliance with process descriptions to inadequate training. Such a conclusion is too simplistic—it is important to analyze the underlying chain of circumstances as a means of initiating appropriate, reactive, or proactive measures.

Specifications hence need to be developed independently. It is important in this context to accommodate different learning types and availability of the insights of experts in the psychology of learning.⁸ A flexible learning system with a variety of learning modalities should be offered.

Both training providers and recipients recognize the responsibilities of employees in the learning process. Accordingly, self-study (read and sign, etc.) is very important and widespread. However, this learning method lacks feedback on learning outcomes, which could be the reason for the lower acceptance among users. Nor is there any means of checking learning intensity or identifying misunderstandings regarding content. However, self-study is nonetheless essential as a basis for learning.

In this survey, face-to-face/classroom training had the highest level of acceptance. The main reason seems to be the feedback on learning outcomes, which can be gained from following class discussion and asking questions about the content. The drawback of this learning modality is the comparatively high cost. The results of this survey indicate that the use of this method has become rather rare.

Online training, on the other hand, evidently seems to have become widespread. There are still acceptance problems among users nonetheless, in particular if the training provides no means of checking learning outcomes. Interestingly, online training programs with options to review learning outcomes have significantly higher acceptance rates than those without.

The survey results also clearly show that training measures should not be analyzed in isolation from their particular context. Complexity or novelty of processes and availability of complementary training modalities are key factors in constructing a flexible training system.

There seems to be high congruence between providers and recipients of training with regard to the amount of time required. On average, the actual time required from the training recipient's point of view exceeds the nominal amount by only two days (11 versus nine days). This is a good result. On average, the time required for training according to the results of this survey is about 5% of total working hours, with a very evident wide range of variability among the companies.

Analysis of frequency of training and refresher cycles indicates that "not forgetting" is the primary element addressed. A two- to three-year cycle may be appropriate, but is not based on hard evidence.

Conclusion

The starting point for this survey was the impression that employees are relatively dissatisfied with their SOP systems. A detailed survey—which would appear to be representative given the extent of feedback obtained—indicates that satisfaction may not be dependent on the size and structure of the SOP system. It does identify potential for improvement in this area of such importance for the quality of clinical trials.

Some suggestions can be derived from the above results, which will be closer looked at in a second article (Proposals for an Ideal SOP System):

• Responsibility, workflow, and personal accountability should be clearly defined in an SOP.

• SOP should address certain groups of users and contain only information relevant for those.

• SOPs should be user friendly, as short as possible, and as detailed as needed for clarity.

• SOPs should be oriented to a process and not to specific functions.

• SOPs should be easy to search and according to simple process maps.

• Training should be combined with adequate feedback and learning control.

Gabriele Schmidt,* PhD, is Associate Director, Clinical Research/Country Clinical Quality Manager, Germany at Sharp & Dohme GmbH, Lindenplatz 1, 85540 Haar, Germany, e-mail: [email protected]. Dieter Baier, PhD, is General Secretary of Medical and Scientific Excellence at Roche Pharma AG. Arthur Hecht is Head of QM-M&R, Germany at Boehringer Ingelheim Pharma GmbH & Co KG. Michael Herschel, MD, is Director Clinical Research at Glaxo Smith Kline Pharma GmbH & Co KG.

*To whom all correspondence should be addressed.

References

1. United States Environmental Protection Agency, "Guidance for Preparing Standard Operating Procedures (SOPs)," QA/G-6, (2007), http://www.epa.gov/quality/qs-docs/g6-final.pdf.

2. W. Bohaychuk and G. Ball, "Standard Operating Procedures for Clinical Research Personnel—Part 23," Qual Assur J, 9 (1) 40–49 (2005).

3. ICH GCP, guidelines, 5.1.1, http://ichgcp.net/.

4. A. B. Jones, "Principles in Quality Assurance," Qual Assur J, 6 (4) 219–225 (2002).

5. A. Blackmore, "The Myth of the SOP," Qual Assur J 8 (1) 3–12 (2004).

6. R. Hattemer-Apostel, "Standard Operating Procedures—A Novel Perspective," Qual Assur J, 5 (4) 207–219 (2001).

7. ICH GCP section, 5.1.1, 5.4.1, http://ichgcp.net/.

8. M. Spitzer, "Born to Learn," Journal for International Perspectives, 4 (2004).