A Workload Measurement

Related Information

1. EORTC Module 1: Planning Stage

2. EORTC Module 2: Recruitment, Treatment and Follow up Stage

3. EORTC Module 3: Trial Data Management

4. EORTC Module 4: Closure / Final Stage

Introduction and Background

It is increasingly evident to all parties involved in the sponsorship and management of clinical trials that a better understanding of the time, staff and financial resources required to conduct clinical trials is necessary. The increasing use and complexity of multi-modality treatment regimes, the rising costs of clinical trials, the emphasis on the efficient use of available resources and adherence to Good Clinical Practice (GCP), increasing regulatory requirements and demands for quality assurance/control, have resulted in an increased focus on workload issues. Failure to address these issues has resulted in research staff using unproven methods or a simple estimation for determining workload, leading, in some cases, to unrealistic expectations, excessive workload or inefficient use of resources.

The EORTC Clinical Research Coordinators Group recognised the importance of addressing this problem based on understanding the tasks involved, the time requirements and the resources necessary to effectively and efficiently conduct clinical research. The development of a workload measurement instrument (WMI) was seen as a means of providing a tool by which individuals could estimate more accurately the time and resources required to participate in clinical trials. The initial project was supported by the EORTC and funded by a grant from the “Fonds Cancer” (FOCA).

The WMI was developed in seven stages and the first stages: development, validation and revision of a draft checklist of trial related activities, drafting of the WMI, a feasibility pilot study, and analysis and revision of the trial related tasks incorporated into the four modules previously outlined (Berridge and Coffey, 2008). Coincidental to the development of the WMI Jacqueline Briggs had developed a complexity tool for assessing the complexity of a study prospectively (Briggs 2008). It was agreed that, in the longer term, a joint workload and complexity project would aim to independently validate and link together the WMI and the complexity tools.

This paper describes the seventh stage of development of the WMI; a prospective study using the WMI to collect data for the purpose of validating the four WMI modules.

Materials and Methods

The preparatory phase identified the specific actions necessary and the financial and staff resources required to complete the prospective study. The Steering Committee members presented the prospective study outline to a meeting of National Cancer Research Network Managers and identified the networks that would be willing to collaborate. Expression of interest questionnaires were sent out to all UK network managers and to research staff in several European centres to identify research staff and potential trials for inclusion in the prospective study. Fifteen UK networks and a centre in Ireland initially agreed to participate in the study and of these twelve networks returned completed modules as part of the prospective study.

The prospective study focused only on research related activity and not on what would be considered routine care. Workload was only recorded for research staff working on the trials and excluded investigators, pharmacists and day care staff etc. These staff members were excluded as it was felt that it would be difficult to collect accurate workload data from such a wide range of staff and this was accepted as a limitation for the purposes of this study.

As the joint project had been extended to include complexity the four WMI modules (module 1: planning stage, module 2: implementation stage, module 3: trial data management stage and module 4: closure/final stage) were reviewed. In order to facilitate the validation of the complexity tool and create the necessary links between the two tools, it was agreed to divide module 2 into two modules: 2a recruitment and treatment and 2b treatment and follow-up.

A data collection period of six months was agreed and the time frame for completion of data collection was October 2005 to May 2006. Prior to the commencement of the data collection period three training days were held in Newcastle, London, Nottingham and Wales to roll out the study to the interested participants. 43 research staff attended these training days. Information was provided individually for staff unable to attend the training days.

From the training days and returned questionnaires a number of trials were selected on which the prospective study would focus. For modules 1 and 4 it was agreed that any trial in set up or at the closure stage could be included due to the small number of trials likely to be at these phases during the six month time frame. For modules 2 and 3 trials covering as many tumour types, treatment modalities and study types were selected to ensure as inclusive a range of activities as possible. A total of 36 studies were identified and included in prospective study.

Guidelines for the completion of each module were developed and circulated, together with the four modules and the list of identified trials, to 70 collaborators, covering 15 networks that had agreed to participate. Collaborators were asked to record the time taken, in minutes, for each individual task or subtask undertaken as identified within the modules. A subsection – other- was included for recording any additional activities not identified within the modules. A separate section recorded the hospital name, trial name and number, the date for data collection commencement and completion, the reason why data collection was ended and whether more than one form had been completed. This allowed us to identify the total period of data collection and subsequent analysis of workload for a single trial visit or over the course of a trial period.

Data was entered into a database developed in conjunction with the NCRN IT department based on all the identified tasks and subtasks of the WMI modules. The NCRN agreed to provide staff to complete the data input and carry out some preliminary data analysis and a system for quality assurance of the data was agreed.

The Steering Group met regularly to review the progress of the prospective study and as returned modules were received, to carry out the initial quality assurance procedures. All returned modules were double-checked to ensure accuracy and where tasks or subtasks were entered in an incorrect section these were noted and revised prior to entry onto the database. Incorrect or unclear entries were deleted. Regular contact and feedback was maintained with the collaborators throughout the study by email, newsletters including answers to frequently asked questions and presentations at Cancer Research Network Managers meetings.

Results (including limitations)

A total of 414 completed modules were returned from collaborators from 27 hospitals covering 12 UK Cancer Research Networks. Three networks or centres did not return completed modules due to merger of networks, shortage of staff or excessive workload issues. Data was returned on 35 of the 36 studies identified for inclusion in the prospective study. Table 1 shows the modules returned.

As part of the analysis process a series of assumptions were made. These included: research staff were experienced and trained and not involved in the actual delivery of treatment, that research activity was additional to standard treatment, hospital support services and standard equipment were available and accessible to research staff. Cumulatively 130 changes were made 74 subtasks (Table 2). These changes included transfer from one subtask to another where an entry was incorrect, deleting investigators and separating out multiple patient visits into individual visits. The highest numbers of recording errors (9, 15 and 16) necessitating transfer related to the recording of administration activities in other subtasks rather than administration activities.

Following extensive quality assurance procedures all tasks and subtasks within modules 1, 2 and 3 were validated as relevant and appropriate with no tasks or subtasks removed. All subtasks were recorded at least once with a very wide range within each module. For Module 1 the most frequently completed subtasks were those relating to the preparation and submission of documentation to Ethics and Research and Development departments. Conversely the lowest recorded tasks related to organising and attending meetings other than in house meetings for information, training and trial activation.

For Module 2a the most frequently completed subtasks related to screening for eligibility, informed consent and sample preparation. This was consistent with the expectations and personal experience of the project group. The least completed subtasks in Module 2a related to liaising and verifying radiotherapy dose reductions and missed treatment which was anticipated given the very low numbers of radiotherapy related trials included in the study.

For Module 2b and 3 the most frequently completed subtasks related to case report form completion, photocopying and sending trial documents and the preparation and submission of trial amendments and updating study documents following amendments (Table 3)

Two major revisions were identified as necessary. The first change related to the recognition of the importance given to administration and communication by the collaborators. The section ‘other’ consistently included additional administration and communication and it was agreed therefore that these two activities should be included as subtasks for all main tasks in the final modules. In total 110 transfers were made to administration main task and 230 to administration subtask activities and 38 to communications main task with 100 to communication subtask activities. The second change was the recombination of module 2a and 2b with specific sections relating to consent, treatment and follow-up incorporated given the level of confusion that had arisen in completion of these as two separate modules.

Conclusions / Recommendations

Modules 1, 2 and 3 of the workload measurement instrument have been validated and are now complete. Module 4 has been revised in accordance but would benefit from validation as part of a specific prospective study as trials reach closure stage.

It is acknowledged that there were limitations to the prospective study that included the short data collection window (six months), the exclusion of Principal Investigators and other associated staff from the data collection exercise compounded by the difficulty associated with specific local hospital issues all had potential to confound the data collected. It was accepted that these factors would contribute to workload but would not necessarily alter the tasks and subtasks identified for inclusion in the modules and therefore such confounders were not considered significant for the purpose of the WMI validation. For hospitals or departments undertaking a workload analysis in the future it would be appropriate to take these confounders into account.

This WMI was developed to be a generic tool and can be used locally, nationally or internationally to review the trial workload of an individual researcher, team, hospital, or network of centres. It should inform discussions at all these levels and allow centres considering implementing new or additional clinical trials to accurately evaluate the associated workload and to allocate staff and resources accordingly.

The WMI is available from the authors.

References

1. Briggs J (2008) Real World Workload Needs – Developing a process and management tool for scoring complexity in cancer clinical trials. Applied Clinical Trials Oncology & Clinical Trials in the 21st Century supplement; 5 22-24

2. Berridge, J & Coffey, M (2008) Workload Measurement. Applied Clinical Trials, 6 98-101

Acknowledgements

We would like to acknowledge the contribution and support given by the EORTC, FOCA and the NCRN and the invaluable input of Leigh Oates and Jodi Baxter. In addition to the following participating Cancer Research Networks: Cancer Care Alliance, Lancashire and South Cumbria, Mid Trent, Mount Vernon, Norfolk and Waveney, Northern, North London, North East London, North West Midlands, Scotland South East, South West London, South Essex, Wales North, Wales South East.