5 Common Mistakes With Spirometry in Clinical Trials

Spirometry data is pivotal to assessing primary or secondary outcomes in most respiratory trials, but the methodology for data acquisition and data collection is rarely published. In many cases, a significant portion of spirometry data is of inadequate or questionable quality. These quality issues lead to increased data variability, undermine the validity of the results of a trial and require increasing the number of subjects in a trial.

The following mistakes frequently occur in trials using spirometry; active plans to mitigate against these risks can improve your study’s data quality. Central standardization across all testing and data review processes, from end to end, is the best way to ensure that conditions for each patient, test, QC and review are identical.

Mistake One: Lack of Centralized Training

Training is a critical component in driving consistency across and within sites and to ensure that test results are repeatable. Without centralized training, sites will default to administering tests according to their specific guidelines and not internationally recognized ones such as American Thoracic Society/European Respiratory Society criteria. Inconsistent device calibration and methods of coaching lead to non-reproducible test results. Teaching sites reliable methods for conducting spirometry tests gives each technician, across patients and sites, the ability to produce consistent results, ensuring patients are treated appropriately according to their disease pathology.

Coaching is essential to obtain the best effort from the patient. Without proper coaching, a patient may not perform the test with maximal effort thus resulting in poor representation of data. Obtaining accurate results may be extremely difficult with certain patients who suffer from severe diseases, but with a properly calibrated device and a well-trained technician, inaccurate results can be minimized through personalized coaching techniques.

Finally, it is critical to ensure that clinical monitors are carefully trained on the basics of the spirometry testing in a clinical trial, as well-trained monitors are excellent tools for reducing site burden and improving overall data quality.

Sites are responsible for:

• Identifying testing errors and rectifying them

• Performing assessments of repeatability

• Identifying the two highest volume loops for each result

• Performing calibration checks and verification of the spirometer log

Competency should continuously be monitored throughout the lifecycle of the trial and retraining of site staff should be offered when deficiencies are identified or when there are long delays between subject enrollments.

Mistake Two: Lack of Site Standardization

The spirometry core lab should conduct a proactive assessment of the site and the technician’s experience and competency with spirometry testing. This assessment should evaluate:

• Length of experience performing spirometry testing

• Relevant education and certification

• Training history

• Clinical trial experience

• Device and software experience

• General familiarity with ATS/ERS Guidelines

Identifying this information in advance will allow the core lab to identify strengths and weaknesses of the site, in advance of on-study patient testing, and the training and startup activities can be tailored specifically to that site’s needs.

A site’s competency for performing high-quality spirometry testing should be evaluated through the collection of volunteer tests, or a test transfer, whenever possible. A series of tests on a healthy volunteer (likely a co-worker at the facility) can be collected and submitted to the core lab for review.  This process allows for the core lab to ensure that the site is properly adhering to ATS/ERS Guidelines and that there is a sufficient understanding of the device and associated software by the user. Any deficiencies in the test transfer should be reported back to the site and, depending on severity, may require retraining or corrective action prior to performing on-study patient testing.

Mistake Three: Inconsistent Site and Device Management 

Sites will default to their standard device management, which can be incorrect and certainly inconsistent when evaluated across sites. According to a recent study published in the National Institute of Health, it is essential to “Ensure that a site is using spirometry on a regular basis, has a trained spirometry technician, is well-versed with ATS/ERS 2010 requirements, understands the need for calibration and performs and interprets acceptable, reproducible graphs. A high level of oversight is needed during the trial to ensure good quality spirometry reports. Retraining of site staff is the key to avoid major spirometry quality issues.” Software should be configured carefully to provide clear user instruction; ATS/ERS Guidelines prompts error messages that are easily understood by technicians. Basic reference materials should be provided to the sites that include simple step-by-step instructions for setup and use, representative flow-volume loop examples, error message explanations, criteria guidelines and trouble-shooting tips. These tips should include instructions for coaching the effort and liaising with sites’ IT departments in advance of patient visits to ensure that data transfers are able to occur securely and efficiently prior to any patients’ participation in the study. 

Mistake Four: ATS/ERS Guideline Noncompliance

An important consideration when reviewing spirometry data is the ability to detect quality concerns early and to provide meaningful feedback to the sites quickly, so that corrective actions can be taken.

When spirometry test data is reviewed in isolation, it is difficult to make the distinction between quality issues due to a patient's inability to follow instructions, and quality issues related to a lack of proficiency on the part of the technician. Therefore, grouped quality review of multiple tests coming from a single site should be implemented to help minimize risk. These reviews should be programmed at predefined study visits (e.g. screening visit for the initial patients enrolled in a trial at a site). When performing a quality check of a patient’s spirometry test data, the core lab should always ensure that the following conditions are reviewed for compliance and accuracy:

• Acceptability of the measure: criteria to check the acceptability of the flow volume loop.

• Reproducibility in at least two out of three acceptable maneuvers (the two highest volumes are used as long as they are less than a150 ml difference).

• Appropriate and accurate demographics.

• Flow volume loops are of high quality and demonstrate maximal effort.

Performance of calibration in accordance with ATS/ERS recommendations is also critical in ensuring that the spirometer at the site is accurate and functioning properly. The core lab should monitor and collect this data regularly, and should work with the site to ensure the following:

• Equipment is being calibrated with a 3-liter syringe daily prior to any patient testing.

• The syringe must be certified to be accurate by the manufacturer; this test is typically performed annually unless otherwise indicated by the manufacturer.

Mistake Five: Biased Best Test Selection

A site technologist may be privy to other information available at the site which can lead to bias, however a central spirometry reviewer is blinded to the testing circumstances that occur during the spirometry data collection when evaluating efforts. Centralized services enable blinded reviews to be conducted independently from sponsors.

Independent spirometry reviewers are trained on interpretation and quality assessment. The core lab should monitor the consistency of both quality review and spirometry interpretation between reviewers at the start of the trial and whenever a new reviewer joins the team. Discrepancies and variability in the way that spirometry tests are assessed between reviewers should be discussed to establish an ongoing and improving level of consistency.

The primary objective when conducting a central review is to establish precise rules for both the computerized analysis and the central reader’s assessment. Clearly defined rules, that are applied to all data, in the same way, result in better data quality overall. Feedback on the central assessment results should be shared with the sites in order to advance their understanding of central review criteria, which will help to improve the data being collected by the site one patient at a time. It is critical that sites are informed whether the test was acceptable or not after each assessment is made.

Finally, data monitoring is the foundation supporting the prior five steps. The results of data monitoring are presented to the core lab’s medical team to evaluate the evolution of key data points across time, sites and patients. The analysis allows detection of data variance and negative performance trends which may result in queries, corrective actions and retraining for the site. For example, the core lab should have the ability to easily monitor the Forced Expiratory Volume 1 (FEV1) data within a patient for any values inconsistent with what might be expected. FEV1 values can also be monitored across the site to look for consistent poor performance, a clear reflection of inadequate coaching techniques by the site technician. Similarly, trending and monitoring of calibration results, as outlined in mistake four, should be evaluated as part of overall data monitoring to ensure ATS/ERS compliance.

Data quality can be compromised if there is a shift from standard practices. Data inconsistencies can be quickly identified and remedied by leveraging data monitoring processes. Identifying issues early, before they become systemic, is critical in ensuring overall data quality.

Spirometry testing provides important and unique data on the functioning of the respiratory system and there are no surrogate markers for these values. Spirometry tests are complex and it has been a challenge to reach an acceptable level of quality in the context of a drug trial. Starting a study without a detailed action plan to contain, control and prevent quality issues related to spirometry tests are considered a thing of the past. Neglecting quality control and not using qualified pulmonary lab technicians for spirometry testing jeopardizes the validity of study results and may impact subject safety. Data reviewers are unable to effectively identify important trends in lung function when the spirometry data quality is substandard. The most critical aspect to ensuring quality spirometry data begins and ends with supporting sites and easing their burden. Additionally, engagement between clinical monitors and the spirometry core lab staff is key to supporting sites. The goal for all parties is to improve data quality and drive endpoints by creating simple processes and requirements that a site can easily follow; ultimately helping sponsors, sites and most importantly, patients, breathe easier.

Bertrand Sohier, MD, Senior Medical Director, Global Head Pulmonology & Cardiovascular Therapeutic Areas Eric Trueblood, MD, Associate Medical Director Lisa Pifalo, RRT, Senior Scientist Rohit Sood, PhD, MD, Senior Medical Director   all with PAREXEL