Utilizing DHTs for Clinical Trial Endpoints

Digital health technologies (DHTs) is an emerging term for technologies that use computing platforms, connectivity, software, and sensors for health care and related uses. In clinical drug development, there are some familiar uses, such as connected blood glucose meters in diabetes studies. However, there’s been limited uptake at scale outside certain therapeutic areas.

Taking a look back at the historical development of DHTs, Holter monitoring achieved the first radio-wave broadcast of an ECG signal in 1947. The measurement and transmission equipment was housed in a 38kg backpack–ironically perhaps enough to provoke a heart condition! Since then, the miniaturization of sensors and circuitry has enabled the development of wearable cardiac monitoring devices that can be unobtrusively worn for many hours, something we now see used in both clinical trials and routine care. Accelerometers that track physical activity (PA) and sleep have enabled passive monitoring with affordable and convenient devices since the early 2000’s,¹ and in the last decade, the devices have become popular components of personal health tracking bands and watches.

Today, we see growing regulatory interest in the use of DHTs for the development of novel clinical endpoints in drug development, apparent in FDA’s development of patient-focused drug development guidances.² Several novel, sensor-based endpoints have been accepted into the FDA’s Clinical Outcome Assessment Qualification program, including accelerometer-based measures for chronic heart failure (CHF), multiple sclerosis (MS), pain, and sarcopenia. In Europe, the EMA qualified the PROActive composite measure, which combines patient-reported outcomes data with activity monitor data for an endpoint in chronic obstructive pulmonary disorder (COPD) trials.³

The interest in DHT usage is compelling. Not only does the technology enable more frequent measurement, but in some cases, measurements may be more accurate or convenient to obtain. Further, DHTs may enable measurement of constructs that have not been possible to measure before, and may enhance the oversight of patients between physical clinic visits. Despite this, our industry is slow to scale up the use of DHTs, perhaps due to the challenges and knowledge gaps below:

• Does a DHT enable us to measure something meaningful?
• Do we understand the evidence needed to support the selection of a specific
  DHT device?
• Do we have good implementation and data management standards?
• Do we understand the relationship between endpoint properties and the specific
  DHT device?

Meaningful endpoints

Gregory House MD, the fictional physician in the eponymous TV drama, had the mantra to never talk to the patient for insight into their condition. We’ve advanced from this approach FDA’s patient-focused drug development guidances establish the need to develop clinical outcome assessments that can be shown to effectively measure constructs that are meaningful to the patient.² This applies to endpoints derived from DHT data.

DHTs may offer a suitable approach to measuring a meaningful aspect of health that is related to a study’s concepts of interest. However, careful consideration is needed to determine how to derive meaningful outcome measures from DHT data. For example, while we may rightly consider PA as an important concept to measure for many disease indications, is total steps per day or the time spent in moderate to vigorous PA (MVPA) always the most meaningful measure? A COPD or CHF patient, for example, may find the ability to conduct short bouts of purposeful walking at a steady, continuous pace the most meaningful. In this case, a measure of bouts of walking for a defined duration and pace may represent the most meaningful measure to derive from daily activity profiles.

In their work with the DIA Study Endpoints Community, Walton et al. described a useful framework for the development of meaningful endpoints.⁴ In essence, the process requires insights into the aspects that are most meaningful to patients–likely established through qualitative research in the patient group–to identify meaningful aspects of health from which to define concepts of interest for health measurements. By understanding these valued aspects, pertinent endpoints can be defined, and an appropriate measurement approach can be identified–which may include a DHT. See Figure 1 below as an example.

Evidence needed to support DHT device selection

Perhaps one of the main barriers to scaling up of use of DHTs in clinical trials is the underlying concern as to whether the data will be accepted by a regulator. However, several cross-industry consortia have made valuable contributions to define “fit-for-purpose” DHTs, and the evidence needed to support their use for clinical endpoints and regulatory decision making. Contributions were made by the Critical Path Institute’s ePRO Consortium,⁵ the Clinical Trials Transformation Initiative (CTTI),⁶ the DIA Study Endpoints Community,⁴ and the Digital Medicine Society.⁷ Using these, we now know enough about how to select a fit-for-purpose device and the associated supportive evidence for this concern to no longer be an issue.

Good data management and implementation standards

The literature doesn’t always provide a robust consensus on the implementation and data management standards for DHT use. In my review of clinical research studies that used an accelerometer to study PA in COPD, I found little consistency in implementation approaches, including placement location, duration of wear time, and endpoint studied.⁸ For example, the 76 studies evaluated reported 80 different study endpoints derived from accelerometer data. Despite this, we can adopt an endpoint-driven approach to define implementation and data management approaches.

Using the example of PA measurement, the endpoint of interest drives implementation considerations. For example, placement location may be driven both by patient acceptability and optimization for measurement. The hip is close to the center of mass and is often the best location for energy expenditure measurements, at least in the lab setting. The wrist may be chosen for better wear compliance, but may overestimate PA or be insufficient at distinguishing light PA. The thigh is optimal for studying sedentary behavior due to the specificity in distinguishing sitting from standing, but only certain devices are suitable for this placement location (e.g., ActivPAL (PAL Technologies, Glasgow, UK)). Wear time may be similarly endpoint-determined. If studying total steps/counts per day or total MVPA time, wearing the device for the majority of the day will be important as well as measuring across a reasonable number of days to obtain a good overall PA estimate. However, if real-world walking speed or cadence is the endpoint of interest, these measures are typically quite consistent during purposeful episodes of walking across the day, so wear time will be important to capture a number of walking bouts and may not require all-day use.

Data collected that does not meet minimum wear time considerations are typically discarded, although work is underway through the DIA’s Study Endpoint Community to explore other approaches in handling missing data amongst the rich, time-series data collected by some DHTs.

Endpoint properties and different devices

An important question to endpoint developers using DHTs as a measurement instrument is whether endpoints are device-dependent or device-agnostic. This information will allow us to answer important questions such as:

• Can I interchange DHT devices within a study or across a development program?
• Can I compare the results from different development programs measuring the same endpoint with different DHTs?
• Can I adopt a bring-your-own-device (BYOD) approach to DHTs?

This is a common question in the field of patient-reported outcomes, and we understand the importance of ensuring the measurement properties of validated patient-reported outcome measures are conserved when implemented in different formats (e.g., paper, smartphone, tablet).

At this time, we do not believe DHTs provide enough equivalence evidence to support device-agnostic endpoints for some key measures. For example, Bender et al.⁹ studied concordance of PA measures provided by Fitbit, Garmin, and Apple devices in healthy volunteers within free-living conditions for 14 days. Their evaluation concluded that step count, distance traveled, and calories burned could vary significantly between devices used concurrently–the total step count differing by up to 30%. Measurement differences occur even with research-grade devices. For example, in the development of the PROActive measure, a monitor-specific data conversion was needed to derive the endpoint consistently using either of the two selected activity meter models.³ However, such differences in processed daily summary and epoch data may be mitigated by applying a common algorithm to raw accelerations data collected by different devices when access to this raw signal data is available. Differences between devices have also been observed in other areas such as optical heart rate sensors.10

Endpoint developers should consider the degree of tolerance between acceptable devices to maintain a reliable endpoint when different measurement devices are used. If this cannot be achieved, a sensible approach would be to standardize use of a particular DHT device within a development program, or select a device manufacturer with backward compatibility in the measurements provided.

Conclusions

DHTs offer great potential to measure constructs more frequently, more accurately or conveniently, or to measure things we have been unable to measure before. These technologies enable greater oversight and monitoring of patients between site visits and within decentralized trial models. We should apply DHTs when they offer an appropriate approach to measure a clinical endpoint related to a study concept of interest. As we have explored, endpoints derived from the DHT-collected data should measure aspects of health that are meaningful to the patient.

We know enough about the evidence needed to support device selection and endpoint validation requirements to scale up the use of DHTs in clinical trials. While consistency between makes and models still needs to be considered, researchers should retain a single device if demonstrating measurement equivalence across devices is a concern.

The drive towards greater use of remote assessments will see more work emerging in defining and validating clinical endpoints derived from DHT data. We can expect to see continued interest and an increase in the use of DHTs in drug development programs to support regulatory decision making.

References

1. Troiano RP, McClain JJ, Brychta RJ, Chen KY (2014) Evolution of accelerometer methods for physical activity research. Br J Sports Med 48:1019–1023. https://doi.org/10.1136/bjsports-2014-093546.Evolution
2. FDA (2017) FDA Patient-Focused Drug Development Guidance Series for Enhancing the Incorporation of the Patient’s Voice in Medical Product Development and Regulatory Decision Making. https://www.fda.gov/drugs/development-approval-process-drugs/fda-patient-focused-drug-development-guidance-series-enhancing-incorporation-patients-voice-medical. Accessed 30 Mar 2021
3. Gimeno-Santos E, Raste Y, Demeyer H, et al (2015) The PROactive instruments to measure physical activity in patients with chronic obstructive pulmonary disease. Eur Respir J 46:988–1000. https://doi.org/10.1183/09031936.00183014
4. Walton MK, Cappelleri JC, Byrom B, et al (2020) Considerations for development of an evidence dossier to support the use of mobile sensor technology for clinical outcome assessments in clinical trials. Contemp Clin Trials 91:105962. https://doi.org/10.1016/j.cct.2020.105962
5. Byrom B, Watson C, Doll H, et al (2018) Selection of and Evidentiary Considerations for Wearable Devices and Their Measurements for Use in Regulatory Decision Making: Recommendations from the ePRO Consortium. Value Heal 21:631–639. https://doi.org/10.1016/j.jval.2017.09.012
6. Clinical Trials Transformation Initiative (CTTI) (2017) Recommendations on developing technology derived novel endpoints. https://www.ctti-clinicaltrials.org/projects/digital-health-technologies. Accessed 30 Mar 2021
7. Society DM (2020) The Playbook: Digital Clinical Measures. https://playbook.dimesociety.org/. Accessed 30 Mar 2021
8. Byrom B, Rowe DA (2016) Measuring free-living physical activity in COPD patients: Deriving methodology standards for clinical trials through a review of research studies. Contemp Clin Trials 47:172–184. https://doi.org/10.1016/j.cct.2016.01.006
9. Bender CG, Hoffstot JC, Combs BT, et al (2017) Measuring of fitness trackers. 2017 IEEE Sensors Appl Symp 1–6. https://doi.org/10.1109/SAS.2017.7894077
10. Bent B, Goldstein BA, Kibbe WA, Dunn JP (2020) Investigating sources of inaccuracy in wearable optical heart rate sensors. npj Digit Med 3:1–9. https://doi.org/10.1038/s41746-020-0226-6

Bill Byrom, PhD, VP, Product Intelligence and Positioning, Signant Health