American physicist and mathematician Freeman Dyson said that the year 2000 was essentially the point at which it became cheaper to collect information than to understand it. This observation, made almost 20 years ago, still rings true today as we consider the growing number of devices that we interact with on a daily basis and that collect all kinds of digital data.

This is particularly the case with our smartphones and the sensors contained in many everyday appliances that connect and deliver data through the internet of things (IoT). Modern smartphones contain sensors that were originally in place to enable certain handset functionality, but the data they generate are now being leveraged in other novel ways to add value to the user. For example, most of them contain an accelerometer sensor. This is used to understand the 3D spatial positioning of the device and to detect when the device is rotated to enable the screen display to switch between portrait and landscape modes. However, this is the same sensor used in many activity monitoring devices, such as Fitbit or Garmin wearable health trackers-and the same 3D accelerations generated and stored by the sensor to determine screen orientation can also be translated into activity parameters such as the number of steps taken by the user while carrying their smartphone. Most devices now contain health and wellness apps to exploit this capability and provide additional value to the user.

Bill Byrom

This interpretation of existing data for new purposes is an exciting area of innovation that we are seeing increasingly in the area of personal health and wellness, and it has huge potential to transform the way in which we capture measurements from patients in clinical trials. Simply put, technology like this is enabling us to provide richer insights and potentially measure new meaningful constructs that we have been unable to assess robustly in the past.

Perhaps most importantly, technology gives us the ability to think originally. The ways in which we are able to leverage existing technologies developed for other purposes, in new and novel ways, to collect insightful health status data from patients in clinical trials is an exciting area of current innovation. At the 2018 Drug Information Association (DIA) Annual Meeting in Boston, there were a number of presentations exploring this precise topic, which generated meaningful and enthusiastic conversation throughout the meeting. Ahead, I provide a brief review of the session that I chaired, entitled “Future of Endpoints,” which discussed three diverse examples at different stages of maturity in terms of their potential application within clinical trials. I further discuss future directions for these approaches, and the kinds of activities needed to enable their ultimate use to support pharmaceutical and regulatory decision-making.

The aim of using technology in clinical trials is to simplify processes, make participation easier, improve quality, facilitate decision-making, and collect reliable, honest data. When collecting health outcomes, it is important to employ approaches that enable the optimal assessment of the study concepts of interest. In some cases, this may involve the use of a technology solution.

Three approaches that were presented in the DIA session are considered in this article. The first, presented by Alejandro Zamorano (PainQx) explored the use of modern brain-sensor headbands to measure electroencephalogram (EEG) signals and develop objective measures of pain. The second, presented by Christian Gossens (Roche) examined the development of new health outcome measures in Alzheimer’s disease using smartphone sensors. The third, presented by myself, explored the use of motion-based gaming technology platforms to develop new objective measures of movement and mobility.

Each approach shows promise in leveraging existing technology solutions in novel ways to deliver health outcomes measures that either provide a richer picture of health status due to the ability to measure remotely, or provide a potentially superior approach to development of sensitive, objective measures compared to current practice.

Use cases

Use Case 1: Leveraging wearable sensors to measure pain

Wearable devices that measure EEG brain activity have been used to enable interaction with gaming systems, develop applications to facilitate activity and communication in impaired patients, and to provide brain training applications in personal health and wellness.¹ Examples of the latter two include the “Mind Speller” application that enables textual and verbal communication using EEG brain signals from patients with reduced motor functioning;² and brain training applications to assist the management of anxiety and concentration by providing insight into types of brain activity using neurofeedback.¹

Portable EEG headbands provide a means to collect this data remotely or without specialist equipment during clinic visits. These are typically worn on the forehead and collect signals using a series of dry electrodes to generate a continuous EEG trace, although some discrete cochlear devices are in development.³ Examples include MUSE (InteraXon Inc., Toronto, Canada), Emotiv EPOC (Emotiv Inc.) and ZenZone (NeuroSky Inc.).

While we discuss later in this article the additional work needed to ensure the reliability, accuracy, and precision of data collected in this way, if the potential use in clinical trials is to be realized, PainQx have conducted significant work on the validation of outcome measures derived from EEG signal data to provide objective measures of pain. In his presentation, Zamorano provided an insightful review of their scientific work to date.⁴

Foundational to this work is the property that chronic pain appears to be associated with increased alpha and theta EEG signals during spontaneous EEG recording, and low amplitudes of event-related potential (ERP) when the patient is presented with various stimuli.⁵ PainQx have developed algorithms to interpret EEG traces to describe the patient’s pain state by mapping quantitative measures of electrical activity in different regions of the brain responsible for the sensation and perception of pain. By filtering out components not related to pain sensation or perception, this “Pain Matrix” provides an objective outcome measure to describe pain incidence and severity. Pertinent areas of EEG activity are isolated, identified, correlated, and weighted to produce an objective score describing the patient’s pain state. This approach has been seen to correlate well with subjective measures of pain and to distinguish between high and low pain in chronic pain conditions.¹

While self-perception of pain nature and severity is a critical element to assess pharmaceutical intervention effects, generally recorded using patient-reported outcome measures (PROMs), this objective measure derived from brain activity monitoring may be useful alongside these traditional PROMs. In particular, in addition to providing additional supportive data to PROM endpoints, EEG-derived outcome measures may provide additional supporting data, may enhance study qualification/screening activity, and may provide a convenient mechanism to evaluate the real-time effects and dose optimization of analgesic and narcotic drugs during treatment.

Measurement using portable EEG headsets opens the door to remote measurement, and convenient measurement in clinic. However, their use relies upon satisfactory reliability, accuracy, and precision of data collected in this way. Some factors for consideration include the reduced number of electrodes, the fact that electrodes connect to the skin in a dry state, that measurements using headbands predominantly represent activity from the frontal cortex, and that device firmware must be relied upon to adequately filter and interpret the signals received. Some of this data is becoming available for appraisal in the scientific literature, and some additional work is needed to assess the scientific acceptability of the approach.

Use Case 2: Leveraging smartphone sensors to enable frequent outcome assessment in remote settings

As described above, the sensors within smartphone handsets are already being used in the wellness industry to provide health and fitness applications. Smartphones are already used in clinical trials to collect electronic patient-reported outcomes (ePRO) data, and leveraging their sensors to collect other data through active performance tests is a novel approach to accumulating additional objective data remotely and conveniently. Christian Gossens, PhD, global head of digital biomarkers at Roche, also presented in the “Future of Endpoints” session and described new work underway in the development and validation of performance outcomes (PerfOs) aimed at studying multiple sclerosis (MS) patients and conducted by leveraging smartphone components and sensors. This work is presented within the Floodlight Open study, currently recruiting online.⁶

The study aims to measure a participant’s ability to perform simple tasks using their smartphone with the aim of understanding the effects of MS on cognition, dexterity, and mobility. For example, the assessment of pinching action between thumb and finger is commonly assessed subjectively using clinician-reported outcomes such as within the Unified Parkinson’s Disease Rating Scale (UPDRS). This assessment measures aspects of dexterity, muscle weakness, and control. The Floodlight app has gamified this test and presented it as a task where subjects use the same pinching action on the touchscreen to “squash” tomatoes between thumb and finger as they appear on screen. In addition, a drawing test where users are requested to draw along the outline of a figure of eight shapes is included to measure other aspects of dexterity, hand-eye coordination, and muscle control.

In addition to enabling objective measures of constructs that have previously been measured subjectively by the clinician, one key advantage of this approach is the ability to study health outcomes more frequently than can be achieved through regular clinic appointments. This has been illustrated previously by Gossens and colleagues in their work on smartphone-delivered tests in Parkinson’s disease (PD). Detecting tremor, for example, using a simple test where the smartphone is balanced on the palm of the hand for 30 seconds and tremor-related movements are detected using the accelerometer sensor has already shown promise in the understanding of tremor symptoms in PD.⁷ This may significantly improve understanding of treatment effects, especially for symptoms that present intermittently or may suffer from poor recall properties.

Use Case 3: Use of motion-based gaming platforms to measure movement/mobility outcomes

Motion-based gaming platforms use depth-cameras to detect body movements and enable users to interact with gaming applications in more immersive ways. The same depth-camera technology, and its associated software development kits (SDKs), can be used to develop custom software with application in education and health. The most commonly used solution is the Microsoft Kinect depth-camera associated with the Xbox gaming system, although other more advanced (yet similarly low-cost) technologies exist, such as the Intel RealSense camera range.⁸ There are numerous applications utilizing this motion capture technology to study or encourage movement in healthcare, particularly in rehabilitation. Being able to track the 3D position and movement of body joints enables the assessment of movement, and the detection of correct exercising during rehabilitation. Jintronix, for example, have developed games using Microsoft Kinect to encourage adherence and engagement with rehabilitation regimens, which have shown good outcomes in terms of reduced readmission rates in orthopedic and stroke patients.⁹ Similarly, being able to track facial landmarks enables the deployment of other health applications, such as rehabilitation systems for patients recovering from facial paralysis-for example, with Bell’s palsy and stroke.¹⁰

Depth-camera solutions offer the potential to make objective in-clinic measurements that may previously only have been possible in more specialized motion laboratory settings or by using subjective clinician-reported outcomes (ClinROs). Simple range of motion, gait, and balance performance tests have been developed that leverage simple depth camera technology, both within and outside the context of a video game, some of which have shown reasonable performance in early validation studies.¹¹

For example, converting the 3D coordinates of body joints into vectors representing the spatial orientation of parts of the body enables simple vector algebra to calculate the angles made between joints and thus provides an estimate of the range of joint motion (see Figure 1). Early validation work compared to goniometer measurements has shown promise for upper extremity range of motion measures for example.^12,13

The use of motion-based gaming technology to develop movement-based outcome measures may enable the low-cost measurement of outcomes not possible outside specialist movement laboratories and may provide advantages over subjective ClinROs in providing measures that may be more sensitive, less prone to inter-rater variability, and capable of measuring more subtle aspects of movement and motion.

Developing endpoints derived from novel use of technology applications

The ability to leverage endpoints derived from these novel approaches, and other approaches leveraging existing technologies in novel ways, relies upon the provision of evidence to support the use of the technology and to support the endpoint derived. Specifically, we must be assured that the device faithfully measures what is intended to an acceptable level of reliability, accuracy, and precision; and that endpoints derived are truly measuring a concept of interest of the study, are sensitive to detect changes in health status as a result of an intervention, and that meaningful change is understood. This is, of course, no different to the approach required to validate any measurement approach associated with any clinical endpoint used to measure intervention effects.

A comprehensive summary of requirements was published by the Critical Path Institute’s ePRO consortium in the context of the use of wearables to develop endpoints to support regulatory decision-making and labelling claims.¹⁴  These are summarized in Figure 2, and also below.

A. Technology assessment

Usability and feasibility: Demonstration that the technology is usable within the target population and feasible within the context of the specific clinical trial.

Reliability: Data generated show satisfactory intra- and inter-device reliability.

Concurrent validity: Demonstration that the technology is truly measuring what is intended.

Responsiveness: Data generated are able to suitably distinguish changes when they occur.

B. Endpoint evaluation

Measures a concept of interest, as defined by the study protocol.

Content and construct validity: The endpoint provides a sufficiently comprehensive measure of a concept of interest that is meaningful to patients and/or the treatment of their condition; and faithfully measures the construct intended.

Ability to detect change: Sensitive enough to detect change when a change exists.

Endpoint interpretability: The change in the endpoint deemed meaningful to patients is understood (e.g., minimally clinically important difference [MCID] or individual responder definition).

Conclusions

There is huge potential for thinking differently about how existing technologies can be repurposed to enable novel measurements for health outcomes and health status in patients. The increased insights obtained through more frequent home-based measurement, and new objective outcome data that was not possible before, enables sponsors to build a far richer and more insightful picture of intervention effects, which will aid early decision-making and contribute to labelling claims in the future. While these remain exploratory in nature and more work is needed to provide the level of validation around these new endpoints, they have great potential to aid drug development and regulatory decision-making, and may also have value in the care and management of patients in routine care.

The life sciences industry should adopt a culture of facilitating the exploration of new technology implementation within trials in an exploratory way, and aim to share experience, information, and access to the technologies showing most promise. Only through extended use will sufficient data and experience of using these new endpoints be accumulated to enable their acceptance in regulatory decision-making.

References

1. Byrom B, Mc Carthy M, Schueler P, Muehlhausen W. Brain Monitoring Devices in Neuroscience Clinical Research: The Potential of Remote Monitoring Using Sensors, Wearables, and Mobile Devices. Clinical Pharmacology and Therapeutics, 2018.
2. Katholic University of Leuven. Campus Insight Magazine. www.kuleuven.be/ci/28/28.pdf. (May 2010).
3. Looney, D.P. et al. The in-the-ear recording concept user-centered and wearable brain monitoring. IEEE Pulse 3(6), 32–42 (2012).
4. Prichep, L.S., John, E.R., Howard, B., Merkin, H., Hiesiger, E.M. Evaluation of the pain matrix using EEG source localization: a feasibility study. Pain Med. 12(8), 1241-8 (2011).
5. dos Santos Pinheiro, E.S. et al. Electroencephalographic Patterns in Chronic Pain: A Systematic Review of the Literature. PLoS ONE 11(2): e0149085 (2016).
6. https://www.floodlightopen.com/en-US/
7. Lipsmeier F et al. Evaluation of smartphone-based testing to generate exploratory outcome measures in a phase 1 Parkinson’s disease clinical trial. Mov Disord 2018; 33: 1287-1297.
8. Breedon, P., Byrom, B., Siena, L., Muehlhausen, W. Enhancing the Measurement of Clinical Outcomes Using Microsoft Kinect. International Conference on Interactive Technologies and Games (iTAG) 2016. IEEE Xplore. http://ieeexplore.ieee.org/document/7782516. (2016).
9. Jintronix. http://www.jintronix.com/wp-content/uploads/2016/07/TNJH-Case-Study.pdf
10. Breedon P et al. Face to face: An interactive facial exercise system for stroke patients with facial weakness, In (Rehabilitation: Innovations and Challenges in the Use of Virtual Reality Technologies; Eds: Wendy Powell, Albert “Skip” Rizzo, Paul M. Sharkey, Joav Merrick; Nova Science Publishers, 2017.
11. Breedon P, Byrom B, Siena L, Muehlhausen W. (2016) Enhancing the Measurement of Clinical Outcomes Using Microsoft Kinect. Interactive Technologies and Games (iTAG), 2016 International Conference on 2016 Oct 26 (pp. 61-69). https://doi.org/10.1109/iTAG.2016.17.
12. Byrom B, Walsh D, Muehlhausen W. New Approaches to Measuring Health Outcomes - Leveraging a Gaming Platform. Journal for Clinical Studies 2016; 8(6):40-42.
13. Lee SK et al. Measurement of shoulder range of motion in patients with adhesive capsulitis using a Kinect. PLOS ONE, 2015.
14. Byrom, B. et al. Selection of and Evidentiary Considerations for Wearable Devices and Their Measurements for Use in Regulatory Decision Making: Recommendations from the ePRO Consortium. Value Health (2018).

Bill Byrom, PhD, is VP, Product Strategy and Innovation, CRF Bracket.