CRO/Sponsor

Clinical Trial Supply Chain

Decentralized Clinical Trials

Early Phase Development

Investigative Sites

Protocol Design

Risk-Based Monitoring

Study Start-Up

Trial Design

Clinical Operations & Strategy

Diversity Action Plans

Industry Initiatives

Recruitment

Analytics

Electronic Data Capture

Metrics & Benchmarks

Real World Evidence

Data Management

CRAACO

Enrollment

Integrated Research

Patient Participation

Institutional Review Board & Ethics Committee

Regulatory

Artificial Intelligence/Machine Learning

eClinical

mHealth

Technology

News

Academy

Publications

All Publications

E-Books

Media

Editorial Podcasts

Editorial Videos

Peer Exchange

Practice Academy

Conferences

Conference Coverage

Conference Listing

Columns

All Columns

A Closing Thought

Clinical Trial Insights

View from Brussels

View from Washington

Resources

Advertise

Blogs

Events

Front & Center

Partner Perspectives

White Papers

Webcasts

Subscribe

Latest solutions, case studies, interviews, practical articles for clinical trials operations, data management, technology, investigative sites and services.

About

Contact Us

Editorial Contacts

Do Not Sell My Personal Information

Privacy Policy

Terms and Conditions

Applied Clinical Trials Online

LinkedIN

Instagram

Facebook

Youtube

Soundcloud

Threads

Spotlight

DE&I

Topic

1rem

Authors

Vice President, Vault R&D and Quality, Veeva Systems

A focus on data and process excellence advances product development.

Five Predictions That Will Drive R&D Effectiveness in 2025

Pandemic sparks new long-term plan for success in R&D.

How Digital Will Transform Product Development in 2022 and Beyond

How site management team leader Emma Earl, a 2021 Veeva R&D Hero, is developing a culture of improvement at Bayer, or what Japanese business strategists have long called Kaizen, in its clinical operations.

Clinical Kaizen at Bayer: Moving Past Spreadsheets to Unify Operations

Video interview with Sepehr Shojaei, VP of Design Solutions, Lightship

Lightship's Approach to Accessible, Inclusive, and Impactful Clinical Trials

Lightship focuses on patient accessibility and centricity in clinical trials, ensuring the patient experience is a positive one. Learn more about:

Lightship’s purpose and mission as a company

How the nursing teams and home visits help improve study participation 

The role of mobile units and how they position Lightship to improve patient accessibility

Sepehr Shojaei, VP of Design Solutions at Lightship, talks about how the company works with pharma and contract research organization (CRO) partners to support clinical trials that are patient-first and accessible.

Latest Conference Coverage

Lightship’s Approach to Accessible, Inclusive, and Impactful Clinical Trials

One2Treat Trial Design

ACT: What are your primary objectives for attending SCOPE 2025, and what insights are you hoping to gain regarding trial design innovations?

Like most people coming to SCOPE, we came here to see what's driving innovations in clinical trials and operations. At SCOPE, you have hundreds of companies, vendors, and clinical startups like us, and they all present in different tracks that are of interest to us. Specifically, I’m most interested in the patient centric trial design track.

ACT: Are there any specific trends in trial design, such as decentralized trials, real-world data integration, or AI-driven protocols that you anticipate being major discussion points?

Those are all very interesting tracks that are well presented here. But as I mentioned, I'm more interested in patient centric trial design and what it entails. While those methods are used in patient centric trial design, they are not the only one that are being used. The FDA recently promoted regulations and guidance on how to include patients in clinical trial development, but they don't really say how to do it. That’s why we come to these kinds of conferences, to promote new, concrete ways on how can you include patient voices into clinical trial design and strategy.

One2Treat s attending SCOPE to explore innovations in clinical trial operations, with a strong focus on patient-centric trial design. They highlight the event’s diverse range of participants, including companies, vendors, startups, and sponsors, all contributing to discussions on trial advancements.

While decentralized trials, real-world data integration, and AI-driven protocols are major industry trends, the speaker is primarily interested in how patient voices can be effectively integrated into trial design. They note that while the FDA has recently issued guidance on patient inclusion, it lacks detailed implementation strategies. Conferences like SCOPE provide a platform to discuss concrete methods for incorporating patient perspectives into trial strategies.

Regarding partnerships, Salvaggio emphasizes the importance of collaboration with site investigators and patient representatives. These stakeholders play a crucial role in ensuring that trials are designed with patient needs and experiences in mind, ultimately improving trial success and patient engagement.

In an interview with Applied Clinical Trials Associate Editor Don Tracy, Samuel Salvaggio, Senior Trial Design Lead, One2Treat, discusses the company's main goals for attending the SCOPE Summit, including patient centric trial design. 

SCOPE Summit 2025: Samuel Salvaggio Discusses One2Treat's Primary Objectives for Attending SCOPE

Dr. Peter Wahl, MLA, MS, ScD, VP and Global Head of Scientific Affairs at CorEvitas, part of the PPD™ clinical research business of Thermo Fisher Scientific, discusses the critical role of protocol-driven registries in generating regulatory grade RWE to serve multiple and evolving evidence needs across the drug development life cycle. He covers key considerations for selecting the right type of registry, explains why registry data is the gold standard in RWD, and how it contextualizes safety and effectiveness to fulfill FDA and EMA post-marketing requirements.

Making the Grade: Registries as Sources of Regulatory-Grade Real-World Evidence (RWE)

Reliable data is critical for making informed decisions about the efficacy and safety of new treatments in clinical trials. While self-reporting from the trial participant themselves on signs, symptoms, feelings, and function is always preferable, in certain populations it is common, or indeed necessary, for data reported by caregivers to form the primary or secondary endpoint. For example, trials in children who are unable to self-report, and individuals with cognitive dysfunction or neurodegeneration such as Alzheimer’s disease often rely on data reported by their caregivers (this does not refer to a healthcare professional). Such observer-generated outcome-related data is termed observer-reported outcome (ObsRO) data.

The challenges of missing data vs inter-rater variability

In the ideal scenario, the same caregiver would respond throughout the trial to maintain consistency in the data being reported; however, it is not always the case that a trial participant will have one, dedicated caregiver making and reporting all relevant observations. Although ObsROs should only be based on observable parameters, there is still potential for subjectivity in the way an observer rates a given behavior, symptom, or event, and, therefore, the potential for variability between raters.

 If multiple raters were to report on the same endpoint, then any discordance between raters when pooling data could obscure treatment effects.

Assuming the same caregiver is always with the participant is unrealistic, especially when data is being captured remotely; among many situations, the reality includes children splitting time between separated parents, being looked after by grandparents, and older populations being cared for by multiple caregivers. This issue needs to be addressed regarding the way ObsRO data are managed, analyzed, and reported to authorities.

It is well documented that missing data presents a large challenge to evaluating treatment efficacy, and although there are established techniques to help deal with missing data during analyses, these cannot be seen to be a replacement for the actual data points, and in any case, will only be appropriate in certain circumstances. This, therefore, leads to the question of whether restricting data collection to a single caregiver and potentially increasing the amount of missing data points is preferable to multiple reporters that could lead to more complete but higher-variability data.

It is worth highlighting that this is not a new issue. It is unlikely that the same caregiver consistently completes ObsRO(s) during a trial for a given participant. Even if it is the same reporter, they may not complete the ObsRO(s) within the specified time window. Yet these measures have traditionally been completed on paper, and ensuring attributability and contemporaneousness is far more challenging for that modality.

ObsROs are increasingly completed electronically, and yet there remains a higher level of scrutiny that electronic data capture is subject to despite the numerous benefits it offers over paper. Have the increased safeguards to data collection offered by electronic data capture, such as unique logins and preventing data completion outside of the predefined time windows, brought greater attention to the issue of multiple caregiver report?

Here, we highlight some areas for considerations for data collection in clinical trials that consist of multiple caregivers.

Training is key for all stakeholders in clinical trials, and if there are multiple caregivers reporting in a trial, it is important that they all receive role-specific training. It can be challenging to provide training on specific copyrighted ObsRO measures (beyond any that are already mandated by the copyright holder), as they have often gone through a validation process to ensure that the instructions and items can be understood without assistance from anyone else. 

When the same individual is reporting throughout, any difference in the way items are interpreted and rated between reporters (e.g., mothers of different participants) in a trial would be less concerning, as there would still be consistency within the reporter associated with a given participant. 

Where there are multiple reporters associated with the same participant (e.g., mother and father), any differences in the way an ObsRO is interpreted can introduce noise into the data when a different reporter is responding at different timepoints. Thus, some form of training to reduce this may be beneficial (such as what constitutes a severe symptom). However, as highlighted, this will need to be carefully thought through for copyrighted measures, and according to the specifics of the protocol.

For home-grown measures, this can be more straightforward. For example, it is common for event-driven diaries to be developed for a specific study, and training on what constitutes a discrete event and how to record it will be a core component of training for all raters.

Setting up the electronic data capture system

It is important to ensure the electronic clinical outcome assessment system is set up to facilitate multiple caregiver reports. This includes each reporter having a unique identifier to ensure data attributability. It is also important that data flows into the same location, and given the risk of duplicate data entry, that devices/accounts are synced as close to real-time as possible. Planning for how duplicate data entry (if it does occur) will be addressed in the dataset and analyses will also be important.

Inter-rater variability is a known phenomenon, and it is well-documented that pooling data from multiple respondents can create noise. An ISPOR Task Force Report on this topic states that it “raises the question of how data from different reporters can support a single claim.”

 However, that mainly addresses the scenario for when different age-group versions of the same measure are being used and where the reporter can vary between parent and child, though the concept of multiple reporters is similar. While one approach could be to pool the data into a single analysis, it would be necessary to demonstrate that there is an acceptable level of agreement between the raters. It would also be important to conduct sensitivity analyses and examine if outcomes are similar. Further guidance on this would be beneficial, including how results should be interpreted if the two reporters have different perceptions.

For any clinical trial, a robust statistical analyses plan is critical to outline the approach that will be taken.

As with any topic related to medical product development, regulatory considerations should inform study design. Understanding what is an (un)acceptable level of missing data and what evidence would be required to approve a treatment where efficacy is based on data from multiple caregivers will help ongoing conversations around the reality of multiple caregiver reports.

While the preference is always for participants to self-report, and where ObsROs are used that it is a single observer reporting throughout, this is not the reality. The request for enabling multiple reporters within the electronic data capture system is becoming more common, and best practices should start being developed. While such guidance does not exist yet, we cannot ignore this reality, and we hope this is an area that receives more attention to ensure the optimal approach is taken.

, PhD, Chief Scientific Officer, uMotif; and 

, PhD, PharmD, Chief Medical Officer, Kayentis.

1. Langberg J.M.; Epstein, J.N.; Simon, J.O.; et al. Parental Agreement on ADHD Symptom-Specific and Broadband Externalizing Ratings of Child Behavior. 

https://journals.sagepub.com/doi/abs/10.1177/1087054714561290

2. Davé, S.; Nazareth, I.; Senior, R.; et al. A Comparison of Father and Mother Report of Child Behavior on the Strengths and Difficulties Questionnaire. 

https://pubmed.ncbi.nlm.nih.gov/18266104/

3. Matza, L.S.; Patrick, D.L.; Riley, A.W.; et al. Pediatric Patient-Reported Outcome Instruments for Research to Support Medical Product Labeling: Report of the ISPOR PRO Good Research Practices for the Assessment of Children and Adolescents Task Force. 

https://pubmed.ncbi.nlm.nih.gov/23796280/

The conundrum of missing data vs. inter-rater variability.

Data Collection in Clinical Trials with Multiple Caregivers

Click the title above for a link to open the Applied Clinical Trials February 2025 issue in an interactive PDF format.

Jim Reilly

Articles

Latest Updated Articles