CRO/Sponsor

Clinical Trial Supply Chain

Decentralized Clinical Trials

Early Phase Development

Investigative Sites

Protocol Design

Risk-Based Monitoring

Study Start-Up

Trial Design

Clinical Operations & Strategy

Diversity Action Plans

Industry Initiatives

Recruitment

Analytics

Electronic Data Capture

Metrics & Benchmarks

Real World Evidence

Data Management

CRAACO

Enrollment

Integrated Research

Patient Participation

Institutional Review Board & Ethics Committee

Regulatory

Artificial Intelligence/Machine Learning

eClinical

mHealth

Technology

News

Academy

Publications

All Publications

E-Books

Media

Editorial Podcasts

Editorial Videos

Peer Exchange

Practice Academy

Conferences

Conference Coverage

Conference Listing

Columns

All Columns

A Closing Thought

Clinical Trial Insights

View from Brussels

View from Washington

Resources

Advertise

Blogs

Brand Insights

Events

Front & Center

White Papers

Webcasts

Subscribe

Latest solutions, case studies, interviews, practical articles for clinical trials operations, data management, technology, investigative sites and services.

About

Contact Us

Editorial Contacts

Do Not Sell My Personal Information

Privacy Policy

Terms and Conditions

Applied Clinical Trials Online

LinkedIN

Instagram

Facebook

Youtube

Soundcloud

Threads

Spotlight

DE&I

Topic

1rem

Authors

The conundrum of missing data vs. inter-rater variability.

Data Collection in Clinical Trials with Multiple Caregivers

Reliable data is critical for making informed decisions about the efficacy and safety of new treatments in clinical trials. While self-reporting from the trial participant themselves on signs, symptoms, feelings, and function is always preferable, in certain populations it is common, or indeed necessary, for data reported by caregivers to form the primary or secondary endpoint. For example, trials in children who are unable to self-report, and individuals with cognitive dysfunction or neurodegeneration such as Alzheimer’s disease often rely on data reported by their caregivers (this does not refer to a healthcare professional). Such observer-generated outcome-related data is termed observer-reported outcome (ObsRO) data.

The challenges of missing data vs inter-rater variability

In the ideal scenario, the same caregiver would respond throughout the trial to maintain consistency in the data being reported; however, it is not always the case that a trial participant will have one, dedicated caregiver making and reporting all relevant observations. Although ObsROs should only be based on observable parameters, there is still potential for subjectivity in the way an observer rates a given behavior, symptom, or event, and, therefore, the potential for variability between raters.

 If multiple raters were to report on the same endpoint, then any discordance between raters when pooling data could obscure treatment effects.

Assuming the same caregiver is always with the participant is unrealistic, especially when data is being captured remotely; among many situations, the reality includes children splitting time between separated parents, being looked after by grandparents, and older populations being cared for by multiple caregivers. This issue needs to be addressed regarding the way ObsRO data are managed, analyzed, and reported to authorities.

It is well documented that missing data presents a large challenge to evaluating treatment efficacy, and although there are established techniques to help deal with missing data during analyses, these cannot be seen to be a replacement for the actual data points, and in any case, will only be appropriate in certain circumstances. This, therefore, leads to the question of whether restricting data collection to a single caregiver and potentially increasing the amount of missing data points is preferable to multiple reporters that could lead to more complete but higher-variability data.

It is worth highlighting that this is not a new issue. It is unlikely that the same caregiver consistently completes ObsRO(s) during a trial for a given participant. Even if it is the same reporter, they may not complete the ObsRO(s) within the specified time window. Yet these measures have traditionally been completed on paper, and ensuring attributability and contemporaneousness is far more challenging for that modality.

ObsROs are increasingly completed electronically, and yet there remains a higher level of scrutiny that electronic data capture is subject to despite the numerous benefits it offers over paper. Have the increased safeguards to data collection offered by electronic data capture, such as unique logins and preventing data completion outside of the predefined time windows, brought greater attention to the issue of multiple caregiver report?

Here, we highlight some areas for considerations for data collection in clinical trials that consist of multiple caregivers.

Training is key for all stakeholders in clinical trials, and if there are multiple caregivers reporting in a trial, it is important that they all receive role-specific training. It can be challenging to provide training on specific copyrighted ObsRO measures (beyond any that are already mandated by the copyright holder), as they have often gone through a validation process to ensure that the instructions and items can be understood without assistance from anyone else. 

When the same individual is reporting throughout, any difference in the way items are interpreted and rated between reporters (e.g., mothers of different participants) in a trial would be less concerning, as there would still be consistency within the reporter associated with a given participant. 

Where there are multiple reporters associated with the same participant (e.g., mother and father), any differences in the way an ObsRO is interpreted can introduce noise into the data when a different reporter is responding at different timepoints. Thus, some form of training to reduce this may be beneficial (such as what constitutes a severe symptom). However, as highlighted, this will need to be carefully thought through for copyrighted measures, and according to the specifics of the protocol.

For home-grown measures, this can be more straightforward. For example, it is common for event-driven diaries to be developed for a specific study, and training on what constitutes a discrete event and how to record it will be a core component of training for all raters.

Setting up the electronic data capture system

It is important to ensure the electronic clinical outcome assessment system is set up to facilitate multiple caregiver reports. This includes each reporter having a unique identifier to ensure data attributability. It is also important that data flows into the same location, and given the risk of duplicate data entry, that devices/accounts are synced as close to real-time as possible. Planning for how duplicate data entry (if it does occur) will be addressed in the dataset and analyses will also be important.

Inter-rater variability is a known phenomenon, and it is well-documented that pooling data from multiple respondents can create noise. An ISPOR Task Force Report on this topic states that it “raises the question of how data from different reporters can support a single claim.”

 However, that mainly addresses the scenario for when different age-group versions of the same measure are being used and where the reporter can vary between parent and child, though the concept of multiple reporters is similar. While one approach could be to pool the data into a single analysis, it would be necessary to demonstrate that there is an acceptable level of agreement between the raters. It would also be important to conduct sensitivity analyses and examine if outcomes are similar. Further guidance on this would be beneficial, including how results should be interpreted if the two reporters have different perceptions.

For any clinical trial, a robust statistical analyses plan is critical to outline the approach that will be taken.

As with any topic related to medical product development, regulatory considerations should inform study design. Understanding what is an (un)acceptable level of missing data and what evidence would be required to approve a treatment where efficacy is based on data from multiple caregivers will help ongoing conversations around the reality of multiple caregiver reports.

While the preference is always for participants to self-report, and where ObsROs are used that it is a single observer reporting throughout, this is not the reality. The request for enabling multiple reporters within the electronic data capture system is becoming more common, and best practices should start being developed. While such guidance does not exist yet, we cannot ignore this reality, and we hope this is an area that receives more attention to ensure the optimal approach is taken.

, PhD, Chief Scientific Officer, uMotif; and 

, PhD, PharmD, Chief Medical Officer, Kayentis.

1. Langberg J.M.; Epstein, J.N.; Simon, J.O.; et al. Parental Agreement on ADHD Symptom-Specific and Broadband Externalizing Ratings of Child Behavior. 

https://journals.sagepub.com/doi/abs/10.1177/1087054714561290

2. Davé, S.; Nazareth, I.; Senior, R.; et al. A Comparison of Father and Mother Report of Child Behavior on the Strengths and Difficulties Questionnaire. 

https://pubmed.ncbi.nlm.nih.gov/18266104/

3. Matza, L.S.; Patrick, D.L.; Riley, A.W.; et al. Pediatric Patient-Reported Outcome Instruments for Research to Support Medical Product Labeling: Report of the ISPOR PRO Good Research Practices for the Assessment of Children and Adolescents Task Force. 

https://pubmed.ncbi.nlm.nih.gov/23796280/

CluePoints Risk Detection

ACT: What are CluePoints’ key objectives for participating in SCOPE 2025, and how do you see the conference shaping the conversation around risk-based quality management?

I think the very first thing is we're delighted to be here, because there's a need to validate the conversation. 12 years ago, a word that didn’t exist in our vocabulary—nor did AI, to be honest. Now it’s front and center. We’re here really to continue that dialogue, to share and expand, but also transition from the potential to the reality. This is what we're doing today, this is what we’re going to do tomorrow. I think that’s the exciting thing about meeting like this, it’s not a false environment, it's a genuine collaborative environment with so many different conversations, and it's nice to be at the center of it.

ACT: How has CluePoints’ approach to risk detection and mitigation evolved in response to the increasing complexity of global clinical trials?

We’re about to live through the ICH GCP E6(R3) release. In fact, we’re hosting a meeting with the FDA in just about five weeks’ time to go over it. It's going to completely change our approach. I actually think one of the biggest outcomes will be to challenge the use of that word risk. It's a word that confuses, and I wonder whether we might move into more of an informed or a data-driven era where risk isn't a negative word, it's a really positive phrase. But those guidelines are going to change everything from chain of custody to the approach we take, to just embedding a transformational, modern methodology into every decision we make.

At SCOPE this year, the key objective is to advance the conversation around risk-based quality management (RBQM), transitioning it from a theoretical concept to an actionable reality. The conference provides a collaborative platform to discuss the evolution of RBQM, particularly in light of emerging technologies like AI, which were not even part of the industry's vocabulary a decade ago. The goal is to validate and expand discussions, ensuring that RBQM is implemented effectively across clinical trials.

A significant regulatory shift impacting RBQM is the imminent release of the updated ICH GCP E6(R3) guidelines. These changes are expected to transform how sponsors approach RBQM, potentially even redefining the concept of “risk” itself. Rather than being viewed as a negative factor, risk management could shift toward a more data-driven, informed decision-making process. These guidelines will influence aspects such as chain of custody and the integration of modern methodologies into trial management.

The future of RBQM is increasingly centered around data as a comprehensive ecosystem rather than isolated data points. Industry stakeholders are prioritizing different drivers—cost, time, resources, or quality—in their approach to RBQM. While all four factors are important, organizations must determine their primary focus to optimize decision-making. The industry is working toward a model where these priorities can be balanced more effectively, ensuring that clinical trials remain patient-centric while also meeting regulatory and operational goals.

Ultimately, SCOPE serves as a critical venue for shaping the future of RBQM, fostering discussions that drive innovation and efficiency in clinical research.

In an interview with Applied Clinical Trials Associate Editor Don Tracy, Richard Young, chief strategy officer, CluePoints, discusses goals of the SCOPE Summit and CluePoints' evolving approach to risk detection. 

Latest Conference Coverage

SCOPE Summit 2025: Richard Young on CluePoints' Approach to Risk Detection in Clinical Trials

As the use of artificial intelligence (AI) has greatly accelerated in recent years, AI has been moving to reshape the clinical trial landscape by addressing critical challenges and improving outcomes across the entire process. According to 

AI is streamlining the clinical trial process through recruiting and enrollment, trial design, protocol optimization, data management and analysis, enhanced patient retention and compliance, regulatory compliance, post-trial analysis, and real-world evidence.

The article states that in trial design, machine learning (ML) models simulate outcomes and enable adaptive protocols to enhance efficiency and decision-making. Additionally, AI automates data collection and performs advanced real-time analyses, reducing errors and uncovering actionable insights, while also improving patient retention through personalized engagement and behavioral monitoring.

 US Pharma and Biotech Summit, Tala Fakhouri, associate director, policy analysis, FDA, discussed the FDA’s point of view on ML and AI in drug development.

“We're actually very excited about AI use,” said Fakhouri. “I think we're seeing that it's increasing efficiencies in different parts of the drug development process. If you think about things such as discovery or protein folding, which again, is outside of what we normally look at, it could potentially cut the development time by years. This is all very exciting, because it could translate into faster, safe and effective drugs coming into the market. It can also fill in certain gaps for rare diseases where we can see a lot of potential use for AI to accelerate the development of drugs. In this type of situation, that's what I would say would be the long term.”

Fakhouri added that in the short-term, the primary challenge that stakeholders are working to address is the adoption curve.

“You need to train your staff, you need to bring in the right expertise, and you need to develop the right tools to solve the right problems,” Fakhouri said. “That's going to take some time and that's why I think the short-term uses of AI are going to be mostly low hanging type of fruits where you're increasing operational efficiency, but then that will translate into the development of safe and effective drugs faster.”

On October 31, 2024, the FDA released updated guidelines on the use of AI for drug development. According to the guidance, the regulatory body recognizes the growing role of AI in drug development, spanning the entire drug lifecycle from nonclinical and clinical phases to manufacturing and post marketing.

In order to support industry innovation and maintain safety and efficacy standards, the Center for Drug Evaluation and Research (CDER) released a draft guidance earlier this year addressing AI’s use in regulatory decision-making. According to the updated FDA guidelines, the guidance is influenced by extensive public feedback, CDER’s review of over 500 AI-related submissions since 2016, workshops with stakeholders, and aligns with broader FDA efforts to ensure responsible AI use.

Fakhouri previously stated that the FDA had no intention of placing restrictions on the use of AI and ML as time goes on.

“We get asked if the FDA regulates large language models or if we’re going to ban generative AI use,” she explained. “My response is that we typically don't regulate linear regression. We look at the data and the information that any modeling technique is producing, and we want to make sure that the information is trustworthy. So, I wouldn't say that we would be banning or prohibiting a certain AI or machine learning type of algorithm, what we're actually interested in is how robust how accurate and how credible the information from these models is.”

According to IBM, there are a number of benefits associated with implementing AI in clinical trials and medicine development, including:

IBM states that a trained ML algorithm can help cut down on research time by giving clinicians valuable search results with evidence-based insights about treatments and procedures while the patient is still in the room with them.

 Peer-reviewed studies have suggested that AI-powered decision support tools can help improve error detection and drug management.

 This includes reducing medication errors, customized virtual health assistance, fraud prevention, and supporting more efficient administrative and clinical workflows.

Increasing Physician-Patient Engagement: 

AI can help provide around-the-clock support through chatbots that can answer basic questions and give patients resources when their provider’s office isn’t open.

There are many thought leaders in the industry who believe that AI has the potential improve healthcare in all areas in the future. However, the biopharma industry has moved at a slower rate to implement AI compared to other fields.

At this year’s Young & Partners Pharmaceutical Executive Summit, Najat Khan, chief R&D officer, chief commercial officer, Recursion, offered her thoughts on why utilization has advanced at a slower rate and how companies can properly integrate AI into their everyday workflows.

“A huge part of it is the mindset, but pharma companies, in general, have high margins,” said Khan. “Whenever an industry has high margins, what’s the reason for change? That's the truth of it. Second, there’s a lack of understanding when it comes to using it. I think everybody knows what the pain points are. But then how do you deploy it? Sometimes, it means that your job changes. That's threatening, right? I've seen that firsthand, and this has happened over many generations. There’s two ways to respond. Either you evolve or you get evolved. You don't have to be a coder to actually understand AI. It's just like another science or another technical topic. But it's more important to understand how to use it, what not to do, what to do—given the complexity of the pharma industry, the regulations that are there, the limited understanding of biology today—there's just a fear of both personally being threatened.”

1. How AI is Revolutionizing Clinical Trials. 

September 24, 2024. Accessed November 1, 2024. https://www.linical.com/articles-research/how-ai-is-revolutionizing-clinical-trials

2. Artificial Intelligence for Drug Development. 

October 31, 2024. Accessed November 1, 2024. https://www.fda.gov/about-fda/center-drug-evaluation-and-research-cder/artificial-intelligence-drug-development

3. US Pharma and Biotech Summit 2024: Artificial Intelligence and Machine Learning Through the Eyes of the FDA Part II. 

May 21, 2024. Accessed November 1, 2024. 

https://www.pharmexec.com/view/us-pharma-and-biotech-summit-2024-artificial-intelligence-and-machine-learning-through-the-eyes-of-the-fda-part-ii

4. What is artificial intelligence in medicine? 

https://www.ibm.com/topics/artificial-intelligence-medicine

5. The Impact of Artificial Intelligence on the Creation of Medicines. 

October 24, 2024. Accessed November 1, 2024. https://www.pharmexec.com/view/young-partners-pharmaceutical-executive-summit-2024-impact-artificial-intelligence-creation-medicines

With the increasing use of artificial intelligence and machine learning in medicine development, what role will they play in the clinical trial space moving forward? 

A New Beginning: The Effect of Artificial Intelligence on Clinical Trials, Medicine Creation 

Alzheimer’s disease (AD), as well as other neurodegenerative dementias, such as frontotemporal dementia (FTD), can be characterized by progressively worsening deficits in several cognitive domains, including language. Patients with these conditions may demonstrate different speech patterns, such as difficulty in retrieving words (anomia), reduced information content, sentence comprehension deficits, and a lack of cohesion in discourse. Despite the prevalence of language characteristics as a symptom of these disorders, tools used to identify the extent of cognitive impairment in the study of AD and FTD do not typically monitor speech or rely on subjective ratings. Growing awareness of the opportunities that language evaluation can bring to clinical research is driving interest in speech-based digital biomarkers to evaluate changes in cognition and achieve more sensitive detection of the impact of treatments.

Challenges of detecting change in AD and FTD trials

Investigational drugs for AD and FTD have an extensive history of failing to demonstrate efficacy in Phase II and Phase III clinical trials. In trying to find reasons for these setbacks there is agreement that the limited sensitivity of current clinical trial endpoints may be a factor. Most of the standard primary endpoints used in AD and FTD research are clinical measures of cognition and/or function, such as the Clinical Dementia Rating (CDR) scale. While global rating scales serve a helpful purpose in providing a quantifiable clinical judgement about disease severity, there is an awareness that these tests do not adequately capture slight changes or improvements in a patient’s condition.

Putting questions on sensitivity aside, these standard approaches to testing are also hugely complex, difficult to administer and time intensive. A single test can take up to an hour to complete. Most also need to be administered by a clinician or psychometrist, meaning that patients are unable to complete them remotely. Repeatability may also pose challenges due to the potential for non-memory impaired patients to learn what to expect if these same tests are revisited regularly.

With the pressing need to develop and evaluate disease-modifying treatments for these diseases, there is an appetite from researchers for new tools and more sensitive means of measuring function and cognition in AD and FTD patients. Language represents a huge opportunity. As more sophisticated, automated, and objective ways of analyzing speech come into the frame, there is potential to gain insight into the language function of patients in a way that has historically not been achievable.

The adoption of speech as a biomarker in clinical research has the potential to capture more subtle speech and language changes in patients with cognitive impairment. People with AD and FTD may struggle to recall words, they may resort to basic vocabulary, construct simpler sentences, and speak at a slower rate. While clinicians and investigators may notice subtle language problems in patients at the early stages of these diseases, they do not currently have the tools to objectively characterize these changes and how they worsen or improve over time.

Technology that can facilitate the automated assessment of speech can provide more objective measures to complement clinical rating scales. Winterlight’s approach involves automatically extracting hundreds of acoustic and linguistic properties of natural speech. The fine-grained data can be used to characterize the speech patterns associated with AD, FTD, and other neurodegenerative disorders. The platform uses short-recorded speech samples captured through speech tasks including picture description, paragraph reading, paragraph recall, phonemic fluency, semantic fluency, and object naming.

These advancements are helping to build a more complete picture of the common aspects of speech and language that decline with the progression of certain dementias. In analyzing just one to five minutes of a patient’s speech, hundreds of linguistic cues can be identified that may offer more accurate insight into the severity of their condition. These cues include lexical diversity, syntactic complexity, semantic content, and acoustics. Integrating speech-based technology into the design of AD and FTD clinical trials has the potential to contribute detailed insights into the subtle ways that patients’ symptoms improve or deteriorate as a study progresses.

From the patient’s perspective, testing via these types of platforms may also bring considerable improvements and reduce burden. As with all clinical trials, keeping AD and FTD patients, and their caregivers, engaged with a study is paramount. Typically, assessments are administered by study sites via tablets. A trained rater would walk the participant through a brief speech assessment, taking under five minutes to complete. The assessment would prompt them to speak in response to simple instructions; this may be a request to describe a picture, or simply asking a patient to express how they are feeling that day. The patient’s speech is recorded via the device’s microphone, with recordings uploaded to a cloud-based platform. The speech recordings are analyzed using Winterlight’s speech analysis platform and outcome measures are provided to the trial sponsor as an exploratory endpoint.

Assessments can also be made available to patients through their own tablets or smartphones, as well as devices provisioned by sponsors, for remote collection. Removing the need for patients to visit a site for this type of testing makes it easier for insights to be collected more frequently than may have been possible using traditional methods. Winterlight’s technology is currently being used in several Phase II and III clinical trials for AD and FTD, with many patients reporting that the assessments are easy-to-use and less taxing than other clinical assessments they have experienced. This work is helping to build greater understanding of the speech changes that occur in these patients over time.

The future of speech assessment in clinical trials

There is tremendous potential for neurological language assessments to improve the way that clinical trials are run, and the impact of new treatments measured. As command of speech and language are ecologically and functionally relevant clinical markers of disease progression, objective tools can help generate novel, sensitive endpoints able to be administered remotely and at lower burden to the patient. While speech-based biomarkers are currently being used as exploratory endpoints, there is increased awareness of the enhancements they may bring to the accurate and sensitive measurement of cognitive health in AD and FTD research. As more evidence-based data and validation is gained, it is anticipated that these measures will eventually be adopted as primary and secondary endpoints and may provide more detailed insight into the subtle ways that new treatments are impacting cognitive health.

 is the Director of Clinical Research at 

Future shows great potential for neurological language assessments in clinical trials.

How Speech-Based Digital Biomarkers are Redefining the Measurement of Disease Progression in Neurodegenerative Dementia Trials

AiCure, an AI and advanced data analytics provider, is working with the Defense Health Agency’s (DHA) PTSD Drug Treatment (PTSD-DT) program to deploy its medication adherence platform to guide patients in their compliance to complex treatment regimens, as well as its digital biomarker solution to capture the subtleties of a patient’s response. As PTSD has a wide range of symptoms, 

 aims to develop a precision medicine approach to prescribing the most effective therapy based on a patient’s unique biological and clinical characteristics. Rich Christie, MD, PhD, Chief Medical Officer at AiCure, spoke to our sister publication 

 Can you provide the background on AiCure’s digital biomarker solution as a whole?

: During a clinical trial, a precise and accurate understanding of a patient’s response to treatment is critical to measure the state of their disease and a drug’s impact. Traditional means of measuring a patient’s well-being, including sporadic in-person assessments or a patient’s self-reported outcomes, only tell clinicians how a patient is feeling at that moment in time. The stretches between these check-ins can tell an entirely different story, filled with critical, subtle cues about their condition.

AiCure’s AI-powered digital biomarker solution enables remote detection of subtle changes in a patient’s health status and response to treatment by capturing audio and visual data between clinic visits. Accessed through AiCure’s Patient Connect application, patients use their smartphone’s front-facing camera to complete brief assessments. AiCure’s algorithm then analyzes behavior, such as emotional expressivity, physical movement and speech patterns. By frequently aggregating these sensitive, objective insights, AiCure empowers pharmaceutical companies to improve their understanding of the disease and treatment side effects, elevating the integrity of their trial data, and optimizing patient outcomes.

 How does the technology work in the context of the DHA partnership?

 The DHA PTSD Drug Treatment Program is conducting an adaptive platform trial designed for the efficient and cost-effective testing of multiple PTSD treatments in both active service members and veterans. AiCure's medication adherence platform guides patients in their compliance to complex treatment regimens. The biomarker solution is described above, and the application features integrated built-in reminders and a communication system, allowing sites to contact the participant if they’ve missed a dose or check-in.

: Can you describe the challenges of PTSD treatment?

 PTSD has a wide range of symptoms and has significant variation in how it presents across patients. Eighty percent of PTSD patients have at least one additional comorbidity such as depression, anxiety or substance abuse disorder, further complicating diagnosis and treatment. Despite the societal, financial, and emotional strain that PTSD exerts upon patients, there are only two FDA-approved therapies on the market.

The PTSD-DT Program's adaptive platform trial will evaluate the effect of therapeutics on a patient's PTSD symptoms over 12 weeks. Because medication compliance has historically been a significant methodological problem in PTSD clinical trials, AiCure's adherence platform will offer patients the necessary support to ensure correct dosing. The digital biomarker solution can be used to capture the subtle changes we discussed earlier, and may help investigators identify the right drug for the right patient, or allow clinicians to intervene in a patient's treatment plan in a timely manner if the drug is not likely to be effective.

 What does it involve in terms of patient interactions?

 Each time a patient doses in the trial, they will use the AiCure application. The smartphone-based application leverages computer vision and AI to remotely assess patient dosing behavior, ensuring the right patient ingested the right drug at the right time. This allows sites to monitor adherence and follow up with participants in instances of non-adherence. For the biomarker assessments, participants will be asked to perform simple tasks, such as describing images presented through the AiCure app in a few sentences to the camera of the smartphone, in order to record and analyze their responses.

 How many patients will participate in the PTSD clinical trial?

Rich Christie, MD, PhD, Chief Medical Officer at AiCure discusses the use of the company's technology in a clinical trial for PTSD.


PTSD Study Uses Precision Medicine Tech from AiCure

The FDA and other regulatory bodies have issued multiple guidance documents addressing technology tools that are used in clinical trials. FDA guidance documents include Computerized Systems;

 is also available online. In 2010, the European Medicines Agency (EMA) issued a Reflection Paper on electronic source records in clinical trials,

 and recently, the Medicines and Healthcare Products Regulatory Agency (MHRA)

 issued a document on data integrity. In 2016, the Chinese FDA issued Technical Guidelines on Electronic Data Capture for Clinical Trials.

 issued a document addressing the use of CDISC standards to facilitate the use of electronic source data in clinical trials.
	
	The regulatory push to the paperless clinical trial has occurred despite a pharmaceutical industry that has been risk-averse in adopting modern-day technology tools that could support clinical trials. This risk aversion is in part due to fear by sites and sponsors of receiving a FDA Form 483.

 What if the FDA turns down an application if it discovers that, for example, a patient was born in 1982, when in the study database it is recorded as 1983, even when there is no impact on the study results? The clinical sites are also fearful of losing business as a result of any FDA Form 483 finding, however minor.
	
	The major stakeholders in the clinical trial enterprise include study subjects/patients, sponsors, clinical researchers, regulators and the public at large. Now that the electronic world has entered the clinical trials space, as part of routine pre-approval inspections there will be an assessment of the impact of electronic systems on 1) informed consent; 2) data integrity and accountability; 3) protocol compliance; 4) monitoring of the study by the sponsor, including risk-based monitoring; 5) patient safety by assuring that all safety events are accurately collected and reported, and 6) drug/device accountability.
	
	The following are examples of regulated clinical trial software currently being used:

		Electronic patient reported outcomes (ePRO)

		Electronic clinical outcome assessments (eCOA)

		Dedicated tablets collecting and storing data in real time and transmitting data on a schedule to the study database

		Web-based systems collecting data in real time which transmit the data to the study database only after the source record is securely received and stored in an independent eSource storage location under control of the study site

		Mobile-based medical apps whose functionality could pose a risk to a patient’s safety if the app were to not function as intended


	
	It is strongly recommended that a risk-based approach to software validation be assessed for any medical software/app, even those not being used in clinical trials, as there are moral and ethical issues if any software used for medical decision-making does not act as intended.
	
	For software to be “reliable,” it needs to validated. Therefore, assuring that software is in a validated state is a key factor when regulators look at software products used in clinical trials. Fortunately, in 2002, FDA provided validation guidelines for software supporting regulated products.


	
	The following sections address some of the needed steps to assure that software used in clinical trials will be adequate to support regulatory scrutiny during pre-approval inspections and the review of marketing applications.
	
	


	
	Here are some critical questions to be asked concerning the “intended use” of the software, and depending on the answers, appropriate steps must be put in place.
	
	1. Why am I using the software?
	2. How is it being used?
	3. Will the data be used to support decision-making such as:

	4. Will the data be used to support primary and secondary outcomes?
	5. Will the data be used to support subject’s safety?
	6. How will patient privacy and confidentiality be maintained?
	
	


	
	A software product used in clinical trials must be validated and compliant with the Code of Federal Regulations (CFR) incorporated in Title 21 (Food and Drugs), Chapter I, Subchapter A, Part 11.

 21CFR Part 11, as it is commonly referred to, sets forth the criteria under which the FDA considers electronic records, electronic signatures, and handwritten signatures executed to electronic records to be trustworthy, reliable, and generally equivalent to paper records and handwritten signatures executed on paper.
	
	There is a distinction between validation and compliance. Validation is objective evidence which demonstrates that a given piece of software performs as intended against the user needs and the documented specifications and requirements.

	While it is true that software requirements and resulting features within the software can support compliance with 21 CFR Part 11, it is also true that no software or hardware system can be 21 CFR Part 11 compliant on its own. The software implementation, including processes and procedures for use of the system, are what constitute 21 CFR Part 11 compliance.
	
	And validation of the software goes beyond just the software requirements and resulting features necessary for Part 11 compliance. Validation is necessary for the entire software application and should be a defined part of the Software Life Cycle (SLC).
	
	


	
	There should be very clear SLC management documentation that contains the software development process from cradle to grave. In fact, many SLCs can go beyond the development process, to include ongoing support and even consideration for end-of-life for the software.
	
	In terms of popular development methodology choices, there is the Agile development methodology versus the traditional Waterfall approach. For the former, when it is very clear what the deliverable will be, one makes a detailed plan and then implements the plan. In contrast, the Agile approach accepts that there will be changes during software development and has built in flexibility to change direction at a moment’s notice.
	
	The first validation element is the software concept/requirements, or in other words, what is the problem we are trying to solve and why do we need this software? Usually an executive or the person driving the software writes the concept document to convey the mission and rationale of this new endeavor. Next, the “requirements” phase must clearly document what the software intends to do. For example, the software should allow for study subjects to enter study outcome data over the web using any device and any browser.
	
	Once the requirements are buttoned down, there needs to be clarity on how the requirements will be executed, which comes under the heading of “functional specifications.” For example, if the requirement is that “changes to the data base need to tracked,” then the functional specification could say that there needs to be an audit trail which must display the original value, the new value, why the change was made, who made the change, and what was the reason for the change.
	
	The next phase is left to the programmers and architects who create the very technical “design document.” This document basically is in “geek-speak” and we leave that document to the specialists. Once programming begins, the programmers work closely with the “testers” to test and debug the program until at some point the development team says the software is ready for a formal user acceptance test (UAT). In an Agile methodology, the design document, test plan and test scripts are written concurrently with development, iterating as features are refined. In a Waterfall methodology, the test plan and test scripts are often written with the design document, all of which is completed before development even begins.
	
	Once the “test plan and test scripts” are written and signed off, formal testing can usually begin right away. During testing, the testers document the “errors, bugs and fixes,” and when all that is completed, the software is “released” and signed off by those responsible and accountable. Post-release, “version control and change management” kick in. A plan is also put in place for the retirement of the software when it will no longer be supported or commercially available.
	
	

The time is now for sites and sponsors to put away fears associated with data acquisition and monitoring technologies-and better support the regulatory push in recent years for standardized adoption of paperless trials.

Florence Mowlem, PhD

Articles

Latest Updated Articles