Using Machine Learning to Predict Placebo Response and Increase Clinical Trial Success

Erica Smith, PhD

Every year, the pharmaceutical industry spends millions in an effort to improve the evaluation of efficacy in clinical trials – including strategies to reduce risk relating to the placebo response. To an extent, the industry has learned to live with this issue because efforts to truly reduce placebo response have seemed to reach their maximum potential. The result? Increased costs. Longer timelines. And still high trial failure rates in Phase II and III. In fact, drugs in Phase III only have about a 50% chance of approval in areas like CNS and cardiovascular disease.¹

In this article, we discuss the prevalence of placebo response in clinical trials, the history of how trials have addressed it to date, and new approaches leveraging machine learning to predict placebo response and reduce the risk of trial failure.

Why do trials fail?

On one hand, a randomized, placebo-controlled trial is designed to clearly demonstrate efficacy of an experimental therapy over the placebo with an acceptable safety profile. On the other hand, it’s not this simple—and the data are rarely this clear.

In part, this is because of the placebo response, the measured improvement of a patient in a trial after receiving a sham treatment. Though there is no active ingredient in the placebo group, patients receiving placebo experience real clinical improvements that make it difficult to discern drug efficacy. The problem is that a high placebo response doesn’t necessarily mean the treatment doesn’t work, but the data variability from the placebo response generate noise that obscures the efficacy signal. So, trials fail.

The placebo response can account for as much as 60% of the measured treatment response across indications with pharmacologic interventions–regardless of indication.2 High placebo response rates are well-recognized in the development of CNS therapies; they have been implicated in a wide variety of indications and therapeutic areas, including immunology^3–5, dermatology^6,7, ophthalmology⁸ and others.This issue is even more pronounced for drugs targeting pain and central nervous system conditions, which have some of the lowest success rates1. For example, in osteoarthritis pain, the placebo response accounts for as much as 75% of pain reduction, 71% of functional improvement and 83% of stiffness reduction in patients receiving active drug.⁹ In depression, the placebo response is responsible for 67% of the measured treatment response, regardless of whether assessments are conducted by the patient or the physician.¹⁰ This undoubtedly has jeopardized the ability to clearly demonstrate efficacy of the study drug, preventing needed therapies or treatments from reaching patient.

To understand more about why trials fail due to the placebo response, let’s discuss where it actually comes from.

Influences of placebo response

By nature, the placebo response is influenced by many factors, including study biases, symptom reporting errors, regression to the mean, clinical site factors, demographics and, finally, the placebo effect–a complex psychobiological phenomenon with significant psychosocial components.

Some of these influences are extrinsic, meaning they can be addressed by altering certain external circumstances. Others, like the placebo effect, are intrinsic, meaning they are unique to the patient and, thus, are more difficult to influence. For example, the placebo effect is a real, multifactorial change in biochemical pathways in the brain, producing a real and beneficial change in symptoms. It’s inherently patient-specific, influenced by the patient’s expectations for improvement and certain well-defined personality traits.

So, when a patient takes a treatment, there’s a chance that the patient will have a placebo effect, contributing to the overall placebo response. Consequently, the patient-specific nature of the placebo effect may introduce a bias and variability in RCTs. This bias and variability have proven difficult, if not impossible, to prevent.

Attempts to address placebo response

Historical attempts to identify placebo responders involved placebo run-in periods and the exclusion of high placebo responders. With this approach, patients showing an improvement upon administration of the placebo during the pre-study period were excluded from the study.

Unfortunately, the results consistently fail to show a substantial benefit of this approach. Meta-analysis in antidepressant RCTs demonstrated that placebo lead-in phases did not decrease the placebo response, nor increase the difference in response between active drug and placebo groups.^11,12 Beyond this, run-in periods require additional exposure of patients to experimental therapies, longer study timelines and increased costs–all for little or no benefit.

Trials have also implemented patient training to address symptom reporting errors and site training to moderate staff-patient interactions and avoid expectation inflation. Patient training helps patients learn how to report their symptoms consistently and reliably before the study begins, eliminating some bias caused by inconsistent symptom reporting. Many clinical trials also train site personnel to use certain communication styles and body language techniques that limit any subtle cues, as the empathy of a clinical investigator may influence patient expectations.

For example, training of patients to accurately report pain symptoms improved consistency (correlation between multiple pain reporting instances) by approximately 10%, yet this difference failed to reach statistical significance (p=0.10).¹³ While these methods may have been moderately successful in reducing data variability, they are also resource demanding (both for the patient and investigator site staff).

Indeed, one universal solution to limit the impact of the placebo response in drug development in a variety of diseases has not yet been found. Optimizing study designs may be helpful but may also be a significant source of study bias—not to mention the strategy has been met with regulatory resistance. For example, use of a Sequential Parallel Comparison Design (SPCD) to identify placebo responders was cited as a potential reason for rejection of a recent application of a drug to treat depression.¹⁴ Patient and investigator site training has been widely employed in pain trials, although they can be challenging to implement in large, multi-center trials. It is also far more difficult to train patients to accurately report symptoms in studies with endpoints that are highly complex and less intuitive than pain (e.g. Quality of Life scales).

Because a universal solution has not yet been available to the industry, clinical trials continue to invest in patient and site training with minimal payoff. The need persists as drug development costs continue to rise and placebo response rates climb.

Using machine learning to predict placebo response

The advent of advanced data analytic techniques like artificial intelligence (AI)–which includes the sub-discipline of machine learning (ML)–have opened an opportunity to approach this decades-old problem with new tools. AI has primarily been used in drug discovery to predict things like bioactivity, toxicity and drug-protein interactions.¹⁵ Applications in clinical trials and drug development have been more limited, focusing on areas like optimizing trial startup, patient matching to trials, and accelerating patient enrollment and recruitment. Considering the complex and dynamic nature of the placebo response, AI poses a ripe opportunity to take the next step forward in mitigating the impact of the placebo response.

Indeed, predictive models powered by machine learning have been built that can predict the individual nature of the placebo response; in other words, predict why some patients experience very high placebo response and some patients experience essentially no placebo response.

Predicting placebo responsiveness for every trial patient would allow statisticians to adjust for the range in the entire clinical trial population. This reduces data variability in both drug-treated and placebo-treated groups, thus improving the ability to detect true drug efficacy.

Once each patient’s placebo responsiveness can be predicted, clinical trial statisticians can account for this in clinical data analysis. The Placebell method developed by Cognivia has been shown to improve assay sensitivity (i.e. the ability to distinguish placebo treatment from drug treatment) by nearly 40%¹⁶ and improve study power by 14% (i.e. decreased risk of trial failure due to false negative results). This approach can be implemented for about 1-3% the total per patient costs for the trial and can be applied to virtually any therapeutic area or indication.

The value of this statistical method is clear. So, how do you implement it? To do this, we need two things: a predictive model and patient psychology data.

1. Predictive model

To predict placebo responsiveness, you need a properly constructed machine learning-based predictive model that combines standard machine learning techniques with deep insight into clinical trials and patient characteristics.

A couple of key things need to come together to build this model.

First, the right features need to be selected as model inputs. For example, patient characteristics like personality, expectation, baseline disease intensity and demographics may be important.

Second, performance evaluation and model selection should be conducted with care for robustness as opposed to maximum variance explained. Linear models are a good place to start because they’re simple and may be easier to interpret. Finally, it’s critical to optimize methods to ensure translatability between trial designs, modes of drug administration and other trial factors.

After training and evaluating the predictive model in disease-specific applications, it can be used to define predicted placebo responsiveness for patients based on their unique psychological characteristics. This brings us to the second critical element: gathering patient psychology data.

2. Patient psychology data

As discussed, we know that placebo response results from not only extrinsic factors but also factors intrinsic to the patient. To understand and predict placebo response, then, trials should gather data on patient psychology and expectations. This is most easily accomplished with a questionnaire.

This questionnaire must address specific facets of a patient’s personality, expectations of the treatment, perceptions of the trial, beliefs and motivations. These characteristics that are intrinsic to the patient are critical to understand how likely they are to respond to a placebo.

When this data is combined with other baseline data (like demographics and medical history) as inputs in the predictive model, the model can assign a score to each individual patient, representing their predicted placebo responsiveness.

This allows statisticians to distinguish the range of placebo responders at the beginning of the study so they can define a covariate, reducing the impact of the placebo response in the final data analysis.

Conclusion

Historically, the industry has used methods available at the time–ranging from excluding placebo responders identified through placebo run-in periods to training patients to reproducibly report symptoms. Unfortunately, the impact of these methods is modest at best–and has plateaued, considering that the placebo response continues to increase over time.^17–19 Ultimately, the overall risk of trial failure is still too high, meaning more repeat trials, lost timelines and premature abandonment of programs.

Using predictive models based on machine learning provides drug developers with more advanced options to manage the placebo response at lower cost, patient and site burden than historical methods.

With the help of modern tools, we can begin to think about placebo response in new ways, helping more clinical trials succeed.

Erica Smith, PhD, Chief Business Officer, Cognivia

References

1. Dowden H, Munro J. Trends in clinical success rates and therapeutic focus. Nature reviews Drug discovery. 2019;18(7):495-496. doi:10.1038/d41573-019-00074-z
2. Hafliðadóttir SH, Juhl CB, Nielsen SM, et al. Placebo response and effect in randomized clinical trials: meta-research with focus on contextual effects. Trials 2021 22:1. 2021;22(1):1-15. doi:10.1186/S13063-021-05454-8
3. Pfaar O, Agache I, Bergmann K, et al. Placebo effects in allergen immunotherapy – an EAACI Task Force Position Paper. Allergy. Published online April 23, 2020. doi:10.1111/all.14331
4. Narkus A, Lehnigk U, Haefner D, Klinger R, Pfaar O, Worm M. The placebo effect in allergen-specific immunotherapy trials. Clinical and translational allergy. 2013;3(1):42. doi:10.1186/2045-7022-3-42
5. Tselios K, Wakani L, Gladman DD, Su J, Urowitz MB. Response to placebo in non-renal, non-neuropsychiatric systemic lupus erythematosus: a systematic review and pooled analysis. Rheumatology. Published online 2020. doi:10.1093/rheumatology/keaa655
6. Ali AA, Seng EK, Afsaneh alavi, Lowes M. 15716 The high rate of placebo response in hidradenitis suppurativa clinical trials: Recommendations for future clinical trials. Journal of the American Academy of Dermatology. 2020;83(6):AB46. doi:10.1016/j.jaad.2020.06.270
7. Andreasen TH, Christensen MO, Halling A ‐S., Egeberg A, Thyssen JP. Placebo response in phase 2 and 3 trials of systemic and biological therapies for atopic dermatitis—a systematic review and meta‐analysis. Journal of the European Academy of Dermatology and Venereology. 2020;34(6):1143-1150. doi:10.1111/jdv.16163
8. Imanaka T, Sato I, Tanaka S, Kawakami K. Predictive factors for the placebo effect in clinical trials for dry eye: A pooled analysis of three clinical trials. British Journal of Ophthalmology. 2017;101(11):1471-1474. doi:10.1136/bjophthalmol-2016-309887
9. Zhang W. The powerful placebo effect in osteoarthritis. Clinical and experimental rheumatology. 2019;37(5):118-123.
10. Rief W, Nestoriuc Y, Weiss S, Welzel E, Barsky AJ, Hofmann SG. Meta-analysis of the placebo response in antidepressant trials. Journal of Affective Disorders. 2009;118(1-3):1-8. doi:10.1016/j.jad.2009.01.029
11. Trivedi MH, Rush J. Does a placebo run-in or a placebo treatment cell affect the efficacy of antidepressant medications? Neuropsychopharmacology. 1994;11(1):33-43. doi:10.1038/npp.1994.63
12. Lee S, Walker JR, Jakul L, Sexton K. Does elimination of placebo responders in a placebo run-in increase the treatment effect in randomized clinical trials? A meta-analytic evaluation. Depression and Anxiety. 2004;19(1):10-19. doi:10.1002/da.10134
13. Treister R, Lawal OD, Shecter JD, et al. Accurate pain reporting training diminishes the placebo response: Results from a randomised, double-blind, crossover trial. PLOS ONE. 2018;13(5):e0197844. doi:10.1371/JOURNAL.PONE.0197844
14. FDA Briefing Document Psychopharmacologic Drugs Advisory Committee (PDAC) and Drug Safety and Risk Management (DSaRM) Advisory Committee Meeting.; 2018.
15. Paul D, Sanap G, Shenoy S, Kalyane D, Kalia K, Tekade RK. Artificial intelligence in drug discovery and development. Drug Discovery Today. 2021;26(1):80. doi:10.1016/J.DRUDIS.2020.10.010
16. Branders S, Dananberg J, Clermont F, et al. Predicting the Placebo response in OA to Improve the Precision of the Treatment Effect Estimation. In: Proceedings of the 2021 OARSI Connect, Virtual World Congress on Osteoarthritis. ; 2021:Late Breaking Abstract.
17. Tuttle AH, Tohyama S, Ramsay T, et al. Increasing placebo responses over time in U.S. clinical trials of neuropathic pain. Pain. 2015;156(12):2616-2626. doi:10.1097/j.pain.0000000000000333
18. Increasing Rates of Placebo Response Over Time in Mania Studies | Psychiatrist.com. Accessed May 23, 2021. https://www.psychiatrist.com/jcp/assessment/research-methods-statistics/increasing-rates-placebo-response-mania-studies/
19. Gopalakrishnan M, Zhu H, Farchione TR, et al. The Trend of Increasing Placebo Response and Decreasing Treatment Effect in Schizophrenia Trials Continues. The Journal of Clinical Psychiatry. 2020;81(2):0-0. doi:10.4088/JCP.19r12960