Real World Psychiatric Data Contradict Clinical Trial Results: A Data Analysis on Wellbutrin

While there has been debate regarding the validity of psychiatric randomized controlled trial data and their effectiveness in real world clinical settings, randomized controlled trials set the gold standard in psychiatric drug development. Nevertheless, little research has been done on the impact of psychiatric treatment in real world clinical settings. The emergence of post-marketing psychiatric databases that collect patient outcomes in clinical and real-world settings are generating patient outcomes that challenge psychiatric clinical trial data validity. This article will evaluate some concerns regarding psychiatric clinical trial design, and will analyze post-marketing patient reported outcomes data on the antidepressant, Bupropion HCL (Wellbutrin), from Treatment Online, an online psychiatric treatment platform.

Concerns with Psychiatric Randomized Clinical Trial Design

Albeit randomized clinical trials are the gold standard, skeptics have raised concerns regarding the process, specifically, (1) low levels of certainty in psychiatric trials are not likely to influence practice, (2) psychiatric clinical trial sample sizes tend to be lower than clinical trials in other disease indications, (3) concerns with placebos and heterogeneity question validity, and (4) outcomes measures design do not appropriately reflect clinical settings.¹

Firstly, due to the flexible nature of treatment options in psychiatric clinical settings, leveraging psychiatric clinical trial data in a controlled setting can be challenging; for example, evaluating the effectiveness of a diabetic drug on HbA1c levels offers a high level of predictability and measurability compared to an antidepressant that allows patients to feel less depressed.^1,2 Secondly, psychiatric clinical trial sample sizes tend to be lower than other trials; to demonstrate, the literature suggests that a typical psychiatric clinical trial enrolls 20-100 subjects. Moreover, larger clinical trials (such as the Wellbutrin trial that we will analyze in this article, which allocated 323 patients on Wellbutrin and 185 on placebo³) are still considered modest compared to clinical studies in other diseases.¹

Thirdly, it is difficult to analyze moods (i.e., depression), and uncover whether improvement in psychiatric trials is a result of intervention, the drug, or placebo. Furthermore, placebo arms may contain patients that exhibit less chronic, less disabling, and less severe diseases.¹ Lastly, many psychiatric clinical trials evaluate patient outcomes over the course of weeks, whereas most psychiatrists are interested in long-term patient outcomes.¹

Scholars recommend that large trials in real clinical settings with simple outcomes measures, few exclusion criteria, and long-term follow-up are needed in order to evaluate the validity of psychiatric clinical trial data and the true impact of psychotropic drugs on patients.^1,2

The Treatment Online Study on Wellbutrin

Treatment Online is an online psychiatric platform that engages patients and clinicians. The platform is capable of collecting patient reported outcomes through validated measures including Beck scales, mood measurement, drug side effect reporting, and socioeconomic behaviors for a variety of psychiatric indications. Over the course of five years, Treatment Online ran a pilot on and collected data from nearly 2,000 patients in New York City.

In this case study, we evaluated patient reported outcomes on specific side effects in clinical settings for Wellbutrin, and we compared those side effects with Wellbutrin’s clinical trial results. We identified 382 patients who have reported currently taking Wellbutrin, and 53 of the patients reported side effects on Treatment Online. Of those 53 patients, 32 reported feeling depressed, 6 reported not feeling depressed, and 13 to not provide a response on whether they were feeling depressed. Treatment Online collected a variety of side effects through surveying these patients, as delineated in Table 1. Due to the low amount of patients in the Treatment Online cohort, we used statistical techniques to determine the 95% confidence interval range for each side effect.

Table 1 demonstrates that Treatment Online side effect ranges for dry mouth, dizziness, and headache fit within Wellbutrin clinical trial expected side effect rates, whereas, palpitations, decreased libido, and delayed orgasm/retarded ejaculation ranges did not overlap with clinical trial rates. It is important to emphasize that the Treatment Online study was conducted on a population in one region, whereas the Wellbutrin trial was conducted in numerous regions.

Academicians’ Perspective of Clinical Settings Databases

With the emergence of patient reported outcomes databases comes a high level of uncertainty with reporting objectivity, and associated data lack validity due to uncontrolled settings, unlike in randomized controlled clinical trials, where the data is more valid.  “If you gather enough data (post-marketing patient reported data), it would begin to mimic the ideal world clinical trial, but, I don’t think the numbers (data volumes) are there yet,” said John Hixson, MD, Assistant Professor from the Department of Neurology, UCSF School of Medicine, at ePatient Connections. “I believe that if patient communities and online sources started to reveal evidence that contradicted a trial, I think the academic community would be skeptical,” added Hixson. 

Patient databases, nonetheless, offer insights that can invoke further research and investigation in clinical settings. “These types of enterprises are fantastic for hypothesis generation… this is exactly where the questions for the next research endeavors are going to come from and if the numbers get big enough, I do think that we could use those data points,” said Hixson.  Additionally, academics believe that online communities have the potential to contradict clinical trial data in the future. “If the numbers get bigger, I would believe the contradictory evidence,” added Hixson.

Post Marketing Databases and Online Communities Offer Opportunity for Research

The emergence of post marketing patient databases and online communities offer an opportunity to evaluate real world clinical settings with real world interventions. While the case study in the Treatment Online pilot contains gaps, the data introduces the use of psychiatric medical products in real-world settings, and invokes further research into existing medical product effectiveness. For biopharmaceutical enterprises, post-marketing databases and online communities can offer insight into competitive medical products, and allow clinical teams to uncover opportunities for growth in specific psychiatric indications. Alternatively, these databases and communities introduce risk, as the FDA may evaluate data from the sources and, correspondingly, initiate further investigations and impose psychotropic drug indication restrictions.

References:

* Assumes less than 1.00% incidence

[1] Roger T. Mulder, Chris Frampton, Peter R. Joyce, Richard Porter.  Randomized Controlled Clinical Trials in Psychiatry. Part II: Their Relationship to Clinical Practice, Australian and New Zealand Journal of Psychiatry, 2003

[2] Linsey McGoey, Profitable Failure: Antidepressant Drugs and the Triumph of Flawed Experiments, History of the Human Sciences, Vol. 23, No.1, pp.58 – 78, 2010

[3] https://www.gsksource.com/gskprm/htdocs/documents/WELLBUTRIN-TABLETS-PI-MG.PDF