A Pandora's Box for Clinical Trials?

In August 2006 I wrote a column about the exciting potential of Web 2.0, the latest and greatest "philosophy" of Web sites that involves collaboration, shared knowledge, and very flashy design. Two well-known examples are Wikipedia and Facebook, which most people have encountered nowadays. In writing that column, I didn't discuss or consider the potential dark side of this technology.

Paul Bleicher

Recently, I have been exposed to some of the most interesting examples of Web 2.0 in medicine, and I have begun to see an even greater potential for Web 2.0 in our industry, but also a substantial risk for the future of clinical trials and pharmacovigilance, especially in serious disease indications with small numbers of patients.

It occurs to me that the Pandora myth may be a telling metaphor for some aspects of Web 2.0 for the biopharmaceutical industry. In the Pandora myth, after Prometheus steals fire from the gods and gives it to mortal men, Zeus creates the beautiful and seductive first woman, Pandora, to punish men. Pandora is given a jar (erroneously described as a "box") and told not to open it, but curiosity overcomes her and she does—releasing all the evils of mankind. She closes the jar quickly enough to keep Hope inside.

Prometheus brings fire

Web sites and communities designed with a Web 2.0 philosophy are powerful tools for the participants in these communities. Virtually unregulated interaction and collaboration can generate information and knowledge far greater than the sum of the individual's knowledge. This is one of the key messages in the book, Wikinomics, by Don Tapscott and Anthony Williams. A story from this book that dramatically illustrates this point is that of Rob McEwen, chief executive officer of Goldcorp, a gold mining company.

In 1999, Goldcorp owned 55,000 acres in Canada that had been seemingly mined to exhaustion, although there were some hints of data suggesting that gold might still be found in the mines. However, company geologists couldn't agree on whether these leads were significant or worth pursuing. Inspired by a conference at MIT on open source software, McEwen made a bold and astounding move: He created a contest on the Web with cash prizes totaling $575,000 to be awarded to the best suggestions as to where to mine the gold. His company posted all the proprietary geologic data they had on the region on their Web site.

The response to this challenge was beyond McEwen's imagination. Submissions from 52 geologists reaped 110 potential deposits, of which half were previously unknown by the company. Many contestants won prizes from $10,000 to $100,000, and 10 were actually hired by the company. An astounding 80% of the identified deposits yielded a useful amount of gold, and the company has since mined over 8 million ounces of gold (currently in futures markets near $1000 per ounce, retail!). If ever there was a demonstration of the collective power of the Web, this is it. Unleashing power of this magnitude is surely akin to Prometheus bringing fire to earth.

The usefulness of this model for collaboration is being tested in the pharmaceutical industry by Innocentive (www.innocentive.com). In this collaboration marketplace, major pharmaceutical and food companies are posting challenges anonymously for some of their most difficult problems in discovery, medicinal chemistry, manufacturing, assay development, and even clinical trials statistics.

Potential solvers of these problems can register and submit solutions to the challenges, with hopes of winning prizes from $5000 to $1 million for accepted solutions. Participating companies have awarded over $3 million so far for 215 solutions, with solvers coming from India, China, Eastern Europe, and even the United States. Many of the successful solvers aren't even trained in the same field as the problem. Despite the risks of competitive disclosure, companies post challenges because the power of the Web exceeds, by order of magnitudes, the resources they could make available within their own companies.

Zeus creates Pandora

The power of Web 2.0 collaboration may extend into the clinical development and marketing phases of the drug lifecycle, where its demonstrable benefits may potentially be overshadowed by some major risks. The risks to which I am referring are related to the open, collaborative discussion of drug efficacy and safety in communities of physicians and patients. As I will show, this might even extend to a quasi-analytical analysis of data that may be highly influential. A Web 2.0 community with huge potential benefits and risks is PatientsLikeMe, and will be the focus of our discussion.

PatientsLikeMe (www.patientslikeme.com) is a beautifully constructed social networking Web site that allows patient communities to form around the diseases they jointly suffer from. Unlike many bulletin board attempts at this concept, PatientsLikeMe provides patients with diseases such as ALS, MS, Parkinson's, and HIV/AIDS to create a publicly viewable (in most cases), structured Web site with their photo, a narrative, and a longitudinal collection of the most important objective and subjective variables in following the course of the particular disease.

For example, a typical MS patient's site may show a longitudinal graphic of their disability score with relapses indicated, and a patient profile-like graphic of their symptoms (with severity) and treatments over time. The visual display is nothing short of stunning, and the data are highly organized and accessible.

In addition to individual data, the site aggregates data so that an MS patient with anxiety can discover, with one click, a pie chart of its severity in thousands of patients with anxiety and a list of dozens of medications prescribed for anxiety. In addition, they can link to the profile of other patients with anxiety and search for discussions on anxiety on the forum.

PatientsLikeMe is not alone. Similar sites exist for diabetes, depression, alcoholism, and other health issues. If one of my relatives or I had MS or any other serious disease, the information on such sites would be powerful and compelling. These sites are, in their construction and attractiveness, quite beautiful and seductive—just like Pandora.

Pandora opens her jar

The nature of openly available, patient-reported data found on these sites is disconcerting in some aspects. The first, and most immediate concern, is that such patient-reported information contains a potentially enormous number of reported adverse events. Will regulatory agencies hold companies responsible for identifying and reporting such data as they must for published reports? If that is the case, will they have to pay each Web site for the aggregated data and manually sift through narratives to identify issues? These are not new ideas and are being actively considered on both sides of the regulatory divide.

Another concern relates more to the synergistic power of collaboration in this setting. The power of suggestion and confirmation bias will surely lead patients to cluster in their reporting of symptoms. The aggregated statistics on symptoms and medications are presented with scientific graphing techniques, suggesting an accuracy and quality that may not be inherent in the source of the data. A patient reading these graphics who discovers that most other patients have a particular subjective symptom or side effect from a medication may over report such a symptom. If an influential member or two from the community start on a new medication and report a particular adverse effect, the occurrence of that adverse effect may be magnified within the community. With diseases like ALS, where a substantial percentage of the patient community may be participants in such a Web site, the effect of this could be unfairly damaging to a new, potentially effective medication.

Finally, there has recently been a very interesting development at PatientsLikeMe that holds amazing opportunity but increases potential risks. On a public blog at the site (blog.patientslikeme.com/), one of the founders discusses a recent paper suggesting a benefit for lithium in slowing the progression of ALS. Rather than wait for the slow process of a follow-up clinical trial, PatientsLikeMe started "the first real-time, real-world, open and nonblinded, patient-driven trial" now with 117 patients involved. These patients have started lithium with their physician and are tracking their progress on the Web site.

PatientsLikeMe believes that through their data modeling techniques, they can "determine the significance of each reported change in each patient as he/she deviates from his/her predicted course." The results will likely be reported on the Web site with the same quality of graphics and display found elsewhere on the site and will appear to be a critical look at this question with eight times as many patients as were enrolled in the originally reported clinical trial.

Aside from data quality and placebo effect issues (which they do take seriously), the bias from openly available data is enormous, and the potential for incorrectly concluding that a possibly ineffective and toxic drug is effective is very high. In addition, these activities could potentially starve active clinical trials from recruiting the needed patients to complete their blinded analysis of the same problem.

Another concern will emerge as participants in PatientsLikeMe begin to enter conventional clinical trials. There open reporting of their experiences with study medications, along with any adverse experiences they encounter, would likely be available individually and possibly in aggregate on the site. Again, in tightly affiliated patient communities, this could lead to bias and even unblinding of conventional clinical trials.

While these concerns aren't anywhere near the evils of Pandora's jar, they certainly have the potential to create some major headaches in the halls of biopharmaceutical companies in the foreseeable future.

To their credit, the company has been proactive in engaging pharmaceutical companies about some of these concerns.

Hope Remains

As in the Pandora story, there is certainly reason to be hopeful about the value of communities like PatientsLikeMe. As they are compelling for patients, they may become a major center of gravity for some disease communities, especially if they align with officially recognized patient advocacy groups.

In the Web 1.0 world, we might have thought of these as sources for improving patient recruitment in clinical trials—in the past, a model that has been less than stellar in accomplishing this goal. Instead, we should think of them as potential registries of patients, which are possibly available for improving our understanding of the biopsychosocial aspects of disease, and disease therapies. They might be used to generate hypotheses, follow patients in naturalistic settings, and even test some ideas in a more structured fashion.

Technology is here to stay, and it will bring changes in communication and interactivity that we can't even anticipate, along with many opportunities and some risk. Much as we would like to, we can't stop Pandora from opening that jar; we can only try to maximize our benefit from the hope that remains.

Paul Bleicher MD, PhD, is the founder and chairman of Phase Forward, 880 Winter Street, Waltham, MA 02451, (888) 703-1122, paul.bleicher@phaseforward.comwww.phaseforward.com. He is a member of the Applied Clinical Trials Editorial Advisory Board.