Readers respond to the NEJM article mentioned in November's From the Editor
My initial reaction to the NEJM piece was “everyone should read this.” I still feel that way, but for slightly different reasons. As the ACT editor acknowledges, some of the issues cited such as CRO staff turnover, continuity and problems with the billable hours model have been known for some time-but rather than dismiss that as “old news” it should cause us to question what that means practically: inessence, as the NEJM author acknowledges, it causes someproblems, but sponsors have found ways to deal with it.
The much bigger issue here is accountability. When asked to suggest a topic for an Outsourcing conference recently, I submitted “The TeGenero Disaster: Whose Problem Is It?” Normally one would say, “well the sponsor of course, they approved the trial,” but the sponsor went bankrupt as a direct result of the trial and now the CRO is being sued. Should they be?
If the CRO concerned (or any CRO) holds themselves out to be experts in a field, shouldn't they be held directly accountable to various agencies as experts, rather than just the sponsor
Are the instances of other problems cited isolated or indicative of pervasive issues? If they're the same as you could probably find in many instances with various sponsors, then why shouldn't CROs face the same degree of scrutiny and accountability as sponsors? When will the FDA step up, clarify, and lead?
Monitor turnover and re-assignment is a significant issue leading to at best poor relationships with sites that haveto constantly change practices and documentation to suit the individual preferences of monitors to questionablequality of data due to inconsistant advice and poor protocol knowledge on the part of the front line staff.
The NEJM article was clearly written to negatively portray the CRO industry. There was no mention of the auditfunction where sponsors should be monitoring CROs from a variety of services. Audits should focus on staff qualifications, facilities, process, and data. Unfortunately, the world is not a perfect place and these types of problems will always exist.
The general theme of the NEJM article appears to be that increased reliance on outsourcing to CROs is having an adverse impact on clinical drug development trials. I do not believe that the author has presented consistent evidence to support this thesis. Are CROs displacing academia in drug development trials?
The author contends that “contract research organizations (CROs) have gradually taken over much of academia’s traditional role in drug development.” This is not true. Academia’s traditional role in drug development has been to provide a source of expert investigators and study sites. CROs, on the other hand, monitor study sites, and, in some cases, also provide project management and data analysis services. These activities were traditionally performed by pharmaceutical industry sponsors-not academic investigators. A more accurate statement would be that CROs have gradually taken over some aspects of the pharmaceutical industry sponsor’s traditional role in drug development.
There is, however, evidence that increased reliance on CROs has reduced the number of trials contracted to academic investigators. The reason for that is simple. In order to ensure speed and efficiency, the CRO must identify study sites that will have access to the required patients and be expected to perform well. The CRO may propose study sites but the final choice is up to the pharmaceutical industry sponsor. The majority of “garden variety” patients are seen in private practice and non-academic clinics and hospitals, not in academic settings which focus on the atypical patient. Consequently, for many projects CROs would avoid academic sites, because these sites would not be expected to have access to certain types of patients.
Are clinical drug development projects impaired by reliance on CROs?
The author describes some unfortunate events at Phase I study sites which were owned by the CROs SFBC and Parexel. Dramatic as they may be, these events have little relevance to the discussion of whether CROs or academia would do a better job of conducting clinical trials. Academia has generally not been involved in Phase I trials. These trials are usually conducted using healthy volunteer subjects at Phase I sites that are specifically designed and staffed for this purpose. Some of these sites are owned by large CROs; others are not.
Two additional cases cited by the author actually argue in favor of using CROs. In both cases the author claims that the sponsors, Aventis and Bayer, failed to report problems which were ultimately disclosed by the CROs, PPD and i3 Drug Safety. These cases demonstrate how the CRO can provide an additional check on safety and efficacy evaluations of new compounds.
Is the quality of CRO services improving?
Surveys over the past decade document that the quality of CRO services is steadily improving. In 20051 Ken Getz and I reported a survey of 337 pharmaceutical industry respondents in which improved performance was noted by 39% of the respondents, while only 13% indicated that overall CRO performance had worsened. We interpreted these results as suggesting an improvement in the perception of CRO services, as compared with a 1998 survey of CenterWatch 2 in which 40% of respondents reported that overall CRO performance had worsened.
Could academia take the place of CROs?
The author suggests that the academic research organization (ARO) model might be better suited to performing the duties of a CRO. AROs have been in existence for nearly a decade. In 20003 I observed that, in response to pharma’s criticism that academic sites were too bureaucratic, too slow and too costly, several universities created central offices (referred to as AROs or AMCs) to facilitate grant and contract negotiation with sponsors and CROs. Academia’s interest in this model has waxed and waned largely as a function of the availability of federal grant monies to support the more desirable non-pharmaceutical clinical research projects. A common complaint of the ARO model is that AROs are staffed largely by faculty members who understandably have a greater interest in teaching and conducting original research than performing routine drug development trials. In any case, it is illogical to suggest that responsibility for monitoring the academic site's performance (the primary role of CROs) be given to the academic institution itself.
Can sponsors, CROs, academia, and regulatory authorities work more effectively together?
Obviously, the answer is “yes.” Sponsors should establish win-win relationships with CROs by having more realistic expectations, establishing and using performance metrics for themselves and their providers, and paying a fair price for deliverables. CROs should be more selective about the projects that they accept, they should establish and adhere to transparent quality standards, and they should place more emphasis on employee training and oversight. Academia can serve an important role helping pharmaceutical industry sponsors design and interpret clinical trials and provide access to atypical patients. Regulatory authorities should recognize that the CRO represents a fourth player in the traditional sponsor-investigator-patient triad. While the pharmaceutical sponsor understandably bears responsibility for the quality of the data and the protection of study subjects, there is a practical limit to how much oversight of the CRO that the sponsor can maintain.
References1. Vogel, J.R. and Getz, K.A., "Successful Outsourcing: Tracking the Evolving Use of Full-Service and Niche-Service CROs,” Applied Clinical Trials, 14 (6), 54-60, 2005.2. Whitaker, R. The Evolving CRO Market, CenterWatch Monthly, 1998, Volume 5, Issue 9, page 5.3. Vogel, J.R., “Successful Outsourcing: Understanding and Evaluating Outsourcing Alternatives,” Applied Clinical Trials, 9 (4), 60-64, 2000.
I was also surprised by the article at NEJM and distributed to my colleagues as soon as it appeared. I agree with thearticle in that CRO quality is globally poor, at least in our specialty (oncology).
From where I sit, the recent article in the New England Journal of Medicine is both harmful and helpful. Harmful in that it is largely a collection of assertions supported by limited circumstantial facts and spurious observations with neither context nor balance. Helpful in that the article sheds more light on the misperceptions of industry-funded clinical research and contract research organizations (CRO). These misperceptions are not only alive and well but also growing-particularly within academia and government.
As an independent observer of the clinical research enterprise for more than 20 years, I am not aware of any data that shows that the quality of drug development suffers when a CRO is involved. A recent study that I conducted at the Tufts Center for the Study of Drug Development (TCSDD) found that high CRO usage was associated with faster cycle time with no difference in quality. In this study, quality was defined as the number of errors per case report form and the number of 483s filed against investigative sites managed by CROs when compared to those managed by the sponsor directly.
There is no evidence to support the false notion that CRO employees represent a “de-skilling” of the clinical research labor force. My research at TCSDD shows that as sponsor headcount growth has flattened during the past five years, CRO headcount growth has been rising 7% annually with disproportionately faster growth in positions requiring more highly skilled clinical research professionals. Indeed, the CRO industry is filling many of its open positions with professionals who had been trained and employed by pharmaceutical and biotechnology companies.
The statement that delineating clinical research into manageable steps represents a “commodification of research projects that have begun to kill clinical research” is simply absurd and naive. All clinical research professionals-those involved in government and industry funded programs-manage their studies according to standard processes and procedures and use case report forms to capture study data. My research has shown that the quality and performance of clinical trials improves with higher levels of upfront and ongoing project management.
There is no question that there are major scientific, operating and ethical challenges facing the clinical research enterprise. As a broad enterprise, these challenges must be addressed collectively. By focusing solely on CROs and the drug development industry, the author neglected to mention and contrast questionable practices seen in government-funded clinical research. A growing body of evidence, for example, shows high levels of conflicts-of-interest among NIH researchers and institutional IRB members. The rate of non-compliance and fraud among NIH-funded clinical researchers has been rising steadily. Unlike the Food and Drug Administration, however, the Department of Health and Human Services’ audit and inspection data is far less transparent. My own research comparing NIH-funded and industry-funded investigators has shown that the former are typically slower, with poorer quality and higher relative cost.
One is left with the impression that the October 4, 2007 Perspective is yet another example of the NEJM’s relaxed editorial standards. And the tendency is to simply ignore the article’s assertions and misrepresentations. I believe that a failure to respond is dangerous. I applaud the efforts of Applied Clinical Trials to open a dialogue and I encourage clinical research professionals to seek out opportunities to educate and inform professionals in the health care provider and government-funded clinical research communities. The NEJM article, therefore, serves as a valuable blueprint for identifying key targets for outreach and education.
I came to the CRO industry after 15 years at the world’s largest pharmaceutical company...and after 15 years of seeing the CROs do it better, faster, and with greater attention to detail. Since joining the “Dark Side” I’velearned the majority of CRO employees are at least as well-trained as their big pharma counterparts and far moreattuned to accurancy, ethics, and subject safety. Let us not forget, however, the drug industry is a market driven business; it isn’t a charity or a non-profit. The sniping at CROs smacks of hypocrisy and jealousy. After all, we have to pay to read the NEJM or see a doctor.
CROs pay lower salaries, which result in higher turnover. The higher turnover brings the CRO to hire young people with low skills and without providing enough and continuative trainings. Hours spent in training are hours not used to follow the projects and get money from the sponsors.
Accountability should be addressed in the “Transfer of Regulatory Obligations” (TORO) and submitted to FDA. Any sponsor transferred regulatory obligations to the CRO arethe responsibility of the CRO, hence,the CRO answers to FDA for those transferred responsibilities.
Perhaps money should not be the driving force behind drug development. But an argument could be made that speed to market is a driving factor that is legitimized by patient safety and a desire for improved health care. Academia did not fulfill their role in improving the efficiency or speed of pharmaceutical research. Drug development is a difficult and imperfect endeavor. I would prefer to see Dr. Shuchman recommend a better paradigm than simply criticize the existing model. She took the easy way out.
Regarding device clinical research. CROs are not the manufacturer of the device nor the physician using the device. CROs are not unbiased, they have their own inherent issues that can negatively affect the research. CROs have not been hired to perform the research at my company, they have been hired to assist with monitoring/ CRA duties. Even with oversight it is apparent that they get obsessed with minor issues that have no affect on the research and are not able to provide good unbiased reviews. It would be more beneficial if the CROs developed an attitude of working with the manufacturer and physician rather than being so concerned with their own needs.
I echo the sentiment that CROs have less experienced staff and higher turnover, but this is at equal to staff turnover and quality of work that I have seen at various academic institutions. Many thought leaders/investigators and scientists at Academic institutions are overwhelmed and not actually providing oversight to the clinical trials they commit to. It is the sponsor that takes ultimate responsibility for the conduct of the trial to provide adequate oversight and develop a partnership with all partners in drug development. CROs provide essential services that require trained resources to clinical trials that sponsors in many cases don’t have and academic institutions or collaborative groups do not provide.
I have worked with SMOs sponsored by academia and have not found them to be superior to private CROs. Clinical research is always difficult, no matter who performs it and we all need to improve our processes and subject protection efforts. Currently, jobs at major pharma are being outsourced to CROs. There is a back and forth with employees between pharma and CROs with the same employees working at each depending on the pharmaceutical fad in fashion at the time. Global changes in CROs and pharma may even lead to an increase in the educational quality of employees working in the clinical trial field (although I will regret losing my income to an MD in India).
ACT correctly responds to this article, and should be commended in its restraint. Upfront, the NEJM author (with no clinical trials on PubMed) confuses "revenue" and "worth." The errors continue from there. CROs are regulated by GLPs, the listing on the 1572, IRB, sponsor, and frequent random audits by FDA. There’s a huge turnover problem at academic sites. So what if (a big if) CRO employees earn less? Try getting a study through a grants office that wants to own all "discoveries" the site makes. If they follow the protocol, how can they make discoveries? Junk CROs should be uncovered and put out of business. But the job of the CRO is to conduct trials ethically, that of the academic is to care for patients. Let's both do the job we are best at.
Having spent years both at sponsors and CROs I can appreciate both perspectives. If indeed there is erosion of quality in clinical research associated with CROs increased involvement, sponsors must share the blame. I have seen many cases where clinical trial budgets were negotiated like used car purchases with the pressure on the CRO to "find a way" to get it done for a reduced amount. If quality is to be preserved CROs and sponsors are going to have to work collaboratively to agree on (and fund) the elements needed to ensure quality.
Communication is a huge issue between sponsors and CROs. The FDA should make clear and definitive recommendation regarding who is responsible for what and force those in charge to take steps to ensure quality and patient safety not just meet timelines.
This is a complex topic that should be examined in more detail. More research will be needed to determine if the problems noted in the NEJM article are indeed correct, and if so, to what degree and in what types of CROs they exist. Understanding the business models of all stakeholders can provide insight into how and why the companies involved do business together. This information will help to shed light on any problems that are found to exist. Open dialogue is needed to continually identify problems and improve our practices.
I thought this was an excellent article. I've worked for both sponsors and CROs over the last 7 years. Some companies may have rushed into outsourcing and "jumped on the band wagon" without thinking about the long-term costs, risks and possible consequences for their specific company. Some outsourcing can be a good thing when vendors are carefully and cautiously chosen. I've seen some outsourced projects eventually end up costing more than inhouse projects. Ultimately companies need to choose the most cost effective ways to get the work done, but whether they can actually know these ways in advance is doubtful. The clinical trials industry keeps changing so quickly.
I read Dr. Shuchman’s article regarding the risks and benefits of the CRO boom with great interest. In full disclosure, I own a mid-sized CRO specialized in medical device and biotech research and am always interested in improving the services and standards for our company. My initial observation of Dr. Shuchman’s publication is that this article is focused almost solely upon what she considers risks and the only “benefits” identified are wrapped in disqualifiers. It is apparent to me that Dr. Shuchman is a proud advocate of academic medical research. I would actually agree with this assessment. Many of the most interesting and innovative research technologies and treatment alternatives come from this arena. It is abundantly apparent also, however, that Dr. Shuchman has an extremely shallow understanding of the CRO business and the commitments and efforts that most CROs make to insure the protection of research subjects and excellence in clinical research practices.
The title of the article characterizing the use of a contract research organization as “commercialization” is an excellent example of a basic misinterpretation of the role of a CRO in the research process. The use of a CRO is a staffing decision and in and of itself, in no way “commercializes” the investigational process. The CRO is an extension of the sponsor. Depending upon the infrastructure and resources available within the sponsor environment, the actual responsibilities that the CRO assumes vary. The mere fact that certain activities are outsourced does not demean their quality. In fact, I routinely “outsource” the filing of my income taxes and my medical care to experts who are much better qualified to do these activities than I am. I suspect my physicians would be quite offended if I referred to the excellent care they provide to me as the “commercialization” of medicine.
Dr. Shuchman also makes several negative comments regarding the quality of the work CROs perform and the qualifications of CRO employees. It is a certainty that there are CROs that do not hold high enough standards for trial execution and personnel training and development, but this is true as well for academic medical centers and sponsors. This is clearly evidenced by the number of AMCs whose research activities have been shut down by Food and Drug in recent years and the number of warning letters on file for both them and sponsors of clinical trials. When a service vendor is selected, it is incumbent upon the sponsor or AMC client to investigate the qualifications of this service provider. The size of an organization is a very poor indicator of its quality. Quality is driven by current and enforced quality standards and ongoing quality assessment. There are many excellent CROs with standards that are, in fact, much higher than those of their clients. A current trend in industry, however, has been to move the decision-making regarding CRO selection out of the hands of the clinical experts and into the hands of a sourcing officer. This sourcing officer rarely possesses the necessary acumen and skills to adequately assess the “fit” and qualifications of a CRO and may, as a result, chose the vendor based solely upon size or price. If there is a problem with the quality of CRO services, this is quite likely a primary source of it.
The assertions that Dr. Shuchman makes regarding the lack of experience CRO employees possess is clearly a biased and uninformed position. There are many CROs that are quite selective and have, again, higher performance standards and expectations than either academic medical organizations or sponsor companies. For example, my first job in clinical research was not with a CRO, it was in an academic medical setting. As a result, the period of time in my career when I was the lowest paid and the least experienced was when I was working for the type of institution that Dr. Shuchman heralds. While I will hire relatively inexperienced personnel from time to time, this is made possible only by the fact that we provide excellent training and development for our employees. We, in fact, have a specific career ladder developed for clinical research professionals that includes position specific training, co-monitoring, mentoring, a requirement that they seek certification, and a comprehensive employee development plan. I have rarely worked with a sponsor company offering as extensive a program as we have and as a result, we are often hired by our clients and research institutions to train and develop their own employees.
There are several statements in Dr. Shuchman’s article regarding the “commodification” of CRO services. She discusses the fact that CROs break the clinical trial schedule into steps. On this, she is absolutely correct. In fact, if she were to review the FDA’s Guidelines for the Monitoring of Clinical Investigations, she would note that the agency too breaks the process down into discreet steps. This is not the reckless practice that Shuchman refers to, but is instead, good project management. Recognizing that there are steps that must be followed and a natural order of things in the execution of a clinical trial is a necessary part of good clinical practice. Failure to manage the trial to appropriate time lines runs the risk of unduly prolonging a trial, thereby increasing the probability of protocol deviations, inability to adequately follow research subjects, and investigator and coordinator turn-over.
The statements regarding a CRO’s role in communication with the FDA and in payments to research subjects clearly indicate that Dr. Shuchman fails to understand the basic nature of the relationship between the various parties engaged in the research process. The CRO, by definition, is contracted to the sponsor of the clinical trial. This mandates a fiduciary obligation on the part of the CRO to the sponsor to inform them of any irregularities or problems incurred during the conduct of the trial and to try to proactively manage them. While a CRO can be audited by the agency, the primary relationship here is not between the CRO and the FDA, but between the CRO and the sponsor. In the example of SFBC International that Dr. Shuchman uses, she accuses the CRO of manipulating payment to research subjects. Payment amounts and schedules for research subjects, however, are outlined in the informed consent document. Although a CRO often has the opportunity to give input to this issue, it is a core responsibility of the sponsor and the ethics committee to make the final decision in this matter.
The world of clinical research is complex by nature. Balancing the needs and resources of a trial sponsor with those of the larger medical and patient community is not an easy endeavor. The contract research organization offers a clinical trial sponsor access to personnel who work across multiple therapeutic specialties and with multiple research centers and manufacturers. The insight into best practice that is gained from this experience is unique to the CRO industry and offers the research sponsor an invaluable opportunity to improve their internal processes and move the overall quality of research practiced to a higher level. An excellent CRO, and there are many, offers the sponsor of a clinical trial not only a specific set of services, but through consultation and implementation of new practices, can significantly improve the quality and standards of research for all entities involved.
As a sponsor, I generally agree that the quality of work from CROs has decreased over the last decade, clearly a result of waning experience in the CRO ranks. I believe this is directly attributable to the boom in biotech companies. As larger pharma companies consolidate, more start-up biotechs emerge providing bountiful opportunities for experienced professionals. Naturally, this draws talent from the CROs who cannot compete with the compensation packages provided by the start-ups.
It should be kept in mind that the transition to CROs came about in part as a result of frustration with academic centers of impeccable reputation and vast brainpower, who couldn't follow an SOP or a schedule to save their lives. Try convincing someone from Harvard that data QC is more important than the seminar they're about to give. Thanks for running this editorial. I'd love to hear your thoughts on the other great insult to our industry: The restriction by JAMA that manuscripts from industry must be reviewed by an independent academic statistician.
Pros work in the CROs, but graduate students work in the academia during the transition between graduate and the real job. Academic certifications are only to start working in the industry. CROs' resources come from pharma with experience CROs' growth says, Pharma realize, CROs are better. A lab assistant makes a much higher than a graduate student. Students expect certifications, not $ from the acad., non-profit organization in the capitalistic economy. Average income for US CEOs is about $3,600,0000 but the average income for the head of academia in the US is not comparable. It's not practical to expect a quality outcome from a graduate student, temporary employee. Academia can assist pharma to some extent, but a CRO is a better place for the industry based on Ricardo's "Comparative Advantage." (edited by Applied Clinical Trials for readability)
Having spent 15 years in big pharma in clinical research and then 12 years in major CROs as a global project manager for big pharma clients--there is a lot of truth in this article. Big pharma want to save dollars and CROs are pushed to do so. From the scientific side big pharma was (I am not sure that is still the case) more dedicated to the scientific goal. CROs are dedicated to being on time on budget and to customer satisfaction--I am not convinced this leads to good scientific results in the long run.
Three observations: 1. The source of the claim that CRO staff are less educated, experienced, etc. is not factually-based, but subjective observation in an NBER working paper. This is a claim that begs substantiation. 2. In some of the examples the author cited, the CRO (i3 Research) actually worked to force the sponsor to be more ethical. 3. The author references Academic ACROs as a viable option to improve quality, without any insight to the many glaring reasons why these organizations have limited utility: operational structure, speed, capacity, globalization, legal, etc. They retrofitted this business model onto existing academic organizations.
Currently I am working for a sponsor company, but I have more than 10 years working experience in CROs, including a leading CRO mentioned in the article. I don't agree with the claims that CRO workforce is less skilled, less experienced. The CROs tends to keep some very experienced people and pay them well. They are more cost sensitive and try to fit the right level of people to right positions. From my observations, CROs definitely run trials more efficiently and are better organized, that's the way they earn money. Their philosophy is, do it right, from the beginning. As CRO has a lot of experience on different trials, most of the time they know better than sponsors, especially medium or small size sponsors, on how to run the trials.
Based on my experience in working with CROs, I believe many of the claims in the article are true to some extent, and I agree CRO model is not serving the best interest to the public in drug development.
I don't think that CROs people are less experienced than industry andacademic, but I definitely think that sponsors delegate too much theirresponsibility to CROs. If the sponsor doesn't take care of his own study,how can they blame CROs doing the same? On the other side, CROsadvertisement state that the sponsor should not worry anymore about thetrial, and this is definitely misleading. Finally, CRO is a tool, if you canuse it properly, it works, if you can't, don't blame the tool, blameyourself.
The well documented article by Miriam Shuchman (N.Eng.J.Med. 2007;357:1365-68) is warmly welcome: indeed it is noteworthy that the CRO boom, which completely changed the global scenario of clinical trails in less than 20 years, received so little attention by the scientific community.CROs are relatively new organizations. They are greatly dependent on the visions of their Boards: in my opinion it is difficult to make general statements about them. I’d rather prefer to make reference to my personal experience.
In my 32 years long experience in the drug industry, I had the opportunity to spend 2 years as VP Regional Operations Europe at ICON: here are my observations.
1-Commercial pressure=CROs are profit organizations, so their main interest is to attract business and to generate profits for the benefit of the shareholders. However the commercial pressure may be so strong to bypass the technical capabilities of the organization: sponsors are completely unaware of the human and know-how limitations to perform their trials.
2-In a continuous pressure to save costs, training is kept in house, and very few CRO staff are allowed to attend external conferences and courses. In house training has the disadvantage of avoiding confrontation with the real world. In addition, most training is arranged via global teleconferences, without any verification of the results.
3-The save money approach results also in a large proportion of home based staff, not only CRAs but also PMs. The quality of these remote staff is seldom verified, and frequently sponsors are not informed of the physical locations of the staff assigned to their projects.
Finally, Miriam Shuchman reports that CROs in 2003 played a substantial role in 64% of pre-NDA trials: it is likely that this proportion is today close to 80%. However, in a meeting organized in London by IFAPP, Fergus Sweeney (Head of EMEA GCP Inspectors) disclosed that EMEA inspections were addressed to CROs in only 3% of cases.
In a very aggressive and competitive environment, CROs have a clear vision of their mission: they are looking for their own growth and commercial success. Sponsors should exercise a more frequent and rigid control to better understand what they buy at an expensive cost. Regulators should better address their inspections to reflect the changed scenario of global clinical trials.
The ambiguity of the regulatory environment in which the CROs operate is concerning. It is about time that the CRO comes in to public sphere.