Key Factors in CRO Selection

Overall, sponsor companies and CROs have a fairly consistent view of the criteria used to select CROs, although there are some notable differences. The pharmaceutical industry makes widespread use of CROs and, most likely, this trend will increase in the coming years. But little empirical research exists on why and how pharmaceutical companies select CROs.

The University of the Sciences in Philadelphia (USP) and TTC, a drug development data company, are conducting a multiyear research program on the factors associated with improved speed and cost effectiveness in clinical studies. Since outsourcing to CROs represents an integral part in developing drugs, USP and TTC will also analyze how CROs are selected and used in clinical trials. Part of this CRO analysis involved a Web-based survey of sponsor pharmaceutical companies and CROs to understand the following:

• Current and expected use of outsourcing

• Specific outsourcing processes and techniques, such as project budgeting, CRO selection, managing studies, bid grids, and managing cost overruns

• Perceptions of how CROs and sponsor companies interact in the areas of business development, CRO selection, and project management

• Training needs for professionals to improve CRO selection and CRO project oversight.

Previous studies have reported that industry professionals involved in outsourced clinical studies expect the overall level of outsourcing to continue to grow. In addition, quality and timelines of a CRO's work dominate as the basic selection criteria.^1,2 This article accepts that previous work quality is a central criterion for selection and focuses on other issues that might be involved in the CRO selection. Set within the anticipated continued growth of CRO usage, the article examines these additional key criteria that individuals working in drug development use to select CROs, highlighting the differences between the selection criteria that sponsor companies emphasize and how CROs think sponsor companies decide.

Using factor analysis, we also determined that the individual elements used to select CROs can be summarized within a set of five broader dimensions: Processes for Issue Identification and Resolution; the Specifics of Project Execution; Team Chemistry; CRO Experience; and CRO Capabilities.

Outsourcing up close

Pharmaceutical company participants indicate that approximately two thirds (64%) of all post Phase I studies involve the use of CROs for one or more parts of the study. However, when measured by pharmaceutical company size, there are very large differences in current outsourcing practices. For example, within the largest 20 pharmaceutical companies, half (52%) of all post Phase I studies use CROs, a number that jumps to 88% for smaller companies. These smaller pharmaceutical companies often do not have the permanent in-house staff to conduct all aspects of a clinical trial, especially where the trial involves many sites and multiple countries. The smaller companies may, for any number of reasons, also lack the employee headcount or expertise to carry out clinical trials.

While many industry professionals expect the use of outsourcing to increase [see Table 1], only one quarter of people from pharmaceutical companies think that their company is better than competing companies in selecting CROs, with no significant differences based on the size of the companies involved. That is, large companies are no more likely to think they are especially good at CRO selection than are small companies. Similarly, there are no appreciable differences based on a participant's geographic location, whether it be North America, Europe or other locations worldwide. There is a slight but statistically insignificant difference in the age of individuals who think their company is better than others in selecting CROs—they are a year older, but have the same number of years of industry experience.

Reasons for selection

There is a general agreement between the reasons pharmaceutical companies cite for selecting an outsourcing organization and how CROs think pharmaceutical companies select them. There are also some striking differences in the individual decision criteria. Survey participants evaluated 17 individual considerations often used in selecting a CRO and rated each item on a scale ranging from very important to not important at all [the 17 criteria arose from our literature review and preliminary interviews with industry professionals]. Five of these 17 items were deemed to be especially crucial in the selection of CROs—that is, specified by at least half of all the study participants.

Team chemistry. The most important overall quality in selection is a sense of positive sponsor–CRO project team relationship. Both sponsor company and CRO respondents think that the sponsor wants to know above all else that they could work with the CRO. More than any other single selection criterion, sponsor companies stressed the importance of a good relationship between the two company teams. These teams must work closely with each other for an extended period of time, often on projects involving multiple countries and large sums of money. The sponsor company needs to know that both its team and the CRO's team share the same values, work ethics, and general approaches. CROs also clearly understand the need sponsors have for teams that work well together.

The CRO's project management team contributes significantly to the overall positive sponsor–CRO project team relationship. Yet sponsors were far more likely than CROs to stress the importance of the project management team as a CRO selection criterion. While more than two thirds (68%) of the pharmaceutical company participants stressed this criterion in their selection process, only half (51%) of the CRO respondents thought sponsor companies valued this as very important.

CRO experience. Sponsor companies and CROs placed great weight on a CRO's relevant experience, both in the same indication as the study and in the CRO's overall experience in the appropriate therapeutic area. Sponsor companies and CROs differed somewhat on the importance of a project member's background. While only 45% of the pharmaceutical company participants considered this very important, 62% of the CRO respondents considered it very important to sponsor company decision makers. Sponsor companies seem more interested in the overall cohesiveness of the CRO team members than their vitae. A vitae may be a constructive starting point, but positive sponsor–CRO project team relationship is vital to the project's success. As one pharmaceutical company project manager put it: "CVs can only take you so far. You want to know that the CRO's team fits with the way you want things done and the way you want to work."

Project execution plan. Another essential criterion (i.e., raised by more than a third of the study's R&D professionals), was the project execution plan. On no other single criterion, though, was there a greater difference between sponsor company and CRO responses. Fifty-nine percent of pharmaceutical company professionals considered the execution plan as fundamental in hiring a CRO; only 28% of respondents from CROs placed the same importance on the execution plan.

In their responses to this criterion, the pharmaceutical company decision makers appear to be emphasizing a level of project specificity that many of the CROs may not appreciate. Far more than the CRO project team's background, pharmaceutical companies want to see a convincing project execution plan, including such concrete items as resources, countries, and site selection. According to one pharmaceutical company outsourcing professional: "Every CRO claims to be customer oriented, responsive, and so forth. Exactly how a CRO proposes to carry out a project...now that can be convincing—or not."

Problem solving processes. Sponsor companies are also more likely to value the internal processes CROs use to make appropriate senior CRO managers aware of project problems. Forty-seven percent of the pharmaceutical company participants considered this capability to be essential in selecting a CRO. In contrast, only a third (34%) of the CRO respondents thought it was essential for sponsor companies.

Other criteria. A smaller number of respondents, from both sponsor companies and CROs, considered the CRO's role as a preferred provider and the CRO's geographic scope to be very important. CRO staff turnover was somewhat more critical to the sponsor companies than many CROs thought was the case.

Of the remaining criteria, one in particular merits its own discussion: the role of the lowest price bidder. Sponsor companies and CROs differ in how they view the role of the lowest price. According to an experienced business development manager from a major CRO, "Pharmaceutical companies rarely say price is important. But we know that it really is." But in this study, fewer than one in 10 (7%) of the sponsor company respondents considered the lowest price as decisive in selecting a CRO. In contrast, more than a third of the CROs (38%) felt price was a fundamental selection measure for sponsor companies.

Dimensions of selection

There were major differences in the importance the survey participants placed on the 17 individual items. People rarely make decisions using one consideration at a time. Rather, they relate one consideration to another and decide what's best. Companies follow the same process in selecting CROs. In fact, as mentioned earlier, sponsor companies use five general dimensions in evaluating and selecting a CRO:

• Processes for Issue Identification and Resolution

• Specifics of Project Execution

• Team Chemistry

• CRO Experience

• CRO Capabilities

The study used factor analysis to distill these five dimensions from the 17 individual survey items.

Factor analysis only provides a mathematical solution to how the various items relate to each other and how they relate to any underlying dimensions. Factor analysis does not label the individual factors. Items heavily associated with a factor are said to "load on" that factor. Items that load more heavily than others have a bigger explanatory role in understanding the communality of the items associated with that factor. Only a review of the items associated with a given factor can provide a rationale for what the individual items may have in common.

The individual factors appear in the order of the variance they explain, which is often a function of the number of individual items "loading on" that factor. That order does not necessarily mean that a given factor's explanatory importance is more or less than the factors before or after it. [Table 2, available on the Applied Clinical Trials Web site, summarizes which of the individual selection criteria load on each factor; a more complete matrix can also be found on the Web site.]

In this survey, five items load heavily on the first factor, Processes for Issue Identification and Resolution. The individual items all concern the same general issue area: processes used to identify and resolve project and financial issues. Three items are associated with the second factor, the Specifics of Project Execution. Critically, price is inversely related to the other criterion in this factor. The more participants stress the other two items of the factor—quality of the project team and the project execution plan—the less likely they are to emphasize price. Conversely, those who stress price put less weight on the quality of the other items in the factor. The third factor covers the Project Team's Chemistry, including the individuals, their backgrounds, staff turnover rates, and the sponsor company's sense that they can work with the CRO's personnel. Two items, both related to relevant CRO experience, load on the fourth factor: CRO Experience. The fifth factor addresses items associated with a CRO's Capabilities, including the CRO's relationship to the sponsor company.

The factor analysis provides a score for each individual for each factor. The higher the score for an individual participant on a factor, the more prominent that dimension is to the individual. The relative importance of these five decision-making dimensions differs within key groups involved in selecting and using CROs. For example, sponsor companies and CROs view the importance of one dimension, the Specifics of Project Execution, quite differently.

As we have seen, sponsor companies value the quality of the execution plan far more in selecting an outsourcing partner than many people in CROs realize. CRO personnel tend to think that price plays a larger role in the selection process than sponsor companies do. There is also one important difference that relates to an organization's size. Participants from the 20 largest organizations, whether they are sponsor companies or CROs, differ from individuals in smaller organizations on one decision-making dimension: CRO Capabilities.

Larger pharmaceutical companies and CROs give more weight to a CRO's size, geographic reach, and relationship with the sponsor companies. Large organizations, both sponsor companies and CROs, may be more comfortable working with one another as large organizations, especially for bigger studies requiring extensive geographical coverage.

Differences by geography

The greatest differences across the dimensions are associated with a respondent's geographic location. The components of the CRO selection criteria are important to both Europeans and Americans. However, Europeans and Americans do differ on three of the dimensions.

For example, Europeans are far less interested than others in the details and mechanics of identifying and resolving issues and, ultimately, in a CRO's willingness to accept accountability. Americans are more likely to consider Execution Specifics and CRO Capabilities important. They want to see the details of the project plan and project management team devoted to the study. They are more interested in a preferred provider relationship with a larger CRO having geographic reach.

While Americans stress the mechanics of the CRO/sponsor company relationship, Europeans tend to emphasize more intangibles. These differences by a respondent's geographic location remain significant even when we use ANOVA (analysis of variance) to control for the possible influence of other variables, such as company size and whether a person works for a CRO or sponsor company.

Selection skill levels

Most of the pharmaceutical companies surveyed believe that they are on the same skill level as their competitors in selecting CROs, although a few believe that they are more adept at the process. Interestingly, there is no difference in the selection criteria between those companies who believe they are better at CRO selection and those who do not. No matter how well a survey participant's company believes they handle CRO selection, there is no difference when it comes to the five dimensions of selecting a CRO.

In later research we will examine in more detail the impact of geography in how people approach CRO selection. In addition, we will also present the profiles of companies who do consider themselves better at selecting and managing CROs.

Harold E. Glass, PhD, is a professor of health policy at the University of the Sciences in Philadelphia and the president of TTC, llc, 4548 Market St., Suite M-20, Philadelphia, PA, 19139, email: [email protected]. Daniel P. Beaudry, MBA, is an associate director at TTC, llc.

*To whom all correspondence should be addressed.

References

1. T. Hintz, "Premiere of Exclusive Survey Results: Sourcing Models that Create Value in CRO Partnerships," 16th Annual Partnerships with CROs and Other Outsourcing Providers Conference, IIR, April 14–16, 2007.

2. Contract Pharma, 2007 Annual Outsourcing Survey (May 2007), http://www.contractpharma.com/articles/2007/05/2007-annual-outsourcing-survey.

3. L.R. Fabrigar, D.T. Wegener, R.C. MacCallum, and E.J. Strahan, "Evaluating the Use of Exploratory Factor Analysis in Psychological Research," Psychological Methods, 4, 272–299 (1999).

Two Years Later

The Author's Thoughts

Outsourcing clinical research activities to CROs has become an integral part of the drug development process for pharmaceutical and biotech companies of all types and sizes. Study after study points to a continued growth in CRO usage. Moreover, the financial markets reinforce this message by rewarding CROs with higher P/E multiples than for most established pharmaceutical and biotech companies.

The University of the Sciences in Philadelphia and TTC, llc have been conducting a multi-year study on why some clinical trials finish faster than others. Understanding the role and effectiveness of CRO run studies is a key element of the research. Part of the CRO specific research in this multi-year effort involved a Web-based survey of both sponsor company and CRO professionals to understand the criteria used in the selection of CROs. Other studies have reported the significance of previous work quality as a central criterion in CRO selection. This study accepted that criterion's importance and examined other potentially major issues.

The results in this article are distinctive in two ways. First, the research made use of factor analysis to understand how the various individual criteria used in CRO selection grouped together. People rarely make important decisions using only one consideration at a time. Rather, they bring in a number of elements. Factor analysis allowed us to determine how the individual elements fit together. Second, we examined the decision criteria of sponsor company participants, comparing them to how CRO company participants believed sponsor companies made their decisions.

Our factor analysis of the various individual criteria used in CRO selection revealed five underlying dimensions, in order of importance: team chemistry; CRO experience; the project execution plan; problem solving processes; and price and other less important criteria. There was widespread agreement between sponsor company and CRO respondents, with two exceptions. Unsurprisingly, CRO respondents believed that the final bid price played a more important role in the decision-making than sponsor company participants indicated was the case. More importantly, sponsors value the quality and detail of the execution plan far more in selecting an outsourcing partner than many people in CROs realize.

The Advisory Board's Take

CRO selection remains an important topic for both the pharmaceutical sponsor companies and the CROs.

Pharmaceutical companies have outsourced, are outsourcing, and will continue to outsource drug development activities, and the consensus view is that the amount of outsourcing will only increase over the years. The goal of mergers, acquisitions, or internal reorganizations is often to create leaner and more flexible drug development organizations. The increasing demands on patient (safety) data from regulatory agencies, and the need to develop innovative medicines, drive the need for outsourcing all or part of the drug development activities. And selecting the right CRO is of utmost importance.

From the CRO perspective, it is important to understand why pharmaceutical companies outsource and what the key decision making points are in selecting the right CRO in order to supply the demand. CROs have understood this and have become better service organizations often with client liaison managers focusing on understanding the needs and finding solutions to meet these needs.

The article by Harold Glass and Daniel Beaudry is still important today; it describes the outcome of a very large survey with 419 responders from 116 companies, has a good balance of pharmaceutical company and CRO employees, a good mix of large and small companies, and a relevant geographical mix. Although more and more pharmaceutical companies have dedicated teams responsible for selecting and managing CROs, and have made a core competence of this, there are still many companies, particularly the smaller ones, who lack this competence. This article may help in guiding them on what to look for in selecting the right CRO.

"Team Chemistry" came out as one of the most crucial items in CRO selection, next to having the right CRO project management team and the relevant CRO experience in the therapeutic area or in the indication. This is not surprising, people are still the ones that make it happen. Whether outsourcing is a strategic or a tactical choice, both pharmaceutical companies and CROs would like to be successful in the end.