Inside Investigator Meetings

Nowadays, conducting a clinical study is more complicated than ever. More complex protocols, new technologies, and increased regulatory scrutiny have all made study execution more challenging. Regardless of the rise in difficulty, what really matters is that a sponsor collects good, clean data that reflect the reality of how well its drug works. So the question becomes, why don't sponsor companies put more effort into working with their sites to not only ensure that data quality will be built in up front, but that study operations at the site level will be managed efficiently and effectively?

PHOTOGRAPHY: GETTY IMAGES

Simply thinking about the amount of information a site must understand about a protocol—inclusion/exclusion criteria, procedures related to each visit, adverse event reporting, lab procedures, recruitment strategies—is enough to make one's head spin. Considering that sites may be participating in several studies at any given time, often with different sponsors, it becomes clear that there is a lot of room for error on the site's part.

Given the complexity of studies and the responsibilities of sites, do sponsor companies sufficiently support sites in this regard? The common practice, of course, is to hold an investigator meeting that doctors and study coordinators are supposed to flock to in order to learn about the protocol at hand, ask questions, and otherwise prepare themselves for participation in the upcoming study. This is an age-old practice, but is it still viable? Do steps need to be taken to provide training that is not only more effective but delivered in a more efficient manner as well? A closer look into the characteristics of investigator meetings will help to answer these questions.

The good

A good place to start is with the sites themselves. Do they feel that investigator meetings are worthwhile? According to a recent CenterWatch Monthly survey of 102 investigative sites, 78% of respondents found the overall usefulness of investigator meetings to be "Good" or "Excellent." That certainly bodes well, although skeptics may point out that the respondents really have no point of reference; investigator meetings are all they know. Still, based on these results, it would seem that sites do feel that investigator meetings have something of value to offer.

The opportunity to network is often cited as one of the most important benefits of attending investigator meetings. An investigator who has no experience with the study drug (or with clinical studies in general) can pick up some good information by talking to those with more experience. In addition to the scientific networking, investigators also network in a traditional business sense by making new sponsor contacts in an effort to ensure that they will be utilized in future trials. Either way, having sponsor personnel and site staff get to know each other early on can help with establishing better communication during the study itself.

Having the opportunity to discuss protocol specific subjects directly with the sponsor company is another reason that meetings are seen to have value. Open discussions about inclusion/exclusion criteria are helpful in developing better investigator understanding of not just "what" the criteria are, but "why" as well. Having an open discussion about the scientific background of the study and study drug is a good way to educate the investigators while appealing to their professional side and helping them feel they are a part of the sponsor's efforts to bring the study drug to the masses.

The interactivity of meetings in general is yet another critical aspect of in-person investigator meetings. Regardless of the topic at hand, having a dialogue and getting instant feedback on questions and concerns is something that both sponsors and attendees place significant value on.

In addition to the idea of interactivity, attending an investigator meeting gives both the doctor and study coordinator the opportunity to learn about the protocol without the myriad distractions that exist in their office back home. This change of pace can help the attendees to better focus on the task at hand and allow them to immerse themselves in the study details for a couple of days, potentially leading to better comprehension of the protocol.

Lastly, there are the intangibles. Building a sense of community by bringing people together can ensure better coordination among the sponsor and its sites. Many investigators and study coordinators see the attendance of such a meeting (especially when held in a desirable location) to be a perk of study participation. Irrespective of the reason, a positive experience at the meeting can lead to increased effort on the site's part.

The bad

As mentioned, 78% of respondents in the CenterWatch Monthly survey rated the overall usefulness of investigator meetings as "Good" or "Excellent." But how does one define usefulness? To whom is the meeting useful and for what reason? According to one of the CenterWatch respondents, "The networking is, frankly, the only reason I'm there," referring to his ability to directly ask sponsor representatives for additional studies while at an investigator meeting.

While there may be some benefit to the sponsor being aware that a particular doctor is interested in handling more studies, clearly this is not the intent of the investigator meeting. Whereas scientific networking is typically beneficial for all involved, it is hard to see how this type of use of an investigator meeting is of much value (if any) to the sponsor. In other words, if many of those 78% of respondents that gave favorable marks to investigator meetings were doing so based on their ability to do personal business, one must wonder how seriously the survey results should be taken.

Investigator meetings can be overstuffed with unnecessary or redundant information and can be considered by more than a few investigators to be a waste of time. This is largely due to the fact that attendees are fragmented in terms of job roles (principal investigator, study coordinator, subinvestigator, monitor, etc.) and level of knowledge or experience regarding the study drug or clinical trials in general. As a result, each presentation will be of interest to some while completely boring to others. Typically, meeting materials are geared toward the rookies, which tends to annoy more experienced investigators—the very ones that the sponsors should be trying hardest to retain. The thought of having to go through hours and hours of GCP training for the 29th time can make a veteran investigator think twice before agreeing to attend the meeting. This leads to generally poor principal investigator attendance rates. Figures vary from sponsor to sponsor, but many in the industry place that number in the 40% to 60% range.

These days, more and more sponsor companies are keeping a close eye on costs, and investigator meetings can be a tremendous financial sinkhole. Again, figures vary by sponsor, but most companies peg that number at around $2000 per attendee. For a typical Phase III investigator meeting for a study with 50 sites and two attendees per site (adding in another 20 attendees from the sponsor and its vendors), a sponsor can expect to pay approximately $240,000—nearly a quarter of a million dollars. Larger studies can easily approach the $1 million dollar mark. Given that large pharma companies can easily approach 100 studies per year (or more), there is obviously cause for alarm in the accounting department.

The figures given in the preceding paragraph don't include things such as lost productivity due to the fact that sponsor personnel are tied up for several days while attending the meeting (potentially four or five, including travel time). Given the number of person-hours of work lost during that time, the impact can be significant from a financial aspect, not to mention the potential for disruption of study timelines. From the site perspective, there is a similar productivity issue, with a doctor's practice losing revenue for those days spent attending the meeting. Time spent away from family and friends can be distracting at the least and cause resentment in the extreme.

Finally, do attendees leave the meeting knowing more than they did at the outset? Was the cost of the meeting justified by the increase in knowledge? The effectiveness of the presentation marathon approach (known in some circles as "Death by PowerPoint") used at most investigator meetings is questionable at best. While the sponsor retains records that people showed up at the meeting, they have no way of knowing if anyone actually learned anything. Other factors, such as timing and turnover, can also have an impact on the long-term effectiveness of training. Timing of the meeting relative to first patient screened, for example, can mitigate many benefits gleaned. Considering that a site may not see its first patient for three to six months (or longer) after the meeting, there is a good chance they will forget much of what was learned. Turnover is another issue. If, for example, a study coordinator leaves a site for a new job one year into a three year study, chances are that his or her replacement will not receive the same training that their predecessor did.

Author survey

Despite the issues mentioned earlier with regards to collecting feedback via surveys, this author was able to collect a significant amount of data from attendees at various investigator meetings (21 in total, all Phase III studies) using two different sets of standardized surveys throughout: internal (sponsor focused) and external (site focused). At the end of each meeting, the internal and external surveys were given to sponsor and site personnel and asked that respondents rate their meeting on a number of dimensions, from the quality and delivery of meeting material to logistical aspects such as location and accommodations.

Figure 1. Leading complaints sponsor staff had about investigator meetings according to a survey conducted by the author.

Of the responses collected from sponsor staff, nearly 90% said that the meetings needed at least some improvement, with more than 80 negative characteristics identified, including boring meetings, inadequate meeting content, poor presentation skills, ineffective session design, and a lack of attention paid to attendee comprehension (see Figure 1). From an external perspective, the meeting ratings were highly variable and depended largely on the venue and location, as well as presentation skills and the experience of the sponsor staff running the meeting. The highest rated meetings were those that had an interactive component, such as the use of an audience response system, protocol "Q&A" sessions or discussions on enrollment strategies (see Table 1).

Table 1. The differences between good investigator meetings and poor ones.

Root of the matter

Regardless of how well or poorly one perceives the current state of investigator meetings, there is no question that there is room for improvement. Like any improvement initiative, the first question the sponsor needs to ask when approaching this problem is, "How does this benefit the bottom line?" The value of an investigator meeting needs to be properly defined and evaluated, and any subsequent improvement efforts should be done with that value proposition in mind.

In order to get their drugs to market (and thus make money), sponsors need to collect clean, objective data that speak to their drugs' safety and efficacy, which of course is where the investigators come in. The focus of investigator meetings should be aligned with this higher level objective. It is not much of a leap to assume that better training will lead to better protocol and procedure comprehension, which in turn will lead to better data quality, higher enrolling sites, and an overall improvement in a site's efficiency. Therefore, the question shouldn't be "How do we improve the meetings so that attendees are happier?" but rather "How can we improve the training so that attendees can do a better job conducting the study?" That is, how can the concept of the investigator meeting be reconfigured so as to ensure the higher level objective is met?

At the onset of any study, there are a number of site initiation challenges to overcome. What does site staff know about the study? When do they learn it and for how long will they remember? Did all members of the study site team read and understand the protocol? Do they all have the same baseline knowledge? All of these questions need to be properly addressed if sites are to be fully capable of helping the sponsor to meet its high level objectives. In addition, there is the regulatory aspect to consider, as "GCP compliance and training issues top regulators' concerns."² By addressing and overcoming these challenges, not only will the sites be better prepared to conduct the study, but experienced site personnel will no longer be forced to endure the standard "one size fits all" training that tends to be aimed at those with little or no experience.

One way to accomplish this is to create a series of baseline assessment tests for site personnel. For example, case studies on the subject of adverse event reporting can be used to test how proficient an investigator is at assessing severity, causality, and reporting actions to be taken. Better yet, the case studies can be tailored so that what is presented is actually relevant to the study at hand. Those who do well on the baseline assessment (i.e., experienced investigators) can go through an abbreviated training on the subject; those who do not do well must then go through the standard, lengthier training.

One other issue that bears mentioning is the conflict that exists between the sponsor's motives for holding an investigator meeting and the site's motives for attending them. In many cases, they are the same: to ensure that site staff is educated with regard to the protocol and its related procedures. However, as previously mentioned, there are investigators who attend for strictly business reasons. In the CenterWatch survey, study coordinators consistently cited investigator meeting attendance as a perk of their job and critical to their professional development. Should sponsor companies be paying hundreds of thousands of dollars per meeting just so study coordinators are happier in their jobs? There is undoubtedly a benefit to having coordinators attend the meetings—to share recruitment ideas with sponsor contacts or to develop a better relationship with their investigator—but whether or not that benefit outweighs the cost is debatable.

Five Ways to Improve on a Budget

On the other side of this argument is the sponsor's responsibility to put in the time and effort to plan and hold a meeting that an investigator feels is worth attending when weighed against his or her other commitments (not to mention those of study coordinators, etc.). A colleague of this author was recently at an investigator meeting to do a vendor presentation. While he was checking in at the registration desk, an investigator walked up, took a look at the agenda, and demanded to be flown back home immediately (and he was). An extreme case perhaps, but how many other attendees have at least thought of doing that?

The future

While the primary goal of investigator meetings is to improve the performance of study site staff and ensure quality of data, there are other elements to take into consideration. Resources are never unlimited of course, so while entertaining possible solutions an eye must be kept on costs as well as demands on sponsor personnel such as monitors and data entry workers.

Improved and simplified agendas, increased interactivity, shorter meetings, less travel time, enhanced speaker performance, and an elimination of training redundancies can make meetings better and more effective for all involved. Sponsors should start with training outcomes in mind and plan their meetings around those stated goals. With specific and measurable goals to be met, more thought will have to be given to the way information is presented to attendees. Incorporating adult learning techniques such as interactive case studies or break out sessions can be a far more effective method of training than showing hundreds of PowerPoint slides.

eLearning alternatives should also be explored. Utilization of eLearning systems can save a significant amount of time for those unwilling or unable to attend a meeting while providing standardized training that is self-paced and at least as effective as what is provided at in-person meetings. Moreover, eLearning systems can be used to track who has completed and, better yet, comprehended the training and allow the sponsor to retain training records for future reference, thus addressing potential compliance issues up front.

Since there are benefits to both in-person meetings and eLearning, it just may be that the optimal solution is to use both in combination. For example, experienced investigators could receive their training via eLearning, while inexperienced investigators and coordinators could attend in person. Another option would be to have everyone receive some of the generic training such as GCP training via eLearning, and then hold a shorter and more focused meeting that covers protocol-specific topics such as scientific background and inclusion/exclusion criteria.

Up to this point in time, sponsors have been front loading the protocol-specific training by having it all delivered at the investigator meeting before a study begins. After that, there's really no study-wide resource for the sites to draw on, and field monitors are typically the only alternative once a study is underway. Not only does this practice run the risk of allowing site personnel to forget much of what was covered at the meeting, but having sponsor personnel repeatedly bogged down answering the same questions over and over again to different sites is surely not the most efficient way to do business.

Approaching study execution from more of a site management or site support perspective leads one to the conclusion that there will be far greater benefit in providing some kind of "continuing education" resource that is available to all participants for the life of the study. In addition to addressing issues of timing and sponsor resource demands, this type of approach also solves the problem that occurs when there is turnover among site (or sponsor) staff, namely ensuring that everyone receives the same training.

Conclusion

Despite the fact that investigator meetings are perceived by some as being lengthy, costly, and ineffective, there is no basis for saying they are in such dire straits that they are badly in need of immediate and radical changes. Still, as with other processes that have an impact on critical outputs, they should be incorporated into a system of continuous improvement.

Speaking from a lean perspective, all waste in the system should be eliminated or converted into value. Exploring this concept further, it becomes apparent that there are in fact many areas where waste exists: moving people around unnecessarily, waiting or down time, various sponsor efforts that add no value, etc. If one were to define the customer in this microcosm as being the site staff, then they would be responsible for specifying value; this assumption is only valid, however, if the site personnel's motives for attending investigator meetings were perfectly aligned with the sponsor's—that is, they are there solely to receive training and instructions that will allow them to properly execute the protocol and ultimately deliver a quality drug that fills a market need. Although in theory the sites should to a degree be defining the "value" of the meeting, at present it is the sponsor who completely dictates everything about the meeting, such as the content, timing, and location. Allowing more site input on these matters would correct this.

The lean concept of flow, which refers to how well value flows through the value stream, talks about how end products should always be developed by cross-functional teams so that the various dimensions of customer needs are represented and internally coordinated. Yet most of the investigator meeting presentation materials are developed by people in isolation. Having presenters and chairpersons work as a team will help to eliminate redundancies in the materials and ensure a better flow of the agenda.

The issue of the timing of the meetings was mentioned earlier. Scheduling the meeting so far ahead of study kick off violates the lean concept of "pull," which states that customers should be able to pull value from the value stream when they need it and with minimal effort. Investigator meetings, on the other hand, "push" information on sites when they have no need of it. Conversely, sponsor companies typically don't provide a system (such as eLearning) that allows sites to pull specific information at the precise moment they require it.

Sponsors can realize benefits that include higher quality data, better performing sites, enhanced site relationships, and significant cost and time savings. Sponsors should also develop and begin to track metrics that measure investigator meeting performance across a variety of dimensions, such as cost, timeliness, and effectiveness (for example, inspecting the difference between the baseline and final training assessments). First and foremost, sponsors and sites alike should keep in mind the ultimate purpose of the investigator meeting: to enable the safe and effective execution of a study protocol.

Eric Lake is a partner with Pharmica Consulting, PO Box 2754, Oak Ridge, NJ 07438, email: eric.lake@pharmicaconsulting.com

References

1. S. Space, "Changing Investigator Meeting Formats," The CenterWatch Monthly, August 2006, 1-11.

2. M. Mathieu, "The Global GCP Compliance Report 2006: U.S., EU, and Japan," Annual Compliance Report from Parexel.