What's Behind Clinical Trial Data?

Is 90% of all data collected rubbish? A provocative keynote debate at the recent DIA 2nd European Clinical Forum explored this question.

Is 90% of All Data Collected Rubbish?

Specific to electronically delivered data, Nick Lucas of INC Research, UK, said it is never clear whether what is received is in fact what was requested. Statistics show that 100% of all queries are generated by 40% of all validation checks, raising the question as to what the rest of the validation checks are checking—probably unnecessary data.

Other statistics show that less than 5% of the data in a database changes after it has been entered. Therefore, more than 95% of the data never changes post entry. This means data cleaning efforts are focused on a very small amount of the data, and it could be argued on the need to focus on the 95% of untouched data. Certainly it may be clean, but it may also not be the focus of the Safety/Efficacy parameters required of the study and, therefore, superfluous.

The sheer volume of queries—1 in 10 CRF pages, 10 in 10 CRF pages, or 20 in 10 CRF pages—is an indicator of poor quality data. Using EDC means that the data quality at point of entry should be better. But sites complain that the computer keeps raising queries as data is entered. The answer is to persuade sites to enter better data and the system will not keep raising queries.

Jane Barrett of the Barrett Consultancy, noted that in research, "We do not know, what we do not know." It may well be that 90% of data will show nothing, either positive or negative, but knowing that nothing changes is also useful information. Thalidomide was tested in development in one species, and no teratogenicity was seen. When later tested in another species the teratogenicity seen in humans was confirmed. If the testing is not down in that second species, we do not know all the answers. It is indeed possible that only 10% of data is useful, but the other 90% normal is needed to show that.

Joris Cauquil, of AMITIS/Effi-Stat, France, questioned whether the answers to queries are real, or merely represent the investigator trying to pacify the sponsor. Also, is it necessary to gather so much data, bearing in mind that the more statistical tests performed, the more likely a false positive will result. It is far more important to ensure accuracy of measurement and reliability, both of clinical measurements and software, and to ensure that the criteria of the study are met.

Linda Talley of Eli Lilly, in the United States, reminded the audience that clinical data is the foundation of drug development, and the data collected are used by a variety of different individuals for different purposes. The trial data collection strategies for Phases I, II, III, and IV are very distinct and serve different purposes. But Talley emphasized a need to look at the whole drug development program and not just individual trials when collecting data and determining data needs. The stance should always be addressing data quantity collected versus quality.

Jane Barrett, MBBS, FFPM, LLM, is the Treasurer of the British Association of Pharmaceutical Physicians, and is principal of The Barrett Consultancy, janebarrett@doctors.org.uk