Using Machine Learning and NLP to Improve Central Monitoring Documentation

Image Credit: © Murrstock - stock.adobe.com

Central monitoring helps sponsors to proactively identify and remediate data quality issues during the conduct of clinical trials. In order to meet regulatory requirements and to enable continuous learning and improvement, it is essential to ensure that central monitoring risk follow-up activities are well-documented from initial detection of each risk through to its resolution. The documentation should detail how each risk was investigated along with the outcome, including whether an issue was identified and if/how it was remediated.

Unfortunately, many study teams and organizations struggle to achieve this level of documentation for their risk signals. There are various factors contributing to poor documentation. This includes study team members who may feel they are too busy to document the full story of their risk follow-ups and/or don’t understand its importance.

Recognizing this challenge and seeing an opportunity to encourage and support better documentation practices, CluePoints introduced a new feature into its central monitoring platform at the end of 2021.

Using natural language processing (NLP) and deep learning techniques and methods, this feature automatically scans all of the existing documentation for each risk signal and alerts users (e.g., central monitors) whenever that documentation is unclear and encourages them to provide further insights about signal investigations and follow-up activities. The deep learning solution was trained to identify “unclear” risk signals as those for which the documentation lacked sufficient explanation of what the outcomes were.

By running this algorithm retrospectively on all risk signals processed prior to the release of this feature, we observed that 40% of the signals were unclearly documented. This has been reduced to 30% since the release, or a 25% overall improvement in signal documentation (see Figure 1 below).

Figure 1: The median rate of signals with unclear documentation by study.

Source: CluePoints

We additionally looked at organizations that only started using the central monitoring platform following release of this feature (i.e., over the previous two years).

These organizations are not only benefitting from the NLP feature, but also received additional training on signal documentation best practices as part of their onboarding. The combination of these actions has helped reduce the unclear signal rate even further, with only 20% of the signals in this group remaining unclear, as indicated in Figure 1.

While it is not the complete answer, it is clear that the introduction of this feature has contributed significantly to improving signal documentation by interactively alerting teams to documentation shortfalls in a user-friendly manner. It serves as another successful example of the application of machine learning to improve the conduct and management of clinical research.

Steve Young, chief scientific officer, and Sylviane de Viron
data and knowledge manager; both with CluePoints