Turning Challenges into Opportunities: Evaluation of Digital Health Products

Clinical evaluation of medical devices is a systematic process that requires careful, strategic planning. It’s a process that encompasses the continuous generation, collection and analysis of clinical data from external and internal sources and, performed with the aim of verifying the performance, safety and clinical benefit of a device for its intended use. A critical part of medical device development, this evaluation must comply with the general safety and performance requirements (GSLA) so that the device can be approved by regulatory authorities. In Europe, one key component of the assessment is a technical document referred to as a clinical evaluation report (CER), which is required for several categories of medical devices to be CE-marked and increasingly emphasized by new EU Medical Device Regulations (MDR) released in May 2020. FDA device submissions, although may somewhat differ in format from the EU submissions, also require a collation of evidence demonstrating that the device to be marketed is safe and effective, or that it is substantially equivalent to a legally marketed device. This collation of evidence is in essence the core of a CER document.

Emerging companies and digital health startups, without experience in this area and who are developing novel, first-of-their kind medical technologies, can find CER writing particularly daunting.

Shiri Diskin, PhD, Head of Medical Writing at Bioforum, a data-focused CRO serving life sciences companies worldwide, sat down with Daphna Laifenfeld, PhD, Chief Scientific Officer at Ibex Medical Analytics, a company developing AI-based solutions for cancer diagnosis, to discuss the challenges associated with developing a CER and how to effectively navigate them while creating opportunities that increase efficiencies.

Shiri Diskin: Why is the CER writing process more complex for digital health solutions?

Daphna Laifenfeld: It’s often difficult to identify similar existing commercial digital health products that can be used for comparison, or at the very least, as an example for evaluation criteria. Many times, the product is innovative enough so as to create a new field. This dynamic creates several challenges for sponsors of clinical trials of digital health solutions. First and foremost, clinical evidence of performance must be generated by the sponsor through clinical studies and the use of previously published data or research methodology is very limited. Second, appraisal of literature for its quality and relevance, which is a major aspect of writing the CER, is similarly complex given the scarcity of sources for comparisons.

Furthermore, the evaluations of digital health products that use mathematical algorithms rely on fields of knowledge that are quite distant from the medical field, such as complex statistical models, machine learning, aspects of software development and computational methods. Consequently, it’s difficult to find experts who can synthesize such vast amounts of data and the complex methodologies used during development into a coherent appraisal of the product within the context of its intended use, in a way that will also becompelling to a regulatory reviewer.

SD: At which point in your product development process did you decide to begin the CER writing process and on what did you base the decision?

Laifenfeld: Ibex Medical Analytics develops AI-based solutions that help pathologists diagnose cancer more accurately and efficiently. The Galen Breast solution, our automated tool for analyzing images of breast biopsy specimens, identifies invasive and in situ breast carcinoma. It’s an AI-based in-vitro diagnostic tool and its current intended use is as a ‘second read’ of breast biopsies for quality control of pathologists’ diagnoses. We are currently in the process of applying for a CE mark under IVDD, which does not mandate a CER for our products. However, the CER will be required by the In Vitro Diagnostics Regulation (IVDR), which will come into effect in May 2022. This legislation was formulated by the European council and it is to be used by the European Medicines Agency (EMA) and national competent authorities while assessing medical devices. As part of our overall commitment to stringent quality standards, we view the process of developing a CER as an important step in preparing the technical documentation, clinical evidence and supportive data we use for positioning our product in the market, regardless of the regulatory requirement at this stage. Moreover, the organized presentation of data in the CER can assist us in planning the product roadmap going forward. Ibex is a scientifically rigorous and quality-driven company, and adherence to the strictest requirements is one of our core values. Thus, with the prospect of IVDR coming into effect next year, we essentially decided to have a CER from the outset, and we will continue to update it as more information becomesavailable.

SD: Why did you decide to outsource the CER writing process?

Laifenfeld: As is the case in most fast-growing companies, in-house resources are limited and focused on the company’s core expertise. MEDDEV 2.7/1 rev. 4 specifies that the CER should be written by a person with training and experience in medical writing. The CER should present a highly professional and compelling argument, from both the regulatory and the internal, company point of view. We do not, at this stage, have the specific knowledge and experience required to undertake such a substantial task within our small organization, so we chose to partner with external experts. In addition, we believe that assigning this writing task to a dedicated, external CER professional brings a broader perspective, awareness of best practices and new insights to the process that provides for a more effective and
efficient submission.

For example, with Galen Breast, the process of developing the document and the discussions with the writers forced us to take a break from the daily management of product development and consider wider aspects such as the product’s position within the field and its current risk-to-benefit ratio. The final report can also be used to help guide our short- and long-term development path.

SD: In your view, when it comes to developing a CER, what are some of the advantages and disadvantages digital health products pose?

Laifenfeld: The main advantage is the fact that physical safety is not an issue with software. This means that a substantial part of the evaluation and related components within the document–namely detailed assessments of device specifications, biocompatibility, and safety aspects related to hardware components that interact with the human body–are not required.

This is not to say that there is not a considerable amount of work associated with risk evaluation. Most digital health products can be viewed as decision-support tools. They provide information that supports the patient, the caregiver or the physician with decision making. Thus, the main risk associated with use of such products is arriving at a decision, based on data from the product’s output, that may potentially not benefit or may potentially harm the patient. Such risks should be carefully evaluated and mitigated for each product with its specific characteristics and intended use.

Another type of risk is information security. Digital health products that collect or store any patient data are also bound by confidentiality, cybersecurity and patient privacy requirements and regulation, such as the American Health Insurance Portability and Accountability Act of 1996 (HIPAA) or the EU General Data Protection Regulation (GDPR). Any digital health product that stores patient information, may pose the risk of this private information being accessed by third parties, not ethically allowed to view or use it. A careful analysis of cybersecurity risks should be conducted on a continuous basis to identify and mitigate them early.

For a digital health product, physical tracing of the product or its individual components is not necessary, making unique device identification requirements simpler.

—Shiri Diskin, PhD,Head of Medical Writing at Bioforum