A non-significant difference in a clinical trial means that the observed difference between two groups is not large enough to be statistically confident that it was not due to chance. Typically, a p-value greater than 0.05 is considered not significant. The p-value indicates how likely it is that the observed data or something more extreme will occur given the null hypothesis. The significance level (usually 0.05) is the threshold at which the p-value is considered small enough to reject the null hypothesis.

Example:

A clinical study compares a new implant with an existing implant and finds a p-value of 0.08. This means that the probability that the observed difference was due to chance is higher than 5%. Since the p-value is above the established significance level of 0.05, the difference is considered not significant.

3. Why is this not equivalent to equivalence?

In contrast to testing for a statistically significant difference, equivalence testing aims to show that the differences between two treatments or products are so small that they lie within a clinically acceptable range. This is achieved through specific study designs such as equivalence or non-inferiority studies.

Equivalence studies:

These studies set two predefined limits (equivalence limits) within which the differences between treatments must lie to be considered equivalent. The goal is to show that the effectiveness or safety of the new product does not differ significantly from that of the established product.

Non-inferiority studies:

These studies check whether the new product is no worse than the existing product by only setting a lower limit that the new product cannot exceed.

4. Differences in methodology

4.1 Null hypothesis

When testing for statistically significant differences, the null hypothesis is usually that there is no difference. In equivalence studies, however, the null hypothesis is that the treatments are not equivalent. The study must provide enough evidence to refute this null hypothesis.

Statistical significance tests play a central role in both types of studies, but the objectives and interpretation of the results differ. In classic tests of statistical significance, one looks for evidence that an observed difference did not occur by chance. The null hypothesis is rejected if a statistically significant difference is found (p-value < α).

In equivalence studies, however, the null hypothesis is that the treatments are not equivalent (that there is a significant difference). To refute this null hypothesis, the study must show that the differences between treatments are small enough to fall within a predefined equivalence range. Statistical significance is also tested here, but a different confidence interval is used. The results must show that the confidence interval of the difference lies entirely within the equivalence region to achieve statistical significance in terms of equivalence.

So in both cases statistical significance is used, but with different goals and interpretations.

4.2 Confidence intervals

While when testing for significant differences, confidence intervals are used to show the uncertainty of the estimate, in equivalence studies, confidence intervals are used to check whether they lie within the established equivalence limits. If the entire confidence interval lies within these limits, equivalence can be assumed.

These differences in methodology make it clear that the mere absence of a statistically significant difference is not sufficient to demonstrate equivalence. There are other factors that must be taken into account to ensure correct interpretation of the study results.

4.3 Lack of power of the study

A study with a small sample size or insufficient power may miss true differences. The lack of a significant difference may therefore simply be due to the study not being sufficiently powered to detect this difference. This is where sample size planning comes into play: careful sample size planning is crucial to ensure the power of the study. The power of a study describes the probability that the study will detect a real effect if it actually exists. Without appropriate sample size planning, there is a risk that a study will not be able to detect significant differences, even if they exist, due to too few participants.

4.4 Confidence intervals and uncertainty of the estimate

A non-significant difference can be associated with wide confidence intervals, which can indicate both clinically important differences and no differences. This shows the uncertainty of the estimate and does not suggest equivalence.

4.5 False null hypothesis

The null hypothesis in most studies is that there is no difference. Failure to reject this null hypothesis does not mean that it has been proven that there is no difference, just that there is not enough evidence to claim the opposite.

5. Examples of problems in clinical trials of medical devices

5.1 Comparison of two implants

In a study evaluating a new hip implant compared to an established product, a p-value of 0.06 was found. Although the difference is not statistically significant, the new implant could still be less effective or safe. A wide confidence interval could range from large superiority to significant inferiority.

5.2 Evaluation of a new diagnostic device

A new diagnostic device is tested against a standard device and the results show a p-value of 0.09. This doesn't mean that both devices are equally good, just that the study didn't find enough evidence to determine a difference. The study may not be large enough to detect small but clinically relevant differences.

6. How should equivalence be checked?

6.1 Equivalence and non-inferiority studies

To test equivalence, specific study designs such as equivalence or non-inferiority studies must be used. These studies have specific hypotheses and statistical methods to show that the differences between treatments are within a predefined tolerance limit.

Example:

An equivalence study could define that the new implant is clinically equivalent if the difference in functionality is within a range of ± 2% compared to the standard implant.

6.2 Confidence intervals and equivalence limits

Instead of just looking at p-values, confidence intervals should also be considered. If the entire confidence interval lies within the predefined equivalence limits, equivalence can be assumed.

7. Practical steps to avoid misunderstandings

Clear study design:

The study should clearly define whether it aims to find differences (superiority study) or to prove equivalence or non-inferiority. This influences the choice of statistical methods and the interpretation of the results.

Adequate sample size:

A sufficient sample size is crucial to ensure the power of the study. This helps detect real differences and avoid false negatives.

Predefined equivalence limits:

Before starting the study, clear equivalence limits should be established based on clinical considerations. This helps to better assess the clinical relevance of the results.

8. Conclusion

The absence of a statistically significant difference in clinical trials does not automatically mean that the medical devices tested are equivalent. Specific study designs and statistical methods are required to demonstrate equivalence. Careful planning and interpretation of study results are crucial to assess the true effectiveness and safety of medical devices. This is the only way we can ensure that new products meet the high standards of clinical practice and offer real benefits for patients.

9. How we can help you

Our statisticians accompany you from data collection through analysis to interpretation of the results. Be safe.

As CRO, we support you throughout the entire process of generating and evaluating clinical data and in the approval and market monitoring of your product. And we start with the clinical strategy! We also create the complete clinical evaluation file for you.

In the case of clinical trials, we consider together with you whether and, if so, which clinical trial needs to be carried out, under what conditions and in accordance with what requirements. We clarify this as part of the pre-study phase: In 3 steps, we determine the correct and cost-effective strategy with regard to the clinical data collection required in your case.

If a clinical trial is to be carried out, basic safety and performance requirements must first be met. The data from the clinical trial then flow into the clinical evaluation, which in turn forms the basis for post-market clinical follow-up (PMCF) activities (including a PMCF study if necessary).

In addition, all medical device manufacturers require a quality management system (QMS), including when developing Class I products.

We support you throughout your entire project with your medical device, starting with a free initial consultation, help with the introduction of a QM system, study planning and implementation through to technical documentation - always with primary reference to the clinical data on the product: from the beginning to the end End.

Do you already have some initial questions?

You can get a free initial consultation here: free initial consultation

How is a clinical assessment based on performance data created?

Abbreviations

MDR Medical Device Regulation; EU Regulation 2017/745

PMCF Post-Market Clinical Follow-up, clinical follow-up

CEP Clinical Evaluation Plan

CDP Clinical Development Plan

Underlying regulations

EU Regulation 2017/745 (MDR)

1 Introduction

As already described in the last blog post, the clinical evaluation for all medical devices - from Class I to Class III - is an essential step for every manufacturer of medical devices. This is derived from Article 61 of EU Regulation 2017/745 (MDR):

“The manufacturer shall determine and justify the scope of clinical evidence to demonstrate compliance with the relevant general safety and performance requirements. The level of clinical evidence must be appropriate to the characteristics of the device and its intended purpose. To this end, manufacturers shall carry out, plan and document a clinical assessment in accordance with this Article and Part A of Annex XIV."

If the “performance data” route was defined during planning in the CEP, all requirements for the process and for the creation of the clinical assessment that result from the MDR and also from MEDDEV 2.7/1 Rev. 4 must still be adhered to . How this works: This blog post provides the relevant answers .

2. The route via performance data

The way to demonstrate the clinical performance of a product through performance data has always been possible and remains so under the MDR (Article 61):

„If demonstration of compliance with essential safety and performance requirements based on clinical data is considered inappropriate, any such exception shall be based on the manufacturer's risk management and taking into account the specific characteristics of the interaction between the device and the human body, the intended clinical performance and the information provided by the manufacturer; this applies without prejudice to paragraph 4. In this case, the manufacturer shall duly justify in the technical documentation set out in Annex II why he demonstrates compliance with essential safety and performance requirements solely on the basis of the results of non-clinical testing methods, including performance evaluation, technical testing ( “bench testing”) and preclinical evaluation, is considered suitable .“

The decision is based on various aspects:

• the result of risk management
• the characteristics of the interaction between product and body
• proof of performance based on product evaluations (technical, in-vitro)
• the result of the preclinical assessment (initial literature search, verification tests, etc.)

This decision must be appropriately explained and documented in the clinical evaluation plan.

This route is preferred when a clinical trial offers little benefit. A typical example of this is the wooden tongue depressor, for which clinical data does not exist in the literature. In such cases, technical data such as breaking strength and workmanship indicate the safety and performance of the product.

As the equivalence route becomes less and less possible and applicable, it is becoming more and more the new standard based on performance data if there is no need to generate your own clinical data.

Below are examples of when this route makes sense:

2.1 Example – Medical Software

Most software products (Class I and IIa) are examples of products where performance data makes sense. The reasoning for this decision is as follows:

The product has been fully verified as part of the software life cycle process in accordance with IEC 62304 and all tests have been successfully completed. The testing included unit testing, integration testing, system testing and usability testing. Based on these tests, it can be shown that the product works effectively.

According to MDCG-2020-1 (Guidance on Clinical Evaluation (MDR)/Performance Evaluation (IVDR) of Medical Device Software), scientific validity is defined as the extent to which the output of the software product is valid based on the selected inputs and algorithms is associated with the desired physiological state or clinical disease. In order to provide proof of scientific validity, a literature search is carried out, which also includes proof of benefit according to the MDR as well as determining the state-of-the-art and identifying the safety and performance of the medical device.

The clinically relevant components of the system are the implementations of the algorithms/questionnaires for diagnosis or the course of therapy. The literature search focuses on scores/detection algorithms as well as on the general use of digital products in the diagnosis/therapy of the indicated indications.

Table 1: Clinical evaluation of a software product

2.2 Example – dentist chair

Another product whose clinical performance, safety and benefits can be easily assessed using performance data and for which a clinical test makes no sense is the dental treatment unit: the dental chair.

Such products are active medical devices that are used to treat children and adults in the dental field. These products are dental treatment devices according to ISO 7494 with a dental patient chair according to ISO 6875. They are intended exclusively for use in dentistry and may only be operated by medical professionals. The dental treatment unit is used as an aid for patient positioning and for treatment in the dental field. Depending on whether dental instruments are part of this treatment unit and, if so, which ones, these products are classified in class IIa or IIb.

Due to the clear intended purpose of these products, the question of whether a clinical trial should be carried out on humans is unnecessary. The claims about the product relate to the ergonomics for both the patient and the practitioner and user of the product. It also emphasizes efficiency and ease of operation, and prescribed procedures and supporting components to facilitate infection control and maintain water quality. These statements are not suitable endpoints for a clinical trial. However, they can be supported with performance data. For example, the topic of ergonomics and ease of use can be proven via the usability test (DIN EN 62366-1). Compliance with the relevant standards and regulations on water hygiene and quality also confirms these claims about the product. The reason for choosing the path based on performance data is now listed here in Table 2:

Table 2: Clinical evaluation of an active product

2.3 Example – Heart rhythm detector

Another example is a Class IIa product that can detect episodes of irregular heart rhythm suggestive of atrial fibrillation through long-term monitoring of pulse parameters over several days to four weeks. It therefore supports the diagnosis by providing evidence of atrial fibrillation.

This product is based on embedded software whose algorithm recognizes the episodes and displays them accordingly. The verification and validation of the software already provides crucial data on how this medical device works. Despite the possibility of conducting a clinical trial on humans, ethical concerns must also be taken into account. An ethics committee examines precisely these aspects. However, there are alternative ways to generate clinical data to support the clinical performance and function of the product. For example, episodes can be played via simulation tests to check whether the algorithm recognizes them correctly. Here, too, no human study is required to provide this proof. The rationale for this route is shown in the table below:

Table 3: Clinical evaluation of cardiac rhythm detector

2.4 Example – Dental Implant

Even with an implantable product, this can be a viable option, as our last example from dental technology shows: The titanium base is part of a dental implant, a class IIb implantable medical device. The titanium base is used to create an individually manufactured implant prosthetic structure. After bonding with a CAD/CAM milled structure, it represents the connecting element to the implant. It can also be sold individually, so that a clinical evaluation must also be prepared for this product.

When conducting a literature search in the field of dental implants, you quickly come across the limitations of such system components. There is still no human study that has exclusively examined the titanium base as a test product. Only in vitro studies or studies on material properties (titanium), etc. were published. How the choice of route based on performance data is justified in this case is shown in Table 4:

Table 4: Clinical evaluation titanium base

3. Design and structure of a clinical assessment based on performance data

A clinical assessment based on performance data has essentially the same structure and design as a traditional clinical assessment of the other two routes. It therefore also includes carrying out a literature search.

The difference is that there is a more extensive section on existing performance/verification data and a section on justification in accordance with Art. 61 (10) of the MDR. This means that this route must be adequately justified on the basis of the manufacturer's risk management and taking into account the specific characteristics of the interaction between the device and the human body, the intended clinical performance and the manufacturer's claims. This is documented in the form of the examples above in a dedicated section of the clinical assessment.

4. Approach this route based on performance data

A clinical evaluation based on performance data therefore begins with a detailed consideration of the preclinical data, also known as verification or performance data.

This data forms the database for this route and provides important information about the safety and performance of the medical device. This is the subject of further evaluation.

4.1 Measurable parameters

When creating a clinical assessment based on performance data, it is therefore just as important as with the other routes to create a list of claims about the product, particularly in relation to performance:

This means a “ non-exhaustive list and specification of the parameters for determining, based on the latest medical knowledge, the acceptability of the benefit-risk ratio for the various indications and the intended purpose or intended purposes of the product ”. (Appendix XIV, Part A, 1(a) of the MDR)

These claims should therefore contain measurable parameters that are derived primarily from the performance data. As already mentioned, these data provide important information about the safety and effectiveness of the medical device and are the basis for further assessment. This data must be documented, analyzed and then thoroughly evaluated.

Proof of measurable parameters is an important part of the clinical evaluation of a medical device. It results from clinical data in the state-of-the-art part of the clinical evaluation as well as the verification/performance data on the product. This data forms the basis for assessing the safety and performance of the product. By carefully analyzing and evaluating this data, informed conclusions can be drawn about the clinical performance and safety of the product.

4.2 Similar Products

When creating a clinical evaluation based on performance data, consideration of similar products plays an important role. The manufacturer should prepare a list of similar products and check whether clinical data are available for these products. This data can provide relevant information about the safety and performance of the device being evaluated. It is important that the manufacturer conducts an appropriate search of the scientific literature, in this case the state-of-the-art literature search (see below), to ensure that all relevant data is taken into account.

If clinical data for similar products are available, they should be included and evaluated in the clinical evaluation. This data may be particularly relevant for post-market surveillance/PMCF planning (MDCG 2020-13)

4.3 Literature search

The literature search also plays an essential and important role when creating a clinical evaluation based on performance data:

The focus here should be on the state of the art, taking into account data on similar or benchmark products, alternative applications and outcomes, as well as measurable parameters.

As already mentioned, a list of claims about the product must be drawn up, particularly in relation to performance, containing measurable parameters derived primarily from the performance data.

Based on the list of similar products created, the literature search is carried out to check whether clinical data for these products are available. To do this, an appropriate search of the scientific literature must be carried out to ensure that all relevant data is taken into account.

As a rule, there is little or no data about the product, which is why state-of-the-art data plays the biggest role in this route. In addition to possible data on similar products, these also refer to alternative applications and their outcomes, again in relation to the measurable parameters.

Not only can measurable parameters be derived from this, but the results of the product can also be discussed based on the performance data in relation to the measurable parameters in comparison with the state of the art.

A thorough literature search that focuses on state-of-the-art data can thus help evaluate the safety and performance of the product in the context of current scientific knowledge. This can be achieved by comparing the product's performance data with that of similar or benchmark products and by considering alternative applications and their outcomes.

It is important that an appropriate search of the scientific literature is carried out to ensure that all relevant data is taken into account. The results of the literature search should be carefully documented and analyzed in order to provide an informed discussion of the safety and performance of the product compared to the state of the art and to draw appropriate conclusions .

5. Conclusion

The performance data route is becoming more and more standard alongside clinical assessment with your own clinical data. The focus here is on the performance data and measurable parameters must be defined, which are mainly derived from the performance data. This data is important for assessing the safety and performance of the medical device. Here too, the basis is a comprehensive literature search, but with a focus on state of the art, similar products and applications.

A thorough literature search that focuses on state-of-the-art data can thus help evaluate the safety and performance of the product in the context of current scientific knowledge. This can be achieved by comparing the product's performance data with that of similar or benchmark products and by considering alternative applications and their outcomes.

It is important that an appropriate search of the scientific literature is carried out to ensure that all relevant data is taken into account. The results of the literature search should be carefully documented and analyzed in order to provide an informed discussion of the safety and performance of the product compared to the state of the art and to draw appropriate conclusions.

This route also results in a consistent and conclusive clinical assessment in accordance with the MDR requirements.

6. How we can help you

As CRO, we support you throughout the entire process of generating and evaluating clinical data and in the approval and market monitoring of your product. And we start with the clinical strategy! We also create the complete clinical evaluation file for you.

In the case of clinical trials, we consider together with you whether and, if so, which clinical trial needs to be carried out, under what conditions and in accordance with what requirements. We clarify this as part of the pre-study phase: In 3 steps, we determine the correct and cost-effective strategy with regard to the clinical data collection required in your case.

If a clinical trial is to be carried out, basic safety and performance requirements must first be met. The data from the clinical trial then flow into the clinical evaluation, which in turn forms the basis for post-market clinical follow-up (PMCF) activities (including a PMCF study if necessary).

In addition, all medical device manufacturers require a quality management system (QMS), including when developing Class I products.

We support you throughout your entire project with your medical device, starting with a free initial consultation, help with the introduction of a QM system, study planning and implementation through to technical documentation - always with primary reference to the clinical data on the product: from the beginning to the end End.

Do you already have some initial questions?

You can get a free initial consultation here: free initial consultation

What routes of clinical evaluation of medical devices are there?

At medXteam, the focus is on clinical data. In this context, as CRO, we not only carry out clinical trials with medical devices in accordance with MDR and ISO 14155, but also offer all other options and forms of data collection and product approval as well as market surveillance. The focus is always on clinical evaluation, both during product approval and during clinical follow-up. But how can a clinical evaluation be carried out? What options are there to provide clinical evidence? And what role do the different routes of clinical assessment play in this? In this blog post we explore these questions, explaining in particular what the three routes of clinical evaluation mean, when they can be used and how they impact different product groups.

Abbreviations

MDR Medical Device Regulation; EU Regulation 2017/745

PMCF Post-Market Clinical Follow-up, clinical follow-up

CEP Clinical Evaluation Plan

CDP Clinical Development Plan

Underlying regulations

EU Regulation 2017/745 (MDR)

1 Introduction

Clinical evaluation is an essential step for every medical device manufacturer. It is necessary to create a comprehensive Clinical Evaluation Report (CER) for each medical device, which includes a thorough literature search. This was already standard procedure before Regulation (EU) 2017/745 (MDR) came into force. According to Article 61 of the MDR, the planning and implementation of a clinical evaluation for all medical devices - from Class I to Class III - is required:

“ The manufacturer shall determine and justify the scope of clinical evidence to demonstrate compliance with the relevant essential safety and performance requirements. The level of clinical evidence must be appropriate to the characteristics of the device and its intended purpose. For this purpose, manufacturers shall carry out, plan and document a clinical assessment in accordance with this Article and Part A of Annex XIV .” (See Article 61 of the MDR)

This process begins early in the development process. The clinical evaluation plan (CEP) is usually created shortly after the product idea, intended purpose and initial risk analysis of the medical device have been determined.

During planning in the CEP, the route of which data should be included for the clinical assessment is determined. This includes initial literature research depending on the defined product as well as a market assessment with regard to similar products and possibly existing clinical data in publications and the state of the art in the area of ​​application of the medical device.

This information makes it possible to define a clinical strategy for the product and record this in the clinical development plan (CDP).

Early timing is critical as the clinical strategy and resulting route for clinical evaluation have a significant impact on the time and cost of the entire development project. It makes a significant difference whether a clinical trial still has to be integrated into the validation part. This can delay the completion of the conformity assessment procedure and the CE marking of the medical device for years.

Early planning is also important because the intended purpose can still change. Since this forms the basis of the development process, changes made at an advanced stage can have a significant impact on the time and cost of the project. (see also our blog post on clinical strategy)

Therefore, every manufacturer should address the following questions as early as possible:

What product class does the medical device have? In our experience, for implantable products in Class IIb and all Class III products, it is essential to use your own clinical data.

What is the difference to existing products? The degree of innovation of the product is crucial here.

This blog post provides the answers to these questions.

2. The 3 routes of clinical assessment

According to the MDR, clinical evaluation refers to a structured and planned process for the ongoing generation, collection, analysis and evaluation of clinical data of a product in order to verify its safety, performance and clinical benefit when used as intended by the manufacturer (MDR Art. 2, Sentence 44). Clinical data is defined as follows: “Clinical data” is information about the safety or performance of a product that is obtained during its use and can come from various sources (MDR Art. 2, Sentence 48):

Clinical studies of the product in question.

• Clinical studies or other studies in the scientific literature that can demonstrate similarity to the product in question.
• Reports of clinical experience with the product or similar products that have been published in the scientific literature after peer review.
• Clinically relevant information from post-marketing surveillance, including post-marketing clinical follow-up.

This results in three possible routes for clinical assessment:

Own clinical data: This route involves conducting a clinical trial with the product in question in accordance with Article 62 of the MDR, which requires careful planning and execution.

Clinical data on equivalent products: Clinical data on similar products from the specialist literature are used here or there is already a clinical study with an equivalent product.

Use of performance data: This route is used when a human clinical trial is not possible or appropriate. Instead, performance data, also called verification data, is used. This data is based on non-clinical testing methods, including performance evaluation, technical testing and pre-clinical evaluation.

Fig. 1 The three routes of clinical assessment

It is important to note that the third route, although specified in the MDR, was already similarly established in Directive 93/42/EEC, MDD. The following sections describe all three routes in detail, with particular attention to the third route.

2.1 Own clinical data

For class IIb implantable medical devices in particular, the generation of own clinical data is the predominant method under the MDR. While under Directive 93/42/EEC the clinical assessment for these products could still be carried out using clinical data on equivalent products, this approach is under no longer possible due to the massively stricter requirements of the MDR. In particular, the requirement to conclude a contract with the manufacturer of the potentially equivalent product in order to obtain full access to its technical documentation (MDR, Art. 61, Section 5) completely excludes the option of using equivalent products:

„A manufacturer of a product which is proven to be similar to a product which has already been placed on the market and which he has not manufactured may also rely on paragraph 4 in order to avoid having to carry out a clinical trial, provided that, in addition to the requirements of that paragraph, the following conditions are met : – The two manufacturers have concluded a contract in which the manufacturer of the second product is expressly permitted unrestricted access to the technical documentation throughout, (…)“

This proprietary clinical data route is mandatory not only for Class IIb and Class III implantable devices, but also for innovative devices with clinical claims regarding the benefit or effectiveness of the device. There are usually no equivalent products for such innovative products, and the performance/verification data route cannot be chosen either, as clinical claims must necessarily be proven by own clinical data.

A concrete example would be a product whose clinical benefit is to reduce pain or improve quality of life. The choice of route for clinical evaluation depends on the degree of innovation of the product, regardless of its classification. This can even apply to Class I products.

2.2 The Equivalence Route

Under Directive 93/42/EEC, MDD or before the introduction of the MDR, the equivalence route was considered the standard procedure - the so-called gold standard - for clinical assessments. However, if one wishes to use clinical data on an equivalent product to support claims about the safety, clinical performance, and clinical benefit of one's product, one must first determine through a literature search whether any clinical data on that product is available. If this is not the case, a mandatory equivalence assessment is not possible. If there is data on this potential equivalent product, then in such a case it is first analyzed whether the potential equivalent product is actually equivalent. Previously, evaluation criteria were used for this analysis, which were set out in the MEDDEV 2.7/1 Rev. 4.3 guide for clinical assessments until the MDR came into force in May 2021.

These criteria targeted the clinical, technical and biological characteristics of the equivalent product, which were compared with the own product to determine whether they are the same or just similar in some aspects. For example, they may have had to be used for the same indications (clinical characteristics), while technical characteristics such as diameter and size could be similar.

With the introduction of the MDR and the associated MDCG document 2020-05 (“Clinical Evaluation – Equivalence: A guide for manufacturers and notified bodies”), these criteria were drastically tightened. Particularly with regard to technical and biological equivalence, the products now have to be identical in terms of their characteristics much more often than before. For example, assessing the equivalence of one software as a medical device may require access to complete algorithms and source codes of the other software, in which case these features would have to be identical. In the case of material medical devices, both products must consist of exactly the same substances and be present in the same concentration, and the product residues must also be identical.

Such detailed data on the potential equivalent product is usually not available because no one has access to such details of a software or exact material concentrations and residues of a product. And that is exactly what makes the equivalence route increasingly difficult or even impossible.

2.3 Performance data

The way to demonstrate the clinical performance of a product through performance data has always been possible and remains so under the MDR (Article 61):

„If demonstration of compliance with essential safety and performance requirements based on clinical data is considered inappropriate, any such exception shall be based on the manufacturer's risk management and taking into account the specific characteristics of the interaction between the device and the human body, the intended clinical performance and the information provided by the manufacturer; this applies without prejudice to paragraph 4. In this case, the manufacturer shall duly justify in the technical documentation set out in Annex II why he demonstrates compliance with essential safety and performance requirements solely on the basis of the results of non-clinical testing methods, including performance evaluation, technical testing ( “bench testing”) and preclinical evaluation, is considered suitable .“

The decision is based on various considerations:

• the result of risk management
• the specifics of the interaction between body and product
• proof of performance based on product tests (technical, in-vitro)
• the result of the preclinical assessment (initial literature search, verification tests, etc.)

This decision must be adequately justified and documented in the clinical evaluation plan.

This route is chosen when a clinical trial makes little sense. A classic example of this is the wooden tongue depressor, for which clinical data are not available in the literature. In such cases, technical data such as breaking strength and workmanship prove the safety and performance of the product.

Although this route has been less used in the past because it was often less well known and the route via an equivalent product was commonly used, it is suitable for a wide range of products.

2.3.1 Example – Medical Software

Most software products (Class I and IIa) are examples of products where performance data makes sense. The reasoning for this decision is as follows:

The product has been fully verified as part of the software life cycle process in accordance with IEC 62304 and all tests have been successfully completed. The testing included unit testing, integration testing, system testing and usability testing. Based on these tests, it can be shown that the product works effectively.

According to MDCG-2020-1 (Guidance on Clinical Evaluation (MDR)/Performance Evaluation (IVDR) of Medical Device Software), scientific validity is defined as the extent to which the output of the software product is valid based on the selected inputs and algorithms is associated with the desired physiological state or clinical disease. In order to provide proof of scientific validity, a literature search is carried out, which also includes proof of benefit according to the MDR as well as determining the state-of-the-art and identifying the safety and performance of the medical device.

The clinically relevant components of the system are the implementations of the algorithms/questionnaires for diagnosis or the course of therapy. The literature search focuses on scores/detection algorithms as well as on the general use of digital products in the diagnosis/therapy of the indicated indications.

Table 1: Clinical evaluation of a software product

2.3.2 Example – dentist chair

Another product whose clinical performance, safety and benefits can be easily assessed using performance data and for which a clinical test makes no sense is the dental treatment unit: the dental chair.

Such products are active medical devices that are used to treat children and adults in the dental field. These products are dental treatment devices according to ISO 7494 with a dental patient chair according to ISO 6875. They are intended exclusively for use in dentistry and may only be operated by medical professionals. The dental treatment unit is used as an aid for patient positioning and for treatment in the dental field. Depending on whether dental instruments are part of this treatment unit and, if so, which ones, these products are classified in class IIa or IIb.

Due to the clear intended purpose of these products, the question of whether a clinical trial should be carried out on humans is unnecessary. The claims about the product relate to the ergonomics for both the patient and the practitioner and user of the product. It also emphasizes efficiency and ease of operation, and prescribed procedures and supporting components to facilitate infection control and maintain water quality. These statements are not suitable endpoints for a clinical trial. However, they can be supported with performance data. For example, the topic of ergonomics and ease of use can be proven via the usability test (DIN EN 62366-1). Compliance with the relevant standards and regulations on water hygiene and quality also confirms these claims about the product. The reason for choosing the path based on performance data is now listed here in Table 2:

Table 2: Clinical evaluation of an active product

2.3.3 Example – Heart rhythm detector

Another example is a Class IIa product that can detect episodes of irregular heart rhythm suggestive of atrial fibrillation through long-term monitoring of pulse parameters over several days to four weeks. It therefore supports the diagnosis by providing evidence of atrial fibrillation.

This product is based on embedded software whose algorithm recognizes the episodes and displays them accordingly. The verification and validation of the software already provides crucial data on how this medical device works. Despite the possibility of conducting a clinical trial on humans, ethical concerns must also be taken into account. An ethics committee examines precisely these aspects. However, there are alternative ways to generate clinical data to support the clinical performance and function of the product. For example, episodes can be played via simulation tests to check whether the algorithm recognizes them correctly. Here, too, no human study is required to provide this proof. The rationale for this route is shown in the table below:

Table 3: Clinical evaluation of cardiac rhythm detector

2.3.4 Example – Dental Implant

Even with an implantable product, this can be a viable option, as our last example from dental technology shows: The titanium base is part of a dental implant, a class IIb implantable medical device. The titanium base is used to create an individually manufactured implant prosthetic structure. After bonding with a CAD/CAM milled structure, it represents the connecting element to the implant. It can also be sold individually, so that a clinical evaluation must also be prepared for this product.

When conducting a literature search in the field of dental implants, you quickly come across the limitations of such system components. There is still no human study that has exclusively examined the titanium base as a test product. Only in vitro studies or studies on material properties (titanium), etc. were published. How the choice of route based on performance data is justified in this case is shown in Table 4:

Table 4: Clinical evaluation titanium base

2.3.5 Conclusion from the examples

In all of these examples, the section on the clinical assessment of the state of the art is also very important. Many product features or functions and in many cases also the clinical benefit can be proven via guidelines, technical documents and standards. What also helps in these examples is the data collection as part of the clinical follow-up (Post-Market Clinical Follow-up, PMCF) - after the product has been placed on the market and bears the CE mark. A clinical assessment based on performance data usually results in measures within the framework of clinical follow-up. These can range from focused literature searches and product registers to application observations and PMCF studies. This makes it possible to specifically close gaps that have not yet been fully documented in the performance data. Such an approach is also recognized and accepted by the notified bodies if the reasons are correct.

3. Conclusion

To date, for many medical devices the equivalence route and the use of clinical data on one or more equivalent devices have been chosen, regardless of the class of the medical device. However, this has changed completely when the MDR came into force. Due to stricter regulations, especially for implantable and Class III devices, this route is hardly possible anymore. This is due to both the difficulty in proving equivalence and the specific regulations, such as the conclusion of a contract between the manufacturers (MDR, Art. 61 Paragraph 5). This change was probably the desired goal of the makers of MDR.

It is therefore crucial to carry out initial literature searches and rethink the clinical strategy at the beginning of the development process. This enables a comprehensive view of the data situation and the state of the art for the product. Determining the intended purpose at an early stage can mean that the route can be taken via performance data, which is now becoming increasingly important and is being used in more and more products. The examples in this article show that this is possible if it can be justified. Nevertheless, a literature search should not be neglected even in a clinical evaluation based on performance data. Data on the state of the art, guideline recommendations and technical standards make a significant contribution to the assessment.

4. How we can help you

As CRO, we support you throughout the entire process of generating and evaluating clinical data and in the approval and market monitoring of your product. And we start with the clinical strategy! We also create the complete clinical evaluation file for you.

In the case of clinical trials, we consider together with you whether and, if so, which clinical trial needs to be carried out, under what conditions and in accordance with what requirements. We clarify this as part of the pre-study phase: In 3 steps, we determine the correct and cost-effective strategy with regard to the clinical data collection required in your case.

If a clinical trial is to be carried out, basic safety and performance requirements must first be met. The data from the clinical trial then flow into the clinical evaluation, which in turn forms the basis for post-market clinical follow-up (PMCF) activities (including a PMCF study if necessary).

In addition, all medical device manufacturers require a quality management system (QMS), including when developing Class I products.

We support you throughout your entire project with your medical device, starting with a free initial consultation, help with the introduction of a QM system, study planning and implementation through to technical documentation - always with primary reference to the clinical data on the product: from the beginning to the end End.

Do you already have some initial questions?

You can get a free initial consultation here: free initial consultation

Why is clinical strategy so important in medical device development?

 At medXteam, the focus is on clinical data. In this context, as CRO, we not only carry out clinical trials with medical devices in accordance with MDR and ISO 14155, but also offer all other options and forms of data collection and product approval as well as market surveillance. Right from the very beginning of a product idea, but also with regard to the MDR transfer of existing products, the clinical strategy plays an important role. It not only paves the way for the collection and evaluation of the necessary clinical data, but is also the basis for time and cost planning. This blog post looks at exactly this fundamental role: What is the clinical strategy and why is it so crucial?

Abbreviations

MDR Medical Device Regulation; EU Regulation 2017/745

PMCF Post-Market Clinical Follow-up, clinical follow-up

CEP Clinical Evaluation Plan

CDP Clinical Development Plan

Underlying regulations

EU Regulation 2017/745 (MDR)

1 Introduction

The development of a new product or the MDR transfer of an existing product presents medical device manufacturers with extensive challenges. One of these is the early development of a clinical strategy. The MDR explicitly requires the creation of a clinical development plan in Annex XIV. This plan is intended to cover all phases from idea to market and beyond, including exploratory studies, feasibility studies, pilot studies, confirmatory studies such as pivotal clinical trials and post-marketing clinical surveillance. The definition of milestones and the description of possible acceptance criteria are essential components.

The importance of such a clinical strategy can be attributed to several key factors. On the one hand, it enables clear structuring and planning of the development process. By defining goals and criteria at an early stage, development time and costs can be controlled efficiently and potential risks can be minimized. On the other hand, the clinical strategy serves not only to meet regulatory requirements, but also to ensure that the product will later bring the greatest possible benefit to patients. It supports manufacturers in making evidence-based decisions at an early stage and in being able to classify the product in terms of its clinical performance, safety and benefit.

In addition, the clinical strategy provides valuable input for risk management. A key part of the strategy includes conducting a comprehensive literature search to integrate existing knowledge about similar products or technologies and identify potential risks at an early stage. This information is crucial to assess, manage and ultimately minimize risks.

How this works and what needs to be taken into account is described below.

2. Product idea, start of development or new beginning with the MDR

Regardless of whether it is a new product, an idea for a new product has been born or the existing product needs to be adapted to the MDR requirements, it is always necessary to check which data and clinical data are required to create the initial clinical assessment become. Once this has been created, it must then be updated regularly and clinical data must be collected in the area of ​​clinical follow-up (Post-Market Clinical Follow-up, PMCF). In addition, a so-called clinical development plan (CDP) must be created in the clinical evaluation plan (CEP), which basically contains exactly that, namely which data are needed and must be collected. In order to be able to determine this, a clinical strategy is needed.

2.1 What is the clinical strategy?

The clinical strategy is a comprehensive, systematic approach that guides the identification, collection, analysis and updating of clinical data throughout the entire life cycle of a medical device. It is used to evaluate and document the safety, effectiveness and performance of the product, both during initial development and when adapting to regulatory requirements such as the Medical Device Regulation (MDR). The core of the clinical strategy is the creation and regular updating of an initial clinical assessment. This evaluation requires careful consideration of what specific data and clinical evidence is needed to support the benefits and risks of the product.

The clinical strategy forms the basis for the Clinical Evaluation Plan (CEP), which includes a Clinical Development Plan (CDP). This CDP defines in detail what data and clinical evidence is required to create the clinical assessment and how it should be collected. This includes both the planning and execution of pre-market studies (such as first-in-man studies, feasibility studies, pilot studies and pivotal clinical trials) as well as the ongoing collection of post-market data through Post-Market Clinical Follow-up (PMCF ) Measures.

The clinical strategy is therefore the basis for the continuous collection of (clinical) data throughout the entire product life cycle. By planning clinical data and assessments early and systematically, the clinical strategy helps to minimize risks, increase development efficiency, identify existing gaps and ultimately accelerate the market launch or MDR transfer of medical devices.

2.2 Building a clinical strategy

The development of a clinical strategy is a complex and indispensable process that covers wide-ranging areas and is deeply embedded in the planning and implementation of the development and evaluation of medical devices. The methodological structure and content of such a strategy are discussed in detail below. The clinical strategy is a comprehensive concept that covers various aspects to ultimately enable well-founded conclusions regarding the product.

2.2.1 Product description with intended purpose

The clinical strategy begins with the detailed product description. This covers the intended purpose of the product, including the technical features as well as the intended indications and contraindications. In addition, the target groups, i.e. patients and users, are defined. This initial classification is fundamental in order to correctly position the product in the medical context and forms the basis for all further steps.

2.2.2 Specific development process

The specific development process for the product is then shown. The relevant performance and safety regulations in accordance with Annex I of the MDR are taken into account. This means that the product is classified further in order to be able to draw the right conclusions for further action.

2.2.3 Documents and evidence

Another essential component of the clinical strategy is the definition of the documents to be created and the evidence required in accordance with Annex II and III of the MDR for the technical documentation of the product. These documents include in particular the necessary documents and test procedures for verification and validation of the product, as this may be important for deciding on the correct route of clinical evaluation.

2.2.4 Identification of similar products

An important step within the clinical strategy is the identification of similar or even equivalent products. This analysis makes it possible to use existing knowledge and data to support the development and evaluation of your own product. By comparing with similar products or applications in the same area of ​​application, it is possible to better estimate what clinical data is already available or required and how it can best be collected.

2.2.5 Literature and safety database search

The clinical strategy then naturally includes a literature search and a search in safety databases for similar or equivalent products. This research serves to record the current state of the art and to ensure that all relevant clinical data and information about safety aspects and similar applications, the area of ​​application of the medical device, etc. are taken into account.

2.2.6 Classification of the product in the application context

The targeted classification of the product in its application context is based on the knowledge gained from the previous steps. In particular, the comprehensive literature search makes a significant contribution to this. By analyzing existing data on similar or equivalent products and assessing the current state of the art, important insights into the practical application conditions and the needs of the target group can be gained.

The integration of this diverse information makes it possible to classify the product in the application context. This not only takes into account the theoretical intended purpose of the product, but also reflects its use in real clinical or home environments. This allows a realistic assessment of product performance under typical application conditions, which in turn optimizes product development and evaluation.

2.3 Conclusions from the clinical strategy

The careful development of a clinical strategy for medical devices brings with it far-reaching conclusions that are crucial to the direction of the product and its explicit strategy. These conclusions not only provide guidance for the clinical evaluation and development process, but also help to optimally prepare the product for market and use or for MDR transfer.

2.3.1 Foundation for the clinical assessment route

The clinical strategy lays the foundation for the clinical evaluation route and strategically aligns the planning of the development process. There are three possible ways to do this:

1. Clinical assessment using your own clinical data

The decision to conduct our own clinical trials is necessary if no alternative routes via performance data or data on equivalent products are viable, or if the product has innovative clinical claims that must be clearly supported with clinical data. This approach often involves higher costs and a longer time frame.

2. Performance/verification data (MDR Art. 61(10))

Using verification and performance data, as well as additional literature on similar products, is another possible route whenever direct demonstration of compliance with essential safety and performance requirements based on clinical data is deemed inappropriate. This approach is typically time and cost effective, but requires sound justification as to why it is considered appropriate.

3. Data on equivalent products

The use of data on equivalent products is another possibility, provided such products exist and clinical data on them are available. This route has long been the gold standard of clinical evaluation. Due to the increased MDR requirements in particular

• implantable class IIb and
• all Class III products

as well as increased requirements for proof of equivalence, e.g. B. for software, this hardly takes place anymore. If this route is nevertheless possible, the need for own clinical data can of course be avoided, which of course potentially accelerates the development process.

2.3.2 Identification of further risks and side effects

The comprehensive collection and analysis of data as part of the clinical strategy helps to identify further, previously unknown risks and side effects. These insights are also of great value for risk management and enable a more precise assessment of alternative uses of the product. By identifying potential risks early, measures to minimize risks can be implemented and the safety of the product for users and patients can be maximized.

2.3.3 Identification and closure of gaps in clinical data

Another critical aspect of the clinical strategy is the identification of possible gaps in the required clinical data. The early detection of such deficits makes it possible to take targeted measures to adequately close these gaps. This can be done through additional in-house data collection (e.g. planning a clinical trial, as a clinical claim cannot yet be proven using clinical data), the collection of post-market or PMCF data or the re-analysis of existing data.

3. Literature search for clinical strategy

On the one hand, the literature search as the core of the clinical strategy makes it easier to classify the product into its area of ​​application. It also leads to the identification of potential risks and helps in the development of risk mitigation strategies. It ultimately supports the formulation of a long-term clinical and development strategy based on the exact data it collects.

The literature search process takes place in several steps:

Fig. 1: Literature search process

Defining the search strategy : The first step is to carefully plan the search strategy. Relevant keywords and search terms are defined that form the framework of the research.

Choosing the right databases : Because of the wealth of information, choosing the right databases is crucial. Each database has its own strengths and specializations that need to be taken into account.

Carrying out the search : The databases are systematically searched using the defined keywords. This phase requires patience and care to ensure that no relevant information is missed.

Analysis and selection of data : After collecting the information, the results are critically evaluated. The most relevant and well-founded studies and reports are selected that help answer the question.

The analysis of “state of the art” data required for the clinical strategy captures the current state of the art.

The literature search includes four steps:

Fig. 2: Literature search step by step

4. Conclusion

Developing and implementing a clinical strategy for medical devices is a crucial step that goes far beyond simply meeting regulatory requirements. It offers a structured approach to systematically evaluate and document the safety, effectiveness and performance of a product. This process makes a significant contribution to optimally adapting the product to the needs of users and patients while at the same time meeting the regulatory framework.

Through the detailed planning and analysis that a clinical strategy requires, medical device developers and manufacturers can make informed decisions that impact the entire life cycle of the product. From initial product conception through market launch to post-market monitoring, the strategy enables ongoing evaluation and adaptation of the product to changing clinical and regulatory requirements. This applies both to new developments and product ideas and to the MDR transfer of an existing product.

The choice of route for clinical evaluation, be it through own clinical data, performance data or data on equivalent products, lays the foundation for the development process and largely determines the time and costs involved. In addition, the systematic detection and assessment of risks and side effects as well as the identification and closure of gaps in the clinical data enable continuous improvement of the product and its (planned) use.

In summary, it can be said that the creation and implementation of a clinical strategy is not only a regulatory necessity, but also represents an opportunity to optimize medical devices and ensure their success in the market. It promotes an in-depth examination of the product and its application context, improves quality and safety for end users and patients and supports efficient product development. In an industry characterized by innovation and constant change, clinical strategy is a central pillar for the long-term success of medical devices.

5. How we can help you

As CRO, we support you throughout the entire process of generating and evaluating clinical data and in the approval and market monitoring of your product. And we start with the clinical strategy!

In the case of clinical trials, we consider together with you whether and, if so, which clinical trial needs to be carried out, under what conditions and in accordance with what requirements. We clarify this as part of the pre-study phase: In 3 steps, we determine the correct and cost-effective strategy with regard to the clinical data collection required in your case.

If a clinical trial is to be carried out, basic safety and performance requirements must first be met. The data from the clinical trial then flow into the clinical evaluation, which in turn forms the basis for post-market clinical follow-up (PMCF) activities (including a PMCF study if necessary).

In addition, all medical device manufacturers require a quality management system (QMS), including when developing Class I products.

We support you throughout your entire project with your medical device, starting with a free initial consultation, help with the introduction of a QM system, study planning and implementation through to technical documentation - always with primary reference to the clinical data on the product: from the beginning to the end End.

Do you already have some initial questions?

You can get a free initial consultation here: free initial consultation