Blog

Clinical research in the oral and maxillofacial area

At medXteam, the focus is on clinical data. In this context, as CRO we not only carry out clinical trials with medical devices in accordance with MDR and ISO 14155, but also offer all other options and forms of data collection. This time the topic in this context is clinical trials in the dental sector. Since this topic is very extensive, we have divided it into two parts. This Part 1 focuses in particular on the study types, design and special endpoints for medical devices used in dental studies.  

Abbreviations

MDR Medical Device Regulation; EU Regulation 2017/745

Underlying regulations

EU Regulation 2017/745 (MDR)
Medical Devices Implementation Act (MPDG)

Sources

www.ebm-netzwerk.de : accessed on August 26, 2024 at 7:49 a.m

BA Just, H Rudolph, R Muche: “Clinical trials in dentistry – and what lies behind them - Clinical Trials in Dentistry – What lies behind”. ZWR - The German Dental Journal 2012; 121(10): 478-486. DOI: 10.1055/s-0032-1330863

WV Giannobile, NP Lang, MS Tonetti, eds.: “Osteology guidelines for oral and maxillofacial regeneration: clinical research”. Quintessence Publishing, 2014.

1 Introduction

Clinical research in the oral and maxillofacial area does not differ in its basic principles from biomedical research in other areas of the human body.

Dental clinical studies can also generally be divided into experimental and epidemiological, prospective and retrospective studies based on the procedure (see also our previous blog posts).

2 .  Dental studies

2.1 Experimental studies

In an experimental study, an experiment (or therapy) is carried out repeatedly, with the number and selection method of study subjects (test participants, patients) as well as the type and extent of information to be collected being determined before the start of the study. The aim of an experimental study is generally to demonstrate causal relationships. In an epidemiological study (observational study), only repeated observations are made without intervening in the process; this type of study is also possible retrospectively. The aim of an epidemiological study is generally to identify and evaluate connections.

However, the following study design is specific to dentistry and is very popular for various reasons:

Split mouth design

The split mouth design is an experimental model in dentistry in which two or more different therapies are administered to a study participant in different areas of the oral cavity. As a rule, the form of therapy is randomly assigned to the area of ​​the oral cavity. This special form of study design eliminates the differences that exist between two patients. Each patient acts as a test and control at the same time. In contrast to study designs that compare different patients with each other, the split-mouth design improves the comparability of different forms of therapy, which may mean that the number of cases can be smaller. However, since the information content obtained is so-called “connected” data, special, “connected” statistical tests are necessary.

2.2 Validity of the data 

The principles of all clinical studies apply to the validity of dental studies: it is a qualitative measure of the validity of a research result - but not a mathematical measure like reliability. Here too, a distinction is made between internal and external validity. Internal validity stands for the clarity of the interpretation of the results and is influenced by systematic errors (bias), such as errors in the design of the study, its implementation, the data collection or errors in the evaluation and analysis of the results.

2.3 Evidence

The same rules apply to clinical studies in the oral and maxillofacial region as to other clinical studies for the classification into evidence classes, randomization, sample size planning and statistical evaluation.

2.4 Good Clinical Practice

GCP or “Good Clinical Practice” refers to internationally recognized, ethical and scientific quality standards and rules for the conduct of clinical studies on humans. Compliance with the GCP serves to ensure the protection and informed consent of study participants (ethical aspect) as well as the collection of credible, valid data. Of course, dental clinical studies also fall under “good clinical practice”.

2.5 Areas

Dentistry is a field in which many medical devices are used. Think, for example, of drills for removing carious lesions, filling materials, orthodontic aligners and also implants. Therefore, MDR and ISO14155 regulate the regulatory approach and conduct of clinical studies.

An area in which many clinical studies are located is periodontics and dentoalveolar surgery. The following sections therefore primarily refer to these two research fields.

3. Endpoints in clinical trials

The basis of clinical research is to establish a clinically relevant hypothesis that can be validated or refuted using scientific methods. With this method, a targeted and clinically relevant question can be systematically defined, measured and analyzed to obtain an answer that is reported in the form of endpoints or results. The results are therefore the consequences or effects of interventions in clinical studies as well as the effects of the biological processes examined in prospective observational studies.

An endpoint is the parameter or variable measured in an intervention or observational study; The result of this measurement provides the answer to the research question or the validity of the tested hypothesis. These endpoints may be assessed and assessed by the patient or subject themselves (patient-assessed endpoints) or by the investigator or clinician on specific aspects of disease progression or response to treatment (investigator-assessed endpoints). True endpoints are also defined as those that represent a tangible impact on the patient (e.g., tooth loss).

In the study methodology of clinical research, surrogate endpoints (intermediate endpoints) are understood to be endpoints that are not themselves of direct importance for the patient, but can represent important endpoints (e.g. reduction in blood pressure as a surrogate parameter for preventing a stroke). Surrogate endpoints are often physiological or biochemical markers that can be measured relatively quickly and easily and are considered to have a predictive function for later clinical events. The prerequisite for reliable statements about the effectiveness of a treatment is a close causal relationship between surrogate parameters and the actual endpoint. It is therefore expected that the measured significant changes in the surrogate outcome as a result of the tested intervention will also significantly influence the true endpoint. This answer is controversial in many respects, especially in the study and treatment of chronic diseases with multifactorial etiology, such as: B. Periodontitis, where evaluation of one aspect of the disease does not exclude a different outcome via a different pathway or the influence of other confounding factors not identified by the surrogate under study.

Clinical research results are also divided into “primary” and “secondary” results. Primary results are those that serve to answer the research question or validate the hypothesis being tested. They are therefore at the forefront of data analysis and serve to provide the conclusions of the study. They must also be used to calculate the sample size of the study. The ideal situation for clinical research would be to use real results as primary outcomes, but as previously mentioned, real results in clinical research are typically difficult to evaluate in short- to medium-term intervention or observational studies.

Secondary outcomes are typically measures of behaviors or lifestyles that significantly influence the outcome of the actual outcome (e.g., tobacco smoking, plaque control). Their assessment is therefore important for controlling the relevant factors that may influence the studied response to an intervention or the onset or progression of a disease process.

The results can also be divided into “qualitative” or “quantitative”. Quantitative results are those that can be expressed using numerical continuous variables, which can usually be subjected to parametric statistics. Qualitative outcomes are verbal or categorical representations of a non-quantifiable variable and can be further classified as nominal (e.g. gender) or ordinal if they can be expressed in categories (e.g. plaque index). Before being used in clinical research, any quantitative or qualitative variable must be evaluated for its validity and reliability in assessing the outcome under study, as well as for its sensitivity and specificity in representing a true result.

In oral clinical studies of tissue regeneration, both real endpoints and surrogate endpoints are used, depending on the question, to evaluate the effectiveness of treatments.

Of course, to ensure the quality of a clinical trial, the endpoints should be applicable to the vast majority of patients and diseases. They should also be clearly defined and easy to validate. Furthermore, high sensitivity/specificity is important for disease diagnosis and disease progression.

Below we present the endpoints and outcomes most commonly used in periodontology and oral surgery, with a focus on tissue regeneration.

3.1 Endpoints in periodontology

Endpoints in periodontal research are used to understand the periodontal disease process and examine the effectiveness of various therapeutic measures. In order to study the disease process, it is important to establish a clear case definition for the periodontal disease being studied (gingivitis, chronic periodontitis, aggressive periodontitis, etc.). Although there are various case definitions in the literature, the most widely accepted is the European Federation of Periodontology (2017) International Classification of Periodontal Diseases and Conditions).

Chronic periodontitis usually progresses slowly, and if no preventive or therapeutic measures are taken, its natural course eventually leads to tooth loosening and even loss of the tooth. However, this progression is usually slow, with periods of loss of periodontal attachment followed by periods of quiescence or even tissue regeneration, which depends on many factors (genetic susceptibility, lifestyle and behavioral risk factors, etc.) affecting the interactions between host and bacteria involved in the pathogenesis of tissue destruction.

When examining the various prevention and therapeutic approaches, the effectiveness of the various measures with regard to their influence on the periodontal attachment level is determined using various endpoints:

through

  • the prevention of attachment loss and thus the maintenance of periodontal health (prevention),
  • the interruption of the destructive disease process and
  • the maintenance of a healthy but reduced periodontium (cause-related therapy) or
  • through the application of regenerative technologies that aim to achieve a new attachment of the periodontium to a previously diseased root surface (regenerative therapies).

In periodontal research, there are two real endpoints: one is the histological evidence of loss of periodontal attachment, and the other is tooth loss, the end result of the disease process.

Histology is the only method available to detect periodontal regeneration and periodontal destruction. However, this technique is limited to preclinical research because for histological evaluation, the affected tooth must be removed in a block with the associated soft tissue for histological preparation. Nevertheless, histological results have historically been used in studies evaluating regenerative technologies. To demonstrate the extent of regeneration, new cementum and connective tissue attachment must be identified coronal to the apical extent of the disease process along the root. In addition to assessing the presence of new cementum and connective tissue attachment as a qualitative histological result to demonstrate periodontal regeneration, histometric analysis was used for quantitative microscopic tissue determination of the attachment (new cementum, connective tissue and epithelium). A notch made during the surgical procedure at the apical extent of the attachment loss was used as a fixed landmark. However, for obvious ethical reasons, these histological results can only be examined in experimental studies, so the evaluation of regenerative therapies must be done in human studies with surrogate results.

Another true end point is tooth loss, as it represents the definitive end of the disease process and the clear failure of any intervention trial. This endpoint is rarely used in clinical trials because this event is rare and takes a long time. However, its assessment is very important in long-term population studies, as well as in longitudinal studies to evaluate the long-term effectiveness of preventive and therapeutic measures, since it allows a true assessment of dental survival and allows the assessment of the risk factors that influence this outcome.

3.2 Primary surrogate endpoints in periodontology

As previously mentioned, the primary endpoints in periodontal research are the assessment of clinical attachment level by periodontal probing and bone level by radiographic examinations.

3.2.1 Periodontal probing

Periodontal probing is the most commonly used non-invasive diagnostic method to assess the progression of periodontitis and evaluate the level of periodontal attachment. This is usually done by carefully inserting the probe into the gingival sulcus and measuring the distance between a fixed reference point, the cemento enamel junction (CEJ), and the point where the probe will be inserted at a certain pressure ( about 25 g) (bottom of the sulcus or pocket). This measurement, called the clinical attachment level (CAL), is not always easy to evaluate because the CEJ is not always available for visual inspection when it is below the gum line. For this reason, the CAL level is usually determined together with the probing pocket depth (PPD) level and the recession level (REC). The PPD values ​​indicate the distance between the gingival margin and the floor of the sulcus/pocket, the REC values ​​indicate the distance between the gingival margin and the CEJ. The addition of PPD and REC expresses CAL; However, in health, gingivitis and early periodontitis there is no recession because the gingival margin is usually above or at the level of the CEJ, meaning that PPD and CAL have similar values. In periodontal research intervention studies, the three measurements (CAL, PPD and REC) must be recorded at baseline and after treatment to evaluate the effects of therapy on disease progression. In these studies, the primary outcome must therefore be the increase in clinical attachment and reduction in probing pocket depth.

Although periodontal probing is the most commonly used assessment method in periodontal research, this measurement has many sources of error that should be minimized in clinical examinations. Their validity and reproducibility depends on the inclination of penetration into the sulcus, on the force of insertion, on the ability to read the measurements correctly (usually within 1 mm), and on the accuracy of correctly transmitting the results. Various strategies to reduce this variability have been recommended, such as: B. the use of constant force probes, stents to guide the probe and electronic reading systems. Furthermore, to ensure the reproducibility of the exploratory measurements in any clinical trial, it is essential to perform calibration studies to ensure that inter-examiner variability is kept as low as possible. Ideally, a single calibrated operator should perform all measurements. If other examiners are used, it is even more important to conduct calibration studies between examiners.

In the past, in clinical trials for periodontal regeneration, intraoperative probing measurements were performed on the treated infraalveolar lesions. In this study design, baseline measurements are taken during the intervention once the defect is completely debrided (cleaned), and the distance between the cemento-enamel junction (CEJ) and the deepest point of the defect is recorded. To evaluate the outcome, a surgical re-entry is required to register this distance (CEJ low point of the defect) after lifting a flap. For obvious ethical reasons, these re-entry studies are now rarely performed unless the second surgical procedure is required to remove a non-absorbable barrier membrane (e.g. e-PFTE).

3.2.2 Bone level x-rays

The use of periapical radiographs is the most common method to detect changes in interdental alveolar bone position relative to a fixed reference point on the tooth (e.g. CEJ). This measurement, similar to the CAL value, provides important information when studying the progression of periodontal disease (loss of bone level), or when studying periodontal regeneration (increase in bone level), or in studies simply attempting to evaluate periodontal therapy , to stop the disease process (stability of the bone level). To detect these changes in bone level, two or more x-rays taken at different times must be compared. Similar to clinical measurements, the most commonly calculated distances are the distance between the CEJ and the deepest root-bone contact. The data is typically expressed in the form of linear measurements of bone formation or loss. Similar to periodontal probing, the validity and reproducibility of radiographic assessment of bone level is subject to many sources of error.

This includes:

  • the x-ray projection,
  • the position of the plate or sensor,
  • X-ray recording and processing as well as the
  • Examiner's ability to interpret the images.

In clinical research, it is therefore important to control these sources of variability by taking periapical radiographs with the correct parallelization technique and using individual radiographic film holders containing impressions of the patient's occlusal surfaces. This ensures a reproducible X-ray angle across the entire series of images. Most of the current X-ray diagnostic systems use digitized images that allow image correction and direct linear measurements by software, improving the reproducibility of these measurements. Ideally, the evaluation of bone level changes is done electronically via digital subtraction analysis; this requires very precise X-ray technology to enable correct overlay of the images.

Assessment of changes in bone level can also be done directly in clinical studies by measuring the distance between the CEJ and the deepest bone contact with the root surface (bone probing). This must be done intraoperatively, and then again at a later reintervention after lifting a flap and cleaning the residual defect. As mentioned above, these invasive reintervention procedures are not recommended for obvious ethical reasons. The use of study impressions of the defect to evaluate the three-dimensional changes in the lesion after the tested regenerative procedure is also conceivable. . Intrasurgical impressions should be taken both at the time of surgery after debridement of the defects and at the end of the study period (usually one year). These prints should provide information about:

(i) number of tooth surfaces affected;

(ii) the depth of the 1-, 2-, and 3-wall components of the defect; and

(iii) the defect perimeter, estimated as the width of the angle and measured to the nearest 30 degrees.

These endpoints also require a re-entry procedure and should be viewed critically for the same reasons as described above.  

3.3 Secondary surrogate endpoints

When conducting clinical trials in periodontics, there are several endpoints that do not necessarily evaluate the main objective of the study or the outcome of the treatment being tested, but which are known to have a secondary influence on the study outcome and which should therefore be evaluated and taken into account. The most commonly used secondary endpoints are plaque accumulation and gingival inflammation. Both measurements are linked, one provides information about the patient's compliance with oral hygiene measures (plaque accumulation) and the other about the degree of infection control (gingivitis), which is usually carried out in the therapy phase that occurs before periodontal regeneration therapy. Plaque accumulation can be measured, for example, with the Full Mouth Plaque Score (FMPS), which dichotomously assesses the presence of visible plaque in 4 to 6 locations per tooth (0, no visible plaque at the soft tissue edge; 1, visible plaque at the soft tissue edge). . It is expressed proportionally and good patient compliance is considered to be achieved when this value is below 15%. Similarly, gingival inflammation can be assessed using the full mouth bleeding score (FMBS), which assesses the presence of visible bleeding on probing in 4 to 6 sites per tooth. It is also expressed proportionally and it is estimated that adequate infection control is achieved when this value is below 15%.

There are other indices to evaluate plaque accumulation and inflammation of periodontal pockets (plaque index, gingival index, etc.), but these are mainly used for therapies aimed at reducing plaque and gingivitis and in which these indices become the main endpoint of the study, which is clearly not the case in regenerative studies.

Another important factor is the patient's smoking habit. Ideally, subjects should be non-smokers, but if this is not possible, the smoking factor should be taken into account during randomization so that the number of smokers in the treatment groups is balanced.

The choice of surgical technique and methodology is another important factor. There are specific surgical techniques for regenerative procedures that are primarily aimed at preserving interdental tissue. These techniques should be clearly described in the research protocol and appropriate training and calibration should be performed before the study begins.

The defect category plays another role. Particularly in periodontal regeneration studies where different approaches are taken to treat infra-alveolar defects, the anatomy of the defect may influence the regeneration outcome, and therefore measurement of this anatomy should be used as a secondary surrogate. This is usually done intraoperatively by direct measurements of the defect with a periodontal probe after the lesion has been completely cleaned. These measurements should include the number of bony walls defining the defect, the infra-alveolar component of the defect, and the defect angulation. The infracosseous component of the defect, as well as the defect angulation, can also be measured radiographically, although the accuracy requires good radiographic technique.

4. Conclusion

In summary, many factors must be taken into account in clinical studies of periodontal regeneration to achieve reliable and meaningful results. The defect category, the choice of surgical technique and methodology, and the patient's smoking habit play an important role. In addition, secondary endpoints such as plaque accumulation and gingival inflammation should be carefully evaluated as they may significantly influence the study outcome. These factors contribute to ensuring the effectiveness and safety of the treatment methods studied and ultimately improving the periodontal health of patients.

To be continued: Look forward to part 2 of our blog series, in which we will delve into further important aspects of clinical research in periodontology. Stay tuned!

5. How we can help you

We would be happy to support you with successful planning and implementation of dental studies. Thanks to our comprehensive expertise in this area with the special features that need to be taken into account, we generate the clinical data you need for your medical device. 

We support you throughout your entire project with your medical device, starting with a free initial consultation, help with the introduction of a QM system, study planning and implementation through to technical documentation - always with primary reference to the clinical data on the product: from the beginning to the end End.

Do you already have some initial questions?

You can get a free initial consultation here: free initial consultation

FDA - MDR: Transfer of the approval strategy to the US market

At medXteam, the focus is on clinical data. In this context, as CRO we not only carry out clinical trials with medical devices in accordance with MDR and ISO 14155, but also offer all other options and forms of data collection. This time the topic in this context is the approval strategies on the US market. Regulatory requirements also apply here and clinical data is sometimes required. But this time the focus is on the question: How can I transfer my MDR approval strategy to the US market with the highest level of efficiency?

Abbreviations

MDR Medical Device Regulation; EU Regulation 2017/745

QMS quality management system

Underlying regulations

EU Regulation 2017/745 (MDR)
Medical Device Implementation Act (MPDG)
Federal Food, Drug and Cosmetic Act (FD&C Act)
Code of Federal Regulations (CFR), Title 21
Quality System Regulation (QSR) – 21 CFR Part 820
Medical Device Reporting (MDR) – 21 CFR Part 803
Unique Device Identification (UDI) – 21 CFR Part 830
Postmarket Surveillance – 21 CFR Part 822

1 Introduction

The global medical device market faces numerous regulatory challenges and requirements that vary across regions. Two of the most important regulatory frameworks are the European Union's Medical Device Regulation (MDR) and the US Food and Drug Administration (FDA) approval processes. Both regulatory authorities have the primary goal of ensuring the safety and effectiveness of medical devices, but their requirements and processes differ significantly.

The MDR, which finally came into force in May 2021, replaced the previous Medical Device Directive (MDD) and brought with it significant changes and stricter requirements. It ensures that medical devices sold in the EU meet the highest safety and performance standards. The MDR requires comprehensive technical documentation, rigorous clinical assessments and continuous post-market surveillance. These stricter requirements pose a challenge for manufacturers who must ensure their products comply with the new regulations.

On the other side is the FDA, which plays a central role in regulating medical devices in the United States. The FDA classification of medical devices into different risk classes determines the approval process that a product must go through before it comes onto the market. The most common approval routes are the 510(k) Premarket Notification, the Premarket Approval (PMA), the Investigational Device Exemption (IDE) and the De Novo Classification. Each of these pathways has specific requirements for documentation and clinical data that must be submitted.

Transferring the MDR approval strategy to the US market is a complex process that requires careful planning and extensive knowledge of the regulatory requirements of both systems. Companies that want to successfully complete this transfer must understand the differences and similarities between the MDR and FDA regulations and adapt their documentation and processes accordingly. This includes identifying synergies, adapting technical documentation and reports, and taking specific FDA requirements into account in risk management and conformity assessment.

With this blog post we would like to create the basis and context for transferring the MDR approval strategy to the US market. We will examine the essential requirements and processes of the MDR and FDA, identify the main differences and similarities and discuss the specific challenges and solutions for the transfer process. The aim is to give companies practical insights and concrete recommendations for action in order to make the transfer process efficient and successful.

2. Medical device approval under the MDR

The MDR requires comprehensive technical documentation covering all aspects of the product life cycle, including design, manufacturing and clinical data. The process involves submitting this documentation to a Notified Body, which carries out a conformity assessment and, if successful, issues a CE marking. Medical devices that fall into class I under the MDR are excluded from the submission process.

Technical documentation under the MDR must contain detailed information about the medical device, including risk management reports, clinical assessments and evidence of compliance with all relevant standards. According to MDR, manufacturers must implement a robust risk management system that includes the identification, assessment and control of risks. Clinical evaluation is a continuous process that uses clinical data to confirm the safety and performance of the device throughout its life cycle. In addition, it is now mandatory for every manufacturer of a medical device to implement a complete quality management system with all relevant processes.

3. Medical device approval under FDA

The FDA categorizes medical devices into three classes (I, II, and III) based on their risk. Depending on the classification, manufacturers must submit either a 510(k) Premarket Notification, a PMA (Premarket Approval), or an IDE (Investigational Device Exemption). Each procedure has specific documentation and clinical data requirements.

The 510(k) Premarket Notification is intended for Class II products that must demonstrate that they are similar to a product already on the market. The PMA (Premarket Approval) is for Class III products that pose a higher risk and require evidence of extensive clinical data on safety and effectiveness.

The IDE (Investigational Device Exemption) makes it possible to carry out clinical studies with Class III products that have not yet been released on the market. The De Novo process provides a way to classify new products that do not have a similar approved product but pose a low to medium risk.

Even under the FDA, it is essential for manufacturers to implement a robust quality management system in accordance with the requirements (21 CFR Part 820) to ensure the quality and consistency of the products.

4. Comparison of the approval processes: MDR vs. FDA

Both systems have the common goal of ensuring the safety of medical devices, but differ in their approaches. The MDR requires strict post-marketing surveillance and ongoing clinical evaluations for each type of medical device, while the FDA offers different approval pathways based on the risk of the product.

MDR approval can be time-consuming and costly as extensive clinical data and detailed technical documentation are required depending on the type and risk class of the product. FDA approval can vary depending on the process (510(k), PMA), with PMA (Premarket Approval) processes typically being more expensive and lengthy than 510(k) submissions.

The MDR requires extensive technical documentation and ongoing clinical assessments. The FDA also requires detailed documentation, but specific requirements may vary depending on the approval route and product classification. Clinical trials are often necessary for PMA (Premarket Approval) and IDE (Investigational Device Exemption), while 510(k) relies on existing clinical data.

5. Transfer of the MDR approval strategy to the US market

When transferring the MDR approval strategy to the US market, much of the data and documentation that has already been collected can be reused. However, it is important to recognize the differences in regulatory requirements and make adjustments accordingly. The technical documentation prepared for the MDR may need to be adjusted to meet FDA's specific requirements. This may include reformatting reports, additional testing, or creating new documents required by the FDA.

The existing ISO 14971 risk management system can be retained in many aspects, but may need to be expanded to meet specific FDA requirements. The conformity assessment must comply with the FDA regulatory framework.

6. Challenges and Opportunities

Typical challenges include differences in regulatory requirements, additional documentation requirements, and the need for additional clinical data. These issues can lead to delays and increased costs.

Successful strategies include planning the transfer process early, working closely with regulatory experts, and carefully adapting existing documentation to FDA requirements. Constant monitoring of regulatory updates and changes is essential as these can impact approval requirements and processes. Companies should remain flexible and adapt their strategies accordingly.

7. Conclusion and conclusion

Transferring the approval strategy from the European Medical Device Regulation (MDR) to the requirements of the US Food and Drug Administration (FDA) is a complex but feasible process. Both regulatory systems have the same goal of ensuring the safety and effectiveness of medical devices, but they differ in their specific requirements, processes and documentation requirements. The success of such a transfer depends on a thorough analysis of the differences and similarities between MDR and FDA, as well as careful adaptation of existing documentation and processes.

An important aspect of the transfer is the identification of synergies where existing data and reports from the MDR process can be used to meet FDA requirements. At the same time, manufacturers must consider FDA's specific requirements, including adapting risk management, clinical evaluations and technical documentation. Implementing a robust quality management system in accordance with 21 CFR Part 820 and complying with Unique Device Identification (UDI) requirements are additional critical elements that must be considered.

Manufacturers going this route should consider the following key strategies:

  • Early planning and analysis of differences and similarities
  • Use synergies of existing data and documentation
  • Make specific adjustments to the technical documentation (e.g. risk management)
  • Review the quality management system with regard to FDA requirements (21 CFR Part 820).

Transferring the approval strategy from the MDR to the US market presents a challenge, but also offers the opportunity to expand market access in the US and promote global growth. With a careful and well-planned strategy, manufacturers can successfully navigate this process

8. How we can help you

We would be happy to support you with a successful and efficient transfer of your MDR approval to the US market. The first step is to decide on an approval process that is suitable under the FDA. We will then work with you to develop strategies with which you can get the most out of your existing documentation and existing clinical data in order to adapt them to the regulatory requirements on the US market in a cost- and time-efficient manner.

We support you throughout your entire project with your medical device, starting with a free initial consultation, help with the introduction of a QM system, study planning and implementation through to technical documentation - always with primary reference to the clinical data on the product: from the beginning to the end End.

Do you already have some initial questions?

You can get a free initial consultation here: free initial consultation

Statistical Significance vs. Equivalence: What Clinical Investigations Really Show

At medXteam, the focus is on clinical data. In this context, as CRO, we not only carry out clinical trials with medical devices in accordance with MDR and ISO 14155, but also offer all other options and forms of data collection and product approval as well as market surveillance. The focus of clinical trials is on the data collected, the evaluation of the data and the interpretation of the results. When interpreting results, a common mistake is to interpret the lack of a statistically significant difference between two treatments or products as evidence of their equivalence. In this blog post we will examine why a non-significant difference does not mean equivalence and what consequences this can have for clinical studies of medical devices .
 
Underlying regulations
 
EU Regulation 2017/745 (MDR)
ISO 14155
 
1. Introduction
 
An essential step after collecting data in clinical trials is their evaluation. Testing statistical significance or equivalence plays a crucial role here, depending on the nature of the study and the aim of the investigation. Statistical significance refers to whether the observed results are likely due to a real effect rather than random fluctuations. Equivalence, on the other hand, means that two treatments or products can be considered equivalent because their differences are not clinically relevant.
 
2. What does a non-significant difference mean?

A non-significant difference in a clinical trial means that the observed difference between two groups is not large enough to be statistically confident that it was not due to chance. Typically, a p-value greater than 0.05 is considered not significant. The p-value indicates how likely it is that the observed data or something more extreme will occur given the null hypothesis. The significance level (usually 0.05) is the threshold at which the p-value is considered small enough to reject the null hypothesis.

Example:

A clinical study compares a new implant with an existing implant and finds a p-value of 0.08. This means that the probability that the observed difference was due to chance is higher than 5%. Since the p-value is above the established significance level of 0.05, the difference is considered not significant.

3. Why is this not equivalent to equivalence?

In contrast to testing for a statistically significant difference, equivalence testing aims to show that the differences between two treatments or products are so small that they lie within a clinically acceptable range. This is achieved through specific study designs such as equivalence or non-inferiority studies.

Equivalence studies:

These studies set two predefined limits (equivalence limits) within which the differences between treatments must lie to be considered equivalent. The goal is to show that the effectiveness or safety of the new product does not differ significantly from that of the established product.

Non-inferiority studies:

These studies check whether the new product is no worse than the existing product by only setting a lower limit that the new product cannot exceed.

4. Differences in methodology

4.1 Null hypothesis

When testing for statistically significant differences, the null hypothesis is usually that there is no difference. In equivalence studies, however, the null hypothesis is that the treatments are not equivalent. The study must provide enough evidence to refute this null hypothesis.

Statistical significance tests play a central role in both types of studies, but the objectives and interpretation of the results differ. In classic tests of statistical significance, one looks for evidence that an observed difference did not occur by chance. The null hypothesis is rejected if a statistically significant difference is found (p-value < α).

In equivalence studies, however, the null hypothesis is that the treatments are not equivalent (that there is a significant difference). To refute this null hypothesis, the study must show that the differences between treatments are small enough to fall within a predefined equivalence range. Statistical significance is also tested here, but a different confidence interval is used. The results must show that the confidence interval of the difference lies entirely within the equivalence region to achieve statistical significance in terms of equivalence.

So in both cases statistical significance is used, but with different goals and interpretations.

4.2 Confidence intervals

While when testing for significant differences, confidence intervals are used to show the uncertainty of the estimate, in equivalence studies, confidence intervals are used to check whether they lie within the established equivalence limits. If the entire confidence interval lies within these limits, equivalence can be assumed.

These differences in methodology make it clear that the mere absence of a statistically significant difference is not sufficient to demonstrate equivalence. There are other factors that must be taken into account to ensure correct interpretation of the study results.

4.3 Lack of power of the study

A study with a small sample size or insufficient power may miss true differences. The lack of a significant difference may therefore simply be due to the study not being sufficiently powered to detect this difference. This is where sample size planning comes into play: careful sample size planning is crucial to ensure the power of the study. The power of a study describes the probability that the study will detect a real effect if it actually exists. Without appropriate sample size planning, there is a risk that a study will not be able to detect significant differences, even if they exist, due to too few participants.

4.4 Confidence intervals and uncertainty of the estimate

A non-significant difference can be associated with wide confidence intervals, which can indicate both clinically important differences and no differences. This shows the uncertainty of the estimate and does not suggest equivalence.

4.5 False null hypothesis

The null hypothesis in most studies is that there is no difference. Failure to reject this null hypothesis does not mean that it has been proven that there is no difference, just that there is not enough evidence to claim the opposite.

5. Examples of problems in clinical trials of medical devices

5.1 Comparison of two implants

In a study evaluating a new hip implant compared to an established product, a p-value of 0.06 was found. Although the difference is not statistically significant, the new implant could still be less effective or safe. A wide confidence interval could range from large superiority to significant inferiority.

5.2 Evaluation of a new diagnostic device

A new diagnostic device is tested against a standard device and the results show a p-value of 0.09. This doesn't mean that both devices are equally good, just that the study didn't find enough evidence to determine a difference. The study may not be large enough to detect small but clinically relevant differences.

6. How should equivalence be checked?

6.1 Equivalence and non-inferiority studies

To test equivalence, specific study designs such as equivalence or non-inferiority studies must be used. These studies have specific hypotheses and statistical methods to show that the differences between treatments are within a predefined tolerance limit.

Example:

An equivalence study could define that the new implant is clinically equivalent if the difference in functionality is within a range of ± 2% compared to the standard implant.

6.2 Confidence intervals and equivalence limits

Instead of just looking at p-values, confidence intervals should also be considered. If the entire confidence interval lies within the predefined equivalence limits, equivalence can be assumed.

7. Practical steps to avoid misunderstandings

Clear study design:

The study should clearly define whether it aims to find differences (superiority study) or to prove equivalence or non-inferiority. This influences the choice of statistical methods and the interpretation of the results.

Adequate sample size:

A sufficient sample size is crucial to ensure the power of the study. This helps detect real differences and avoid false negatives.

Predefined equivalence limits:

Before starting the study, clear equivalence limits should be established based on clinical considerations. This helps to better assess the clinical relevance of the results.

8. Conclusion

The absence of a statistically significant difference in clinical trials does not automatically mean that the medical devices tested are equivalent. Specific study designs and statistical methods are required to demonstrate equivalence. Careful planning and interpretation of study results are crucial to assess the true effectiveness and safety of medical devices. This is the only way we can ensure that new products meet the high standards of clinical practice and offer real benefits for patients.

9. How we can help you

Our statisticians accompany you from data collection through analysis to interpretation of the results. Be safe.

As CRO, we support you throughout the entire process of generating and evaluating clinical data and in the approval and market monitoring of your product. And we start with the clinical strategy! We also create the complete clinical evaluation file for you.

In the case of clinical trials, we consider together with you whether and, if so, which clinical trial needs to be carried out, under what conditions and in accordance with what requirements. We clarify this as part of the pre-study phase: In 3 steps, we determine the correct and cost-effective strategy with regard to the clinical data collection required in your case.

If a clinical trial is to be carried out, basic safety and performance requirements must first be met. The data from the clinical trial then flow into the clinical evaluation, which in turn forms the basis for post-market clinical follow-up (PMCF) activities (including a PMCF study if necessary).

In addition, all medical device manufacturers require a quality management system (QMS), including when developing Class I products.

We support you throughout your entire project with your medical device, starting with a free initial consultation, help with the introduction of a QM system, study planning and implementation through to technical documentation - always with primary reference to the clinical data on the product: from the beginning to the end End.

Do you already have some initial questions?

You can get a free initial consultation here: free initial consultation

How is a clinical assessment based on performance data created?

At medXteam, the focus is on clinical data. In this context, as CRO, we not only carry out clinical trials with medical devices in accordance with MDR and ISO 14155, but also offer all other options and forms of data collection and product approval as well as market surveillance. The focus is always on clinical evaluation, both during product approval and during clinical follow-up. One possible route for creating the clinical evaluation is based on so-called performance data. How can such a clinical assessment be carried out? What options are there to provide clinical evidence? And what role do clinical data play in this? In this blog post, we explore these questions, particularly explaining when and how this route of clinical assessment can be used .

Abbreviations

MDR Medical Device Regulation; EU Regulation 2017/745

PMCF Post-Market Clinical Follow-up, clinical follow-up

CEP Clinical Evaluation Plan

CDP Clinical Development Plan

Underlying regulations

EU Regulation 2017/745 (MDR)

1 Introduction

As already described in the last blog post, the clinical evaluation for all medical devices - from Class I to Class III - is an essential step for every manufacturer of medical devices. This is derived from Article 61 of EU Regulation 2017/745 (MDR):

“The manufacturer shall determine and justify the scope of clinical evidence to demonstrate compliance with the relevant general safety and performance requirements. The level of clinical evidence must be appropriate to the characteristics of the device and its intended purpose. To this end, manufacturers shall carry out, plan and document a clinical assessment in accordance with this Article and Part A of Annex XIV."

If the “performance data” route was defined during planning in the CEP, all requirements for the process and for the creation of the clinical assessment that result from the MDR and also from MEDDEV 2.7/1 Rev. 4 must still be adhered to . How this works: This blog post provides the relevant answers .

2. The route via performance data

The way to demonstrate the clinical performance of a product through performance data has always been possible and remains so under the MDR (Article 61):

If demonstration of compliance with essential safety and performance requirements based on clinical data is considered inappropriate, any such exception shall be based on the manufacturer's risk management and taking into account the specific characteristics of the interaction between the device and the human body, the intended clinical performance and the information provided by the manufacturer; this applies without prejudice to paragraph 4. In this case, the manufacturer shall duly justify in the technical documentation set out in Annex II why he demonstrates compliance with essential safety and performance requirements solely on the basis of the results of non-clinical testing methods, including performance evaluation, technical testing ( “bench testing”) and preclinical evaluation, is considered suitable .“

The decision is based on various aspects:

  • the result of risk management
  • the characteristics of the interaction between product and body
  • proof of performance based on product evaluations (technical, in-vitro)
  • the result of the preclinical assessment (initial literature search, verification tests, etc.)

This decision must be appropriately explained and documented in the clinical evaluation plan.

This route is preferred when a clinical trial offers little benefit. A typical example of this is the wooden tongue depressor, for which clinical data does not exist in the literature. In such cases, technical data such as breaking strength and workmanship indicate the safety and performance of the product.

As the equivalence route becomes less and less possible and applicable, it is becoming more and more the new standard based on performance data if there is no need to generate your own clinical data.

Below are examples of when this route makes sense:

2.1 Example – Medical Software

Most software products (Class I and IIa) are examples of products where performance data makes sense. The reasoning for this decision is as follows:

The product has been fully verified as part of the software life cycle process in accordance with IEC 62304 and all tests have been successfully completed. The testing included unit testing, integration testing, system testing and usability testing. Based on these tests, it can be shown that the product works effectively.

According to MDCG-2020-1 (Guidance on Clinical Evaluation (MDR)/Performance Evaluation (IVDR) of Medical Device Software), scientific validity is defined as the extent to which the output of the software product is valid based on the selected inputs and algorithms is associated with the desired physiological state or clinical disease. In order to provide proof of scientific validity, a literature search is carried out, which also includes proof of benefit according to the MDR as well as determining the state-of-the-art and identifying the safety and performance of the medical device.

The clinically relevant components of the system are the implementations of the algorithms/questionnaires for diagnosis or the course of therapy. The literature search focuses on scores/detection algorithms as well as on the general use of digital products in the diagnosis/therapy of the indicated indications.

Table 1: Clinical evaluation of a software product

2.2 Example – dentist chair

Another product whose clinical performance, safety and benefits can be easily assessed using performance data and for which a clinical test makes no sense is the dental treatment unit: the dental chair.

Such products are active medical devices that are used to treat children and adults in the dental field. These products are dental treatment devices according to ISO 7494 with a dental patient chair according to ISO 6875. They are intended exclusively for use in dentistry and may only be operated by medical professionals. The dental treatment unit is used as an aid for patient positioning and for treatment in the dental field. Depending on whether dental instruments are part of this treatment unit and, if so, which ones, these products are classified in class IIa or IIb.

Due to the clear intended purpose of these products, the question of whether a clinical trial should be carried out on humans is unnecessary. The claims about the product relate to the ergonomics for both the patient and the practitioner and user of the product. It also emphasizes efficiency and ease of operation, and prescribed procedures and supporting components to facilitate infection control and maintain water quality. These statements are not suitable endpoints for a clinical trial. However, they can be supported with performance data. For example, the topic of ergonomics and ease of use can be proven via the usability test (DIN EN 62366-1). Compliance with the relevant standards and regulations on water hygiene and quality also confirms these claims about the product. The reason for choosing the path based on performance data is now listed here in Table 2: