Ease of use meets evidence: The interface between usability and clinical evaluation

In this blog post, you'll learn how to systematically identify potential use-related hazards and capture them in risk management, accurately translate these use risks into clinical endpoints, and then support them with robust evidence using formative and summary usability studies and simulations. You'll learn how to seamlessly integrate the obtained usability data into your benefit-risk analysis and clinical evaluation report.

Abbreviations

MDR	Medical Device Regulation (EU Ordinance 2017/745)
Sota	State of the Art (state of the art)
CEP	Clinical evaluation plan
CERIUM	Clinical Evaluation Report
Pmcf	Post-Market Clinical Follow-up
Pms	Post-Market Surveillance
USAB	Usability

Underlying regulations and norms

EU Regulation 2017/745 (MDR)

En ISO 14971

IEC 62366-1

IEC 60601-1-6

1 Introduction

Clinical evaluation and usability (USAB) are still viewed as separate disciplines in many companies – different teams, different documents, different timelines.

However, the reality under the Medical Device Regulation (MDR, EU 2017/745) is different:
Clinical safety or clinical performance cannot be credibly demonstrated if the product is not fit for its intended use.

The MDR requires manufacturers to demonstrate that a medical device:

is safe
provides the intended service
can be used safely and effectively in the hands of the intended user, in the intended application context

This makes it clear: Data from usability engineering are not a “nice-to-have,” but a central component of clinical evaluation and flow directly into the benefit-risk assessment.

2. Regulatory framework

The regulatory framework for linking usability engineering and clinical evaluation is based primarily on the MDR and the international standards ISO 14971, IEC 62366-1, and IEC 60601-1-6. According to Annex I of the MDR, medical devices must be designed to reduce usage-related risks to an acceptable minimum.

The ISO 14971 standard integrates use-related hazards into risk management: Every potential user error is recorded as a hazard, assessed, and subjected to risk control. Remaining residual risks must be documented and justified in the benefit-risk analysis within the Clinical Evaluation Report (CER).

The IEC 62366-1 standard defines the usability engineering process for all medical devices: starting with the creation of a use specification, through the formal analysis of all use-related hazards, to the summative evaluation. The data obtained (e.g., error rates, processing times) provide precise endpoints that are documented in the Clinical Evaluation Plan (CEP) and later evaluated in the Clinical Evaluation Review (CER).

IEC 60601-1-6 supplements these requirements for electrical medical devices with specific requirements for user interfaces, alarm and display design: For example, the readability of displays and the comprehensibility of alarm messages must be validated and documented.

3. What does usability mean?

In the context of medical devices, the term usability describes far more than just "user-friendly design".
According to IEC 62366-1, usability is the "property of the user interface that supports use and thus achieves effectiveness, efficiency and user satisfaction in the specified usage environment".

This definition clarifies that usability is not just about aesthetic aspects or intuitive operation, but about the safe, effective and error-free use of a medical device by the intended users, in the intended application scenarios and environments.

A usable product reduces the likelihood of use errors and thus directly contributes to patient safety and the fulfillment of regulatory requirements.

Conversely, poor usability and unclear user guidance can lead to risks that cannot be adequately compensated for by either product design or training.

3.1 Usability in the MDR

The MDR explicitly recognizes the importance of usability and enshrines it in several Essential Safety and Performance Requirements (GSPR):

GSPR 5 – “Risks arising from user error should be avoided or minimized through design and construction.”
→ This means: Manufacturers must actively identify which user errors could occur and eliminate or reduce these risks through design decisions as early as the development phase.
GSPR 14.2(a) – “Risks of injury must be minimized in conjunction with the physical characteristics of the product – including volume/pressure ratio, dimensions and, where applicable, ergonomic features.”
→ Ergonomics here is not an optional comfort factor, but a safety-relevant criterion that is directly incorporated into the product design.
GSPR 14.6 – “Measuring, control or display devices shall be designed ergonomically, taking into account their intended purpose, the intended users and the environmental conditions.”
→ This includes, for example, the readability of displays, logical menu navigation, the design of control buttons or acoustic signals – everything must be adapted to the user and their environment.
GSPR 22 – “Special consideration given to the abilities and limitations of lay users.”
→ Devices for home use or for patients themselves must be designed so that even persons without medical training can operate them safely and correctly.

3.2 Why is this important?

A product that meets all regulatory requirements but is complicated or confusing to use in practice will fail to deliver its clinical performance.
Usability is therefore not just a design aspect, but an integral part of safety and performance evaluation and directly impacts the benefit-risk analysis in clinical assessment.

In practice this means:

Usability should be considered early in the development process (not at the end)
Results from usability tests should be incorporated into risk management, IFU, labeling and clinical evaluation
If changes are made to the user interface, risks must be reassessed and usability must be revalidated if necessary

4. The usability engineering process – more than just testing

Before linking usability and clinical evaluation can be successful, key terms and the underlying process must be clearly defined.

Use Specification:
The use specification comprehensively describes how and by whom a medical device is used. It defines user groups (e.g., nurses, physicians, lay users), application environments (hospital ward, home care, emergency response), and usage scenarios (routine injection, emergency response, long-term monitoring). A precise use specification serves as the basis for all further steps, as it defines the context in which risks can occur.

Use-Related Hazards:
A use-related hazard is a potential danger that can arise from the use of the product, not from technical malfunctions, but from misuse, operator error, or misunderstandings. Examples include confusing controls, inadequate cleaning, or unclear menu navigation. Each use-related hazard is identified in a formalized hazard analysis and documented in the risk management process.

Formative Evaluation:
In formative evaluation, prototypes or early product versions are iteratively tested with representative users. The goal is not definitive proof of safety, but rather the early identification and correction of usability issues. Common methods include task analysis, eye-tracking, think-aloud protocols, and structured observations. The insights gained are directly incorporated into design optimizations, ensuring that all the most serious use-related hazards are mitigated in advance in the final product.

Summative Evaluation
: Summative evaluation is the final, standards-compliant usability test. With the final product version, you conduct structured tests with typical users under realistic conditions and defined parameters. The collected key performance indicators (KPIs), such as error rates, processing times, and success rates for safety-relevant tasks, are quantitatively evaluated.

Process overview

Create use specifications
to define user profiles, application environments and usage scenarios.
Use-Related Hazard Analysis
Identification and documentation of potential misuses in the risk register (integration into ISO 14971 process).
Iterative formative evaluation:
Early testing on prototypes, task analyses and usage observation for design optimization.
Implementation of mitigation measures
, design adjustments, revised IFUs, training materials to minimize identified hazards.
Conduct summative evaluation:
final tests with the final product, quantitative recording of the results.
Integration into the CEP/CER:
Transfer of results into the Clinical Evaluation Plan and Clinical Evaluation Report, linking with benefit-risk analysis and risk management.

This structured process ensures that usability and clinical evaluation do not occur separately, but work in close coordination to ensure a safe, intuitive, and evidence-based medical device.

5. Interface: Usability → Clinical Evaluation

Transferring usability results into clinical evaluation ensures that risks typical of use are not only technically addressed but also validated. The process is divided into four consecutive steps:

5.1 Identification and prioritization of use-related hazards

In the first step, all potential misuses and operating errors are systematically recorded:

from the use specification (e.g., turning on the device, making settings, reading parameters).
In task analyses and workshops with representatives of the user groups (nursing staff, doctors, possibly laypersons), you identify use-related hazards such as incorrect button selection, confusion of modes, or unclear displays.
Using the risk table according to ISO 14971, you assess each hazard with regard to severity, occurrence, and detectability. The prioritized risks form the basis for the clinical question.

5.2 Translation into endpoints

For each prioritized use-related risk, define one or more measurable endpoints:

Error rate in safety-relevant tasks (e.g., percentage of users who do not correctly reset an alarm)
Average processing time until a critical process is successfully completed (e.g., start injection)
Number of ignored alarms within a defined observation period.
These endpoints must be clearly operationalized (measurement method, number of subjects, test conditions) and include quantifiable acceptance criteria (e.g., error rate ≤ 2%).

5.3 Integration into benefit-risk analysis and CER

In the clinical evaluation, you link the usability endpoint results with the use-related risks documented in risk management, e.g., as:

Tabular benefit-risk matrix: Each use-related hazard is compared to the measured endpoint, including quantitative results and assessment of the residual risk.
Narrative evaluation: Explanation of how reducing the error rate or shortening processing times improves the clinical benefit-risk profile (e.g., faster therapy, lower patient burden).
Claim derivation: Formulate precise claims based on the endpoint data (e.g., “Alarm reset successful in 98% of cases within 5 seconds”).

6. Example: Usability interface

To illustrate the procedure, let's look at a digital infrared ear thermometer:

Use-Related Hazard: False-negative temperature measurement due to improper probe alignment

Translation in endpoint

Endpoint: Percentage of measurements whose deviation from the reference thermometer is ≤ ± 0.3 °C
Acceptance criterion: ≥ 95% of the measurements must meet the criterion.

Evidence strategy

Summative usability study:
– n = 100 participants (mixed user groups to cover the entire patient population)
– Measurement in real home environments
– Documentation of deviations and the number of incorrectly placed probes
Supplementary simulation:
– Laboratory bench test to determine the dependence of measurement accuracy on insertion direction (± 5° steps)

Integration into benefit risk analysis

Use-Related Hazard	Endpoint	Result	Benefit-Risk Commentary
False-negative measurement due to incorrect probe alignment	≥ 95% of the measured values within ± 0.3 °C	96% fulfilled	Significantly reduces the risk of undetected fever

Narrative

"In the summative study, 96% of users achieved the required measurement accuracy, keeping the residual risk of undetected fever cases to a minimum. The supplementary bench tests confirm that even small deviations in probe alignment only marginally affect accuracy."

Claim derivation
: “The digital infrared ear thermometer offers ≥ 95% measurement accuracy (± 0.3 °C) in real application environments.”

This example illustrates how a use-related hazard is transformed into a precise clinical endpoint, which is validated through usability studies, documented in the CER, and finally converted into a quantifiable claim.

7. Conclusion

The consistent integration of usability into clinical evaluation not only increases user-friendliness, but is also essential for patient safety and regulatory compliance. By systematically identifying and prioritizing use-related hazards, deriving precise clinical endpoints, and planning and conducting formative and summative usability studies, you create robust evidence. The structured linking of this data with risk management and the benefit-risk analysis in the CER ensures seamless traceability and audit-proof documentation. This lays the foundation for medical devices that are both intuitive to use and clinically safe, meeting the requirements of users, auditors, and authorities.

8. How we can help you

We support you from the initial use specification to the final benefit-risk analysis, ensuring that your usability results flow seamlessly into the clinical evaluation. In joint workshops, we first develop your user profile, typical use cases, and identify all potential use-related hazards. We then plan and conduct both formative prototype tests and summative usability studies with representative user groups to generate reliable endpoint data.

We integrate this data directly into your CEP and CER. Together, we define precise endpoints, document the study results statistically, and link them to the corresponding risks in risk management.

In addition, based on your usability metrics, we formulate measurable, regulatory-compliant claims for your benefit-risk analysis and advise you on compliance with ISO 14971, IEC 62366-1, and IEC 60601-1-6 standards. This ensures that your medical devices are not only safe and intuitive to use, but also undergo clinically convincing evaluations and are approved quickly.

Want to know more? Contact us for a free initial consultation!

You can get a free initial consultation here: free initial consultation

Seamless interlinking: How risk management and clinical evaluation go hand in hand

In this blog post you will learn how to transfer identified hazards from risk management to targeted clinical issues, which regulatory requirements of the MDR and relevant ISO standards you have to consider, how to link the benefit-risk analyzes with clinical evidence and derive the clinical data requirement-from the risk management acts to clinical evaluation Report - for a complete traceability are essential, which practice -oriented approaches and best practices promote smooth interdisciplinary cooperation, how they recognize and bypass typical stumbling blocks at an early stage and what outlook there are on further interfaces in product development and post -market surveillance.

Abbreviations

MDR	Medical Device Regulation (EU Ordinance 2017/745)
Sota	State of the Art (state of the art)
CEP	Clinical evaluation plan
CERIUM	Clinical Evaluation Report
Pmcf	Post-Market Clinical Follow-up
Pms	Post-Market Surveillance

Underlying regulations and norms

EU Regulation 2017/745 (MDR)

En ISO 14971

1 Introduction

The tightened requirements of the MDR make a close integration of all processes essential. In particular, the interface between risk management and clinical evaluation plays a crucial role: Only if identified risks are consistently transferred to clinical issues, reliable benefit -risk analyzes can be created and the actual data requirement for the CLINical Evaluation Report (CER) can be derived precisely.

2. Basic terms and process overview

2.1 Definition of Risk Management
Risk management is a systematic, documented, and ongoing cycle for identifying, assessing, controlling, and monitoring risks that may arise from a medical device. It encompasses all potential hazards throughout the entire product lifecycle – from development and manufacturing to use and disposal. According to ISO 14971, the process includes risk analysis (identifying and assessing the severity and probability of occurrence), risk assessment (determining acceptable risk levels), risk control (defining and implementing measures), and verifying the effectiveness of these measures. All steps are documented in the risk management file and regularly updated through reviews and change control processes.

2.2 Definition of Clinical Evaluation
Clinical evaluation is an interdisciplinary process that systematically compiles and evaluates all available clinical data on the medical device and draws conclusions regarding its safety and performance. It forms a central interface with numerous other processes, such as design verification, usability, post-market surveillance, and vigilance. The aim is to demonstrate, through a structured benefit-risk analysis, that the expected benefits of the product clearly outweigh the remaining risks. The results are documented in the Clinical Evaluation Report (CER) and serve as key evidence for notified bodies and regulatory authorities.

2.3 Process map: From risk analysis to clinical evaluation
The transition from risk management to clinical evaluation can be divided into three phases:

Risk analysis and prioritization

Identification of hazards: Systematic recording of all potential risks along the product life cycle (design, production, application, disposal).
Severe grade and occurrence: Assessment of how serious damage would be and how likely his appearance is.

Derivation of clinical issues

Risk -to -evidence mapping: Every risk classified as critically is translated into one or more clinical questions. Example: A high risk of thrombosis in an implantable device leads to the question of whether the thrombosis rate in clinical studies falls below acceptable limit values.
Determination of the data requirement: What type of clinical data (e.g. study data, registry analyzes, real -world -evidence) are needed to reliably evaluate the risk?

Integration into the clinical evaluation

Benefit -Risk analysis: merging of the risk assessment with the clinical benefit; Representation that the expected benefit predominates the remaining risks by far.
Documentation in the CER: Structured proof that all identified risks were addressed and occupied by clinical evidence. Cross -references to risk management clocks ensure traceability.

This process map makes it clear that risk management and clinical evaluation are not isolated disciplines, but contribute closely to the common goal: the market approval of a secure and effective medical device.

3. Interface: Risk management → clinical evaluation

3.1 Derivation of the claims for clinical security from clinically relevant risks

What are clinically relevant risks?

Clinically relevant risks are the potential hazards that can affect health, security or well -being of patients directly and specifically (e.g. tissue damage, infection ...).

Proceed

Every risk identified in risk management is first recorded in the risk analysis and evaluated in terms of severity and probability of appearance.

Clinically relevant risks identified in risk management are systematically translated into verifiable clinical questions. Clinically relevant risks from the risk analysis should be addressed by one or more clinical safety claims/endpoints.

In essence, it is about systematically transferring clinically relevant risks from the risk management acts to the clinical evaluation and providing clinical evidence there. The process is divided into four steps:

a) Identification of clinically relevant residual risks
: The risk management file lists all potential harms that could concretely threaten the patient (e.g., tissue trauma, infection, false alarm, data loss). These residual risks form the basis for the clinical assessment.

b) Translation into clinical endpoints
For each residual risk, one or more measurable endpoints are defined and recorded in the Clinical Evaluation Plan (CEP). An endpoint precisely describes how the respective risk is objectively measured (e.g., incidence rate of irritation, percentage of cleaning cycles performed without microbial detection, number of false alarms per 100 hours of operation).

c) Determining the evidence strategy
According to the three routes of clinical evaluation, select the appropriate data source:

Own clinical studies for direct, prospective data on the product
Equivalence or literature literature on comparable systems
Performance and in -vitro tests or PMCF if clinical studies are not suitable.
For each endpoint, you determine which study designs, sample circumference or test protocols are required to make valid statements.

d) Documentation and Benefit-Risk Analysis in the CER
– Tabular Overview: Compile the endpoints, data sources, and main results in a structured table format.
– Narrative Sections: Explain how the results achieved minimize the respective residual risk.
– Benefit-Risk Table: Clearly demonstrate that the expected clinical benefit significantly outweighs the remaining risks.
– Traceability: In each CER section, reference the corresponding risk in the risk management file to ensure complete traceability.

This procedure ensures that every risk management defined in risk management systematically addressed, documented with a suitable evidence and evaluated positively in the benefit -risk ratio. This forms the basis for convincing admission documentation and long -term patient safety.

3.2 Linking Benefit/Risk Analysis with Clinical Evidence
The benefit/risk analysis forms the core of the Clinical Evaluation Report (CER) and links the residual risks defined in risk management with the data collected during the clinical evaluation. The aim is to demonstrate, through a systematic presentation, that the expected clinical benefit clearly outweighs the remaining risks.

4. Best practices

In order for the interlinking of risk management and clinical evaluation not only formally, but actually effectively, the following proven procedures have established themselves in everyday work:

4.1 Frequent stumbling blocks

Unclear questions:
If clinically relevant risks are not precisely translated into clinical endpoints (clinical safety claims), the data requirements cannot be clearly determined. Avoid vague formulations.
Opaque Traceability:
If clear cross-references do not exist in the CER and risk management file, auditors cannot trace the origin of the results. Focus on traceability from the outset.
Static documents:
A risk management file or CER, once created, fails to achieve its purpose if it is not regularly updated. Even minor design modifications can introduce new risks or change data requirements.
Isolated solutions instead of interdisciplinary teams:
When clinicians, engineers, and regulatory affairs professionals work separately, gaps arise. Plan interdisciplinary reviews with clearly defined responsibilities.
Inappropriate route selection:
Choosing the wrong approach between in-house studies, equivalence data, or performance data leads to unnecessary effort or regulatory inquiries. Make the route decision early on, based on a transparent evaluation framework.

4.2 Continuous update

Define change control triggers.
Specify which changes (e.g., design updates) automatically trigger an update of risk management and CER.
Regular review cycles:
Schedule meetings of all stakeholders at least every six months to update risk assessments and clinical data. Use checklists to ensure no topics are overlooked.
Using digital tools:
Through automatically maintained cross-references, digital tools link clinically relevant risks with the associated clinical endpoints and the corresponding CER chapter sections. This ensures seamless traceability without manual effort, and auditors can check the entire evidence trail with just a few clicks.
Establish documentation roles:
Define clear roles for authors, reviewers, and approvers in both processes. Include metadata in documents, such as version, release date, and revision history.

4.3 Example: Derivation of end points/claims for clinical security from clinically relevant risks

Imagine you have a digital infrared earmometer as a product in the product portfolio.

As part of the clinical evaluation of the thermometer, the following clinically relevant residual risks were identified from the risk management fact:

Ear canal irritation or injury:
Improper insertion of the probe can lead to pain, microtrauma or, in the worst case, eardrum injuries.
Cross-contamination:
Inadequate cleaning or a lack of disposable probe covers can promote the transmission of pathogens (e.g., otitis pathogens) between patients.
Battery leakage / chemical exposure:
Leaking battery fluid can cause skin irritation or burns if patients or caregivers come into contact with it.

Further procedure in the clinical evaluation:
From these risks, you derive one or more endpoints in the CEP (e.g., incidence of ear irritation, contamination rate of reused probe covers) and substantiate them with suitable evidence (studies, user usability tests, cleaning validations). This ensures that the benefit-risk ratio of your ear thermometer is clearly positive and guarantees safe use for patients.

From the clinically relevant risks found in risk management, z. B. the following quantifiable claims result:

Ear canal irritation
endpoint/measurable parameter:Incidence of ear irritation or microtrauma ≤ 0.1%
→ Claim: “Causes no irritation in ≥ 99.9% of cases (if this data was collected in the study(s))”
Cross-contamination
endpoint/measurable parameter:Contamination rate ≤ 1% after validated cleaning and disinfection protocol
→ Claim: "Achieves a contamination rate of ≤ 1% when using the validated cleaning and disinfection procedure." (Data from studies/product register)

Or a common end point/claim would also be possible:

Safe clinical use with a minimal incidence of adverse events– achieved through compliance with EN ISO 109931 (Biological Safety), IEC 60601-1 and IEC 60601-1-2 (Electrical Safety), and IEC 62366-1 (Usability):
– Incidence of ≤ 0.1% of minor ear irritation
– Contamination rate of ≤ 1% after a validated cleaning/disinfection protocol
– Documented cases of battery leakage in customer feedback = 0

5. Conclusion

A consistently networked procedure for risk management and clinical evaluation is the key to an efficient and regulatory correct market launch of medical devices. By specifically transferring identified risks into clinical issues and systematically deriving the clinical data requirement, they create resilient foundations for benefit -risk analyzes and meaningful clinical claims. Regular updates, clear traceability and interdisciplinary cooperation prevent typical stumbling blocks and ensure that both developers and auditors recognize the common thread at all times. Use digital tools and defined change control trigger to keep your processes agile and to meet the growing requirements of the MDR. In this way, they not only lay the foundation for successful approval, but also strengthen the trust of users and named areas in the long term in the safety and performance of their products.

6. How we can help you

We would be happy to support you in making the interface between risk management and clinical evaluation efficiently:

Gap analysis and audit readiness:
We review your risk management file and clinical evaluation for completeness, traceability gaps and compliance (ISO 14971, MDR) and prepare you specifically for audits.
“Risk-to-Clinic” Workshops:
In interactive workshops, we jointly define clinically relevant residual risks, derive precise endpoints, and formulate quantifiable claims for clinical safety.
Data Strategy & Route Decision Route Decision
We advise you on the selection of the optimal data sources – your own studies, equivalence data or performance/PMCF tests – and support you in study planning or PMCF design.
Regulatory support
From the preparation of your notified bodies audits to the submission: We support you in document creation, review and follow-up.

Want to know more? Contact us for a free initial consultation!

It has not been proven: How measurable parameters support their claims in the clinical evaluation - and what they need to know

In this blog post, you'll learn why claims are far more than mere advertising statements, how to correctly substantiate them with measurable parameters, and how to meet the regulatory requirements of the MDR. We'll show you where valid parameters come from, how they are evaluated and documented – and how you can derive sound benchmarks from the state of the art.

Abbreviations

MDR	Medical Device Regulation (EU Ordinance 2017/745)
Sota	State of the Art (state of the art)
CRP	C-reactive protein
VAS	Visual Analog Scale
RCT	Randomized Controlled Trial
Pmcf	Post-market clinical follow-up
KPI	Key Performance Indicator

Underlying regulations

EU Regulation 2017/745 (MDR)

1 Introduction

The development and approval of a medical device in Europe is a complex, highly regulated process. Claims—statements about a product's performance, safety, or benefits—are increasingly taking center stage. These claims are not merely promises; they are statements that are subject to regulatory review and scientific substantiation. The key criterion for their credibility: measurable parameters.

A claim like "Reduces wound healing time by 30%" initially sounds impressive, but it also needs to be substantiated. And that's precisely where the challenge – and opportunity – lies for manufacturers.

2. State of the art – Source of measurable parameters

The "State of the Art" ( SotA) section is a fundamental component of every clinical evaluation according to the MDR. It establishes the medical-technical reference frameworkwithin which a medical device is assessed and thus forms the basis for deriving claims and their measurable parameters.

The aim is to analyze what is currently considered medically and technologically established – with regard to indication, area of application, product class, and functional alternatives. This analysis will yield:

the expectations for a product,
the relevant performance and safety indicators
as well as benchmarks for measurable parameters

derive.

2.1 What the state of the art must reflect

A complete SotA section describes:

2.1.1 The state of the art

What technologies, materials, or designs are currently being used?
What performance metrics are considered standard or "best practice"?
What norms, standards, or regulatory requirements are established?

Example: For an imaging system, this would include, among other things, resolution values, radiation dose, processing times, digital interfaces or possibly AI-supported evaluation methods.

2.1.2 The state of the art in medicine

The focus here is on the clinical context of the product :

Indications: Which diseases or clinical conditions are addressed by the product?
Diagnostic and therapeutic methods: Which procedures are currently considered the standard of care for these indications?
Alternative applications: Are there products or processes that fulfill similar tasks – even if they are structured or technically implemented differently?

Example: For a wound therapy product, these would be common procedures such as hydroactive dressings, negative pressure wound therapy, or enzymatic debridement methods.

2.1.3. Data on comparable products

Examining existing medical devices is essential to identify relevant claims, target values, and evaluation criteria :

What claims are being made there?
What data is available to validate them?
Which parameters are used? – e.g., for a wound therapy product, these would potentially be wound healing time, infection rates, technical measurements, etc.

This information is necessary to classify one's own product approach as either "state of the art" or "innovative and superior".

2.2 Typical content and questions of the SotA

A thorough SotA analysis answers, among other things, the following questions:

Question	Goal
Which therapies are currently the clinical standard for the target indication?	Determination of the basis of comparison
What alternatives are there to my product?	Identification of potential benchmarks
What has been clinically proven for comparable products?	Orientation towards validated parameters
What regulatory requirements must be observed?	Compliance with minimum technical standards

2.3 Sources for the state of the art

For a valid and comprehensive SotA analysis, several complementary sources be considered. Only in this way can an objective and reliable picture be created.

a) Scientific literature

Original studies, clinical trials, systematic reviews
Studies on the effectiveness, safety, and benefits of comparable technologies
Meta-analyses or evidence-based recommendations

Tip: Use targeted databases such as PubMed, Embase or Cochrane Library.

b) Medical guidelines

National and international recommendations (e.g. AWMF, NICE, ESMO, EORTC)
Diagnostic, therapeutic and follow-up recommendations
Inclusion of consensus papers and statements

Practical example: For a product for stroke prevention, ESC guidelines on secondary prevention are relevant.

c) Regulatory and technical standards

ISO, IEC, DIN, CEN standards
Requirements for safety, functionality, software, biocompatibility

Important: Standards are not only important for technical verification – they often also define measurement methods and requirements, thus providing measurable parameters!

d) Data on comparable medical devices

EUDAMED, FDA database, manufacturer websites
Published registration or post-market studies
Safety and performance parameters of comparable products

Don't forget: Also search clinical trials from competitors on clinicaltrials.gov or EU-CTR!

e) Own data sources

Preclinical test reports
Technical Performance Studies
Validation and verification data
Results of early-use projects

Note: This data is particularly valuable for substantiating innovations, provided it is documented in a traceable manner.

f) Further sources (supplementary)

Market analyses, white papers
Interviews with clinical experts
Conference papers and case reports

Caution: Marketing materials are for guidance only – not a reliable source for regulatory statements.

3. Evaluation of parameters in the clinical evaluation

Clinical evaluation is the central instrument for assessing the safety and performance of a medical device in accordance with the MDR. The claims defined beforehand are not simply statements – they must be verified based on objective, measurable parameters and scientifically validated. The evaluation of these parameters is therefore a critical step in achieving regulatory compliance and demonstrably proving the clinical relevance of the product characteristics.

The first step in this process is defining quantifiable criteria. Every claim must be unambiguous and objectively measurable. General or interpretable statements, such as "improved tolerability" or "high efficacy," are insufficient. Instead, precise target variables must be determined against which the claimed effect or property of the product can be verified. These could include, for example, the healing time in days, the intensity of postoperative painmeasured using a Visual Analog Scale (VAS), tissue integritydetermined by histological analysis, or the infection rate compared to a control product. It is crucial that these parameters have a direct bearing on the clinical application of the product and a relevant impact on patient care or safety.

The next step is selecting appropriate evaluation methods. The choice of a specific method depends largely on the type of parameter and the medical context. Controlled clinical trials (e.g., randomized controlled trials or post-market clinical follow-up, PMCF) are frequently used to validate clinical benefit and performance under real-world conditions. Alternatively or additionally, preclinical studies, verification and validation tests , and in vitro or in vivo models can be used, particularly if direct clinical investigation is not (yet) possible. In this context, performance data are also considered, which, as described above, can also be derived from state-of-the-art technology.

In some cases, a comparative analysis with existing literature or existing market products may also be sufficient, especially if the product corresponds to established solutions in its function or technology.

Once valid data have been collected, statistical evaluation and analysis. This serves to demonstrate the significance and relevance of the results. Depending on the study design and data basis, various methods can be used – from comparisons of means and standard deviations to confidence intervals and survival analyses, such as those performed with Kaplan-Meier curves . The aim of this analysis is not only to achieve statistical validation of the results but also to communicate clinically relevant differences in a clear and verifiable manner.

Documentation is an essential component of the assessment . All steps – from defining the parameters and selecting the methods to the evaluation – must be fully, transparently, and comprehensibly documented . This documentation not only serves internal quality assurance but is also a key element for communication with notified bodies and regulatory authorities. Without a complete and structured presentation of the derivation and verification of the claims, the clinical assessment cannot be concluded fully or in a regulatory-acceptable manner.

4. Conclusion

With the introduction of the MDR, the understanding of product performance and safety in clinical evaluation has fundamentally changed. Key terms such as "clinical benefit," "clinical performance," and "safety" are no longer vague concepts, but clearly defined categories that must be substantiated by measurable parameters . These parameters, in turn, support claims —that is, the specific statements manufacturers use to promote, differentiate, and position their solutions within the regulatory framework.

However, a claim is only valid if it is based on objective, quantifiable factors ("measurable parameters") – and not on hypothetical promises. Statements such as "reduces postoperative pain," "shortens wound healing," or "improves diagnostics" are only tenable in practice if they are supported by scientifically valid data. This data, in turn, must originate from a robust methodological approach: from clinical studies, verification and validation processes, or comparative analyses with the state of the art.

The foundation for this is a structured and well-researched analysis of the state of the art – that is, the current state of medical-technical knowledge. This allows for the establishment of realistic benchmarks, the identification of relevant target parameters, and an understanding of the characteristics of comparable products. Claims are derived from this analysis and are then reviewed and documented as part of the clinical evaluation.

Systematic and traceable processes are particularly important : Every step – from formulating the claim and selecting the assessment method to data analysis – must be clearly documented and regulatory-compliant. Only in this way can notified bodies or regulatory authorities verify whether a product actually delivers on its promises. At the same time, this creates the foundation for credible communication with doctors, nurses, and patients.

Furthermore, the clear formulation and validation of claims also offers strategic added value: it creates differentiation in the market, facilitates market access in other countries, and strengthens brand trust. Products whose claims are scientifically substantiated and regulatory-compliant enjoy significantly higher acceptance in everyday clinical practice – and ultimately contribute to better patient care.

It is therefore clear: The quality of the claims and their measurable parameters determines the regulatory approval, clinical relevance, and commercial success of a medical device. A strong product needs strong claims – but these must be proven, not just asserted.

To achieve this, an interdisciplinary approach required: Regulatory Affairs, clinical research, product development, and quality assurance must work closely together. Only in this way can the necessary depth and quality of the assessment be achieved. Those who consider this from the outset not only minimize the risk of regulatory queries but also increase the efficiency of the approval process and create a sustainable competitive advantage.

Investing in a sound claim strategy and the clear definition of measurable parameters is therefore not a bureaucratic effort, but an investment in the future viability of your product.

5. How we can help you

Our expertise lies precisely where you need it: at the interface between regulatory requirements and scientific evidence. We support you with:

The definition of realistic, regulatory-sound claims
The research and derivation of suitable measurable parameters
Conducting benchmark analyses
Clinical evaluation of your products according to MDR
The creation of reliable documentation for notified bodies

Want to know more? Contact us for a free initial consultation!

Clinical exams in practice - monitors chat out of the sewing box

At medXteam, clinical data is our core focus. As a CRO, we not only conduct clinical trials (studies) with medical devices in accordance with the MDR and ISO 14155, but also offer support in the area of monitoring required by GCP. This article explains what is important both on-site and during remote monitoring, and what potential pitfalls to avoid.

Abbreviations

GCP (Good Clinical Practice)

MDR Medical Device Regulation; EU Regulation 2017/745

Underlying regulations

EU Regulation 2017/745 (MDR)
General Data Protection Regulation (GDPR)
Medical Devices Implementation Act (MPDG)
ISO 14155

1 Introduction

Clinical trials are an essential part of medical research and development. They allow new therapies and medications to be tested for efficacy and safety before being made available to the general public. A crucial factor for the success of such studies is monitoring. Monitors play a central role in ensuring the integrity, safety, and reliability of the trials. In this blog post, we take a look behind the scenes and highlight the pitfalls that can arise when monitoring clinical trials. We will discuss the key points that monitors should consider to ensure the successful conduct of the studies.

2. What should be considered during on-site monitoring?

2.1 Communication

Effective communication is paramount for the success of any clinical trial. Monitors must ensure seamless communication between the sponsor, the study team, and other stakeholders, such as Contract Research Organizations (CROs). Regular meetings and progress reviews are essential to keep all parties informed and to identify and address potential issues early on.

Ensuring good communication:

Regular meetings:These should take place both in person and virtually to ensure that all participants are informed and involved. In-person meetings allow for detailed discussions of complex topics and help avoid misunderstandings. Virtual meetings offer flexibility and enable more frequent communication, which is particularly beneficial in international studies.
Progress reports:These reports should be detailed and prepared regularly to document the current status of the study and ensure transparency. They should include information on the study's progress, any problems encountered and their solutions, as well as planned next steps. Clear and concise reporting helps keep all stakeholders informed and maximizes the study's efficiency.

2.2 Preparation

Thorough preparation is key to the success of any monitoring activity. Monitors should ensure that all necessary steps are taken in advance to guarantee a smooth process. This begins with sending a confirmation letter in a timely manner and ensuring that all relevant people and data are available.

2.3 Sending a Confirmation Letter in a timely manner:

Scheduling an appointment:It is important that all relevant members of the study team are present. The duration of the visit should be estimated and communicated in advance. This allows for efficient planning and ensures that all necessary information and resources are available.
Access to patient and study data:Monitors must ensure they have access to all necessary data to perform their duties effectively. This includes both physical and electronic data. Seamless data access is crucial to avoid delays and ensure the accuracy of reviews.

2.4 Current status of the study:

Relevant study documents:Monitors should ensure that all relevant documents, such as the study protocol and patient information, are up-to-date and meet the latest requirements. This also includes verifying that all documents are complete and filled out correctly. Thorough preparation helps to identify and resolve potential problems early on.

2.5 Ensuring Compliance

Compliance with regulations and legal requirements is of paramount importance for the integrity of a clinical trial. Monitors must ensure that the study is conducted in accordance with ICH-GCP guidelines and other relevant regulations. This includes verifying approvals, training, and the storage of study materials.

Compliance review:

Approvals:Monitors should verify that the study has been approved by the relevant authorities and ethics committees. This is a crucial step to ensure that the study complies with legal requirements and is ethically sound.
Training of the study team:It is important that the study team is properly trained and has access to all relevant documents. Thorough training ensures that the team understands and correctly implements the study requirements and protocols.
Storage of study materials:All study documents should be stored securely, e.g., in locked cabinets. Test materials requiring refrigeration should be stored accordingly. Proper storage is crucial to ensure the integrity of the study materials and to minimize potential risks.

2.6 Patient safety

Patient safety is paramount in clinical trials. Monitors must ensure that all patient information is accurate and complete and that all safety precautions are followed. This includes reviewing patient information and documenting serious adverse events (SAEs).

Ensuring patient safety:

Patient information:Monitors should verify that all patient information is correct and fully signed. A comparison with the screening and patient ID lists is essential. This helps confirm patient identity and ensures that all necessary consents are in place.
Documentation of (S)AEs:All serious adverse events (SAEs) must be correctly documented and reported. Monitors should ensure that this documentation is complete. Thorough documentation is crucial to ensure patient safety and to identify and address potential risks early on.

2.7 Data accuracy and completeness

The accuracy and completeness of the data are crucial for the validity of the study results. Monitors must regularly check the entries in the clinical databases and ensure that the study data match the source data. This includes both remote monitoring and on-site source data verification.

Data accuracy verification:

Remote monitoring:It is often possible to check entries in clinical databases remotely. This offers flexibility and allows for more frequent data review.
Source Data Verification:During on-site monitoring, the study data should be compared with the source data. It is important to ensure that all deviations from the protocol are documented. A thorough review of the source data helps to guarantee the accuracy and completeness of the study data and to identify and correct potential errors early on.

3. Remote Monitoring in Practice:

Remote monitoring enables location-independent data control and regular verification of the quality of entries. The focus is on the completeness, traceability, and plausibility of the entered information. In practice, however, it is frequently observed that data is entered incompletely or late. This can compromise data integrity and delay analysis.

A functioning communication flow between the monitor and the study team is therefore essential – especially when questions arise or data gaps are identified. Unfortunately, it is frequently evident, particularly in remotely conducted activities, that information loss or delays occur. Clear responsibilities, timely responses to queries, and a structured data flow help to avoid these pitfalls.

Remote monitoring can be a huge relief – but only if it is well integrated into the overall study process.

3.1Monitoring in times of increasing digitalization

With advancing digitalization, the requirements for monitoring are also changing. Electronic documentation, e-consent procedures, and decentralized study formats present monitors with new challenges – but also offer opportunities. Stronger IT skills, a good understanding of digital workflows, and the ability to communicate effectively remotely are increasingly becoming core competencies.

3.2 Study Documentation

Complete and up-to-date documentation is essential for the success of a clinical trial. Monitors must ensure that all essential documents in the Investigator's File are complete and current. This includes regularly reviewing and updating the documents.

Review of study documentation:

Essential documents:Monitors should regularly check that all necessary documents are present and up-to-date in the Investigators File. Thorough documentation is crucial to ensure the integrity of the study and to identify and resolve potential problems early on.

4. Pitfalls in monitoring

Despite careful planning and execution, various pitfalls can arise during the monitoring of clinical trials. These can jeopardize the integrity and success of the study if they are not identified and rectified in a timely manner.

This includes, for example, the following points:

Source Data:

Incomplete documentation: Very often, the documentation for monitoring visits is incomplete. Sometimes it is only obtained during the visit itself, which is very time-consuming. There have even been cases where patient files were outsourced without notification. This leads to significant delays and makes data verification more difficult.

Relevance of data: Protocols often use the term "relevant" in relation to prior medication, pre-existing conditions, etc. This definition of "relevant" is, of course, very broad and frequently leads to discussions with the principal investigators (PIs). A clear definition and communication of the relevant data is crucial to avoid misunderstandings.

Minor organizational issues: Problems often arise from seemingly minor details, such as initials and the evaluation of study examinations (lab work, ECG, etc.). This appears to be an organizational issue at some centers. The study nurse often has to chase after the doctors to get this actually quite simple task completed. Better organization and clear responsibilities could remedy this.

Organization at the study center / Study implementation:

Staff overload: Unfortunately, it is almost always the case that study coordinators and study nurses are overburdened. It seems that the principal investigators (PIs) do not fully grasp the complexity of a study. The same often applies to sub-investigators, who are "plumbed" on a study in addition to their other responsibilities. This leads to poor compliance and incomplete source data.

Lack of seriousness: Sometimes, as a monitor, you get the feeling that studies are "not taken seriously." This is something that can happen almost as an afterthought. Of course, this isn't true and leads to poor study execution and incomplete source data. Clear communication of the study's significance and requirements could help here.

Cumbersome and poor data systems:

Poorly designed e-CRF: One example is a study conducted with a very poorly designed e-CRF. The e-CRF is cumbersome and confusing for study nurses and CRAs. This often leads to unnecessary queries, frustrating both center staff and CRAs. Improved design and user-friendliness of the data systems could remedy this.

Query Resolution:

Lengthy duration: Solving queries typically takes too long. This is often due to the problems mentioned above. Faster and more efficient query solutions are crucial to avoid hindering the progress of the study.

5. Conclusion

Monitoring clinical trials is a demanding and responsible task. Monitors must consider a multitude of aspects to ensure the integrity, safety, and reliability of the studies. Effective communication, thorough preparation, adherence to regulations, ensuring patient safety, verifying data accuracy, and complete documentation are crucial. By paying attention to these points, monitors can contribute to the successful execution of clinical trials and the delivery of valid results.

6. How we can help you

We would be happy to support you in the development, implementation, and use of a database-driven system. As your CRO, we will also handle the complete data management and monitoring via the EDC system.

We support you throughout your entire project with your medical device, starting with a free initial consultation, help with the introduction of a QM system, study planning and execution, right up to technical documentation - always with primary reference to the clinical data on the product: from the beginning to the end.

Do you already have some initial questions?

You can get a free initial consultation here: free initial consultation

From figures to findings - statistics in practice

At medXteam, clinical data is our focus. As a CRO, we not only conduct clinical trials (studies) with medical devices in accordance with the MDR and ISO 14155, but also offer support in the statistical planning and analysis of study data. This article provides an overview of the most important statistical concepts in clinical studies, from basic explanations and practical examples to more advanced topics.

Abbreviations

GCP (Good Clinical Practice)

MDR Medical Device Regulation; EU Regulation 2017/745

Underlying regulations

EU Regulation 2017/745 (MDR)
General Data Protection Regulation (GDPR)
Medical Devices Implementation Act (MPDG)
ISO 14155

1 Introduction

Statistical methods play a central role in the clinical investigation of medical devices. They are key to data analysis, results interpretation, and regulatory compliance. This article covers the following topics:

Confidence intervals
First and second order errors
Acceptance criteria
Boxplot
Forestplot
Connected data
Sensitivity / Specificity

2. What was that about the confidence interval again?

The confidence interval indicates the range within which an estimated parameter, such as the mean or an effect size, lies with a given probability. It quantifies the uncertainty surrounding an estimate and is therefore an indispensable tool in statistics.

The confidence interval indicates how precise an estimate is. The narrower the interval, the more confident we can be that the true value is close to the estimated value. Conversely, a wide interval suggests greater uncertainty. A confidence interval is often given with a confidence level of 95%. This means that the true value will lie within the stated range in 95 out of 100 cases if the study is repeated under identical conditions.

Example:

Suppose a study shows a mean wound healing time of 10 days with a confidence interval of [8, 12] at a 95% confidence level. This means that the true mean is within this range with a 95% probability.

2.1 In-depth study

How do confidence intervals work and why are they crucial?

The basic idea of the confidence interval: The confidence interval is based on the uncertainty inherent in any sample. It helps to quantify this uncertainty by specifying a range within which the true value of a parameter is highly likely to lie. The more data we collect and the lower the variance of the data, the more precise (i.e., narrower) the interval becomes.

Interpreting a 95% confidence interval: It does not mean that the true value lies within the interval "with 95% probability". Instead, the statement refers to the fact that if we were to repeat the data collection process infinitely many times, the interval would contain the true value in 95% of cases.

What factors influence the width of a confidence interval?The width of the interval depends on three main factors:
Sample size:Larger samples provide more precise estimates because the influence of random fluctuations is reduced. This leads to narrower confidence intervals. With small samples, the intervals are wider because the uncertainty is greater.
Data variability:If the data has a greater spread (i.e., if the values are widely scattered around the mean), the intervals are wider because the uncertainty about the actual value increases.
Selected confidence level:Higher confidence levels (e.g., 99% instead of 95%) result in wider intervals because more uncertainty is taken into account. Conversely, a lower confidence level (e.g., 90%) results in narrower intervals.

Practical implications: A particularly wide interval indicates that additional data is needed to better narrow down the true value.

2.2 Alternative methods for estimating confidence intervals

The classical method assumes that the underlying data is normally distributed and that the sample size is sufficiently large. Alternative approaches can be used when these assumptions are not met or when samples are small

Bootstrapping:This method is ideal when the assumption of normality is violated or when dealing with small sample sizes. It involves repeatedly drawing a large number of samples with replacement from the available data. For each of these samples, the parameter of interest (e.g., the mean) is calculated. The distribution of these estimates then serves as the basis for deriving the confidence interval.
- Advantages:Robust against distribution disturbances; flexibly applicable.
- Application example:For non-normally distributed data, such as highly asymmetrical blood pressure values, bootstrapping provides a precise estimate.
Bayesian confidence intervals (credible intervals):Unlike classical statistics, the Bayesian approach works with probabilities. Here, prior knowledge about the parameter is introduced through a so-called prior distribution. This is combined with the observational data (likelihood) to calculate the posterior distribution. The credible interval then indicates the range within which the true value lies with a certain probability.
- Advantages:Integration of prior knowledge; better interpretability with small sample sizes.
- Application example:If previous studies show that a medical device typically causes a wound healing time of about 10 days, this information can be included in the analysis to reduce uncertainty.

2.3 Practical relevance of confidence intervals in clinical research

Clinical relevance versus statistical significance:Confidence intervals provide more information than a p-value. While a p-value only indicates whether an effect is statistically significant, the confidence interval also shows whether the effect is clinically relevant. For example, a medical device might produce a statistically significant reduction in wound healing time, but this reduction might be so small that it is clinically irrelevant.
Assessment of uncertainties:Regulatory decisions often involve checking whether the lower limit of the confidence interval is above a certain threshold that is considered clinically significant.

3. First and second order errors

Statistical tests carry the risk of errors because decisions are made based on sample data that can only partially reflect reality. First- and second-order errors are therefore important concepts in statistics and particularly relevant in clinical research, where incorrect decisions can have significant consequences.

Two types of errors can occur in statistical tests:

First-order error (alpha error): This occurs when the null hypothesis is rejected even though it is true. This error is also known as a "false alarm." Example: An ineffective medical device is classified as effective.
Second-order error (beta error): This error occurs when the null hypothesis is retained even though the alternative hypothesis is true. This is often described as "missing an effect." Example: An effective medical device is not detected.

Maintaining a balance between these types of errors is a key task in planning clinical trials. The significance level and statistical power play a central role in this.

Example:

A new medical device is being tested. An alpha error would lead to the approval of an ineffective product, while a beta error could classify an effective product as ineffective.

Practical implications: While an alpha error is problematic from a regulatory and economic perspective, a beta error can hinder medical innovation.

3.1 In-depth study

Relationship between alpha and beta errors:There is a direct relationship between these two types of errors. If the significance level (alpha) is chosen more strictly to reduce the probability of an alpha error (e.g., from 0.05 to 0.01), the risk of a beta error often increases. Conversely, relaxing the alpha value reduces the beta error but increases the risk of detecting false effects.
Statistical power: Statistical power is a measure of how well a test can detect a real effect. A power of 80% means that a real effect is missed in 20% of cases (beta error).
Factors influencing test power includesample size, effect size, and the chosen significance level. A larger sample size increases the probability of detecting small effects and reduces the beta error.
Adjustment for multiple analyses:
Interim analyses:In studies with regular data analyses, the probability of an alpha error can increase because each analysis carries the chance of detecting a random effect. Methods such as the O'Brien-Fleming method use stricter thresholds in early analyses to control the overall error rate.
Bonferroni correction:This method divides the significance level by the number of comparisons to keep the overall error rate low. However, this is conservative and can reduce the statistical power when performing a large number of tests.
Bayesian perspective:

Instead of using rigid significance levels, Bayesian statistics assesses probabilities. For example: How likely is it that an effect is larger than a clinically relevant threshold? This can lead to more flexible and interpretable results, especially with small samples.

ROC curves:

The Receiver Operating Characteristic (ROC) curve shows the trade-offs between sensitivity (true positives) and specificity (false positives). It helps to identify thresholds that minimize both alpha and beta errors.

4. Acceptance criteria

Acceptance criteria define the conditions under which a clinical outcome is considered successful. They are crucial for the interpretation of study results and the decision as to whether a medical device is effective or safe.

Acceptance criteria define which results are required to achieve a specific goal. They influence study planning, hypothesis formulation, and ultimately, the approval decision for a product.

Example:

A medical device is being developed to shorten the healing time after surgery. The acceptance criterion is that the average healing time must be reduced by at least 20% compared to standard treatment. The study will verify whether the confidence interval of the result exceeds this threshold.

4.1 Further Study

Non-inferiority, superiority, and equivalence tests:

Non-inferiority test:Shows that the new product is no worse than an existing treatment, within an acceptable tolerance range.
Superiority test:Proves that the product is significantly better.
Equivalence test:Checks if the product is similar within a defined range (e.g. ±10%).
1. Bayesian approaches:
Instead of setting a fixed threshold for acceptance, Bayesian methods calculate the probability that the true effect is greater than a predefined threshold. This allows for a dynamic and probabilistic analysis.
1. Clinical significance:
A statistically significant effect does not automatically meet an acceptance criterion, as clinical relevance must also be assessed. For example, a 1% reduction in pain might be statistically significant but clinically insignificant.
1. Cost-benefit analysis:
Strict acceptance criteria can increase the quality of the assessment, but often require larger samples, which increases the cost and duration of the study.

5. How do I read a boxplot?

A box plot, also known as a box graph, is a versatile statistical tool that easily visualizes the distribution of data. It helps to identify central trends, variability, and potential outliers at a glance and is particularly useful for comparisons between groups.

A boxplot provides a compact representation of the distribution of a data set. The most important components are:

Median:The line in the middle of the box represents the central value of the data.
Quartiles:The bottom edge of the box is the 1st quartile (Q1), the top edge the 3rd quartile (Q3).
Interquartile range (IQR):The range between Q1 and Q3 comprises the middle 50% of the data.
Whisker:The lines above and below the box indicate data values outside the IQR, up to a defined limit (often 1.5 times the IQR).
Outliers:Data points that lie outside this limit are displayed separately as points.

Example

Let's imagine we have data on the healing time of two patient groups (Group A and Group B):

Group A has shorter healing times with low variability, resulting in a compact box with short whiskers.
Group B shows greater differences between patients, resulting in a wider box and longer whiskers.

A direct comparison of the two boxplots can quickly show which group is more homogeneous and whether there are any extreme outliers.

5.1 Further Study

Detailed interpretation:

Median:Indicates the central tendency of the data and is robust against outliers.
IQR:Shows the dispersion of the middle 50% of the data and gives an impression of the variability.
Whiskers and runaways:Help identify extreme values that could potentially distort the analysis.
1. Comparing groups: Boxplots are ideal for illustrating differences between groups, for example, to compare the effect of a medical device on different age groups. Differences in the height of the box or the whiskers can indicate variability or systematic effects.
2. Enhanced visualizations:
Violin plots:A combination of box plot and density plot that shows the entire distribution of the data. Particularly useful for multimodal distributions (e.g., two peaks in the data).
Parallel Boxplots:Multiple boxplots placed side by side facilitate the direct comparison of groups.
1. Use in clinical trials:
Outlier analysis:In a clinical trial, outliers could indicate patients who respond exceptionally well or poorly to a treatment. Such findings can provide clues to individual differences that are important for further research.
Stratification:Boxplots can be used to stratify and visually represent data by subgroups (e.g., age groups, gender).
1. Robustness: Because the median and quartiles are insensitive to outliers, the box plot is particularly robust. However, strongly asymmetrical distributions (e.g., long "tails" on one side) can be misleading. In such cases, alternative representations such as the violin plot can be helpful.

6. How do I read a forest plot?

A forest plot is an indispensable tool in meta-analysis, enabling the presentation and interpretation of results from multiple studies or subgroups. It displays estimates and their confidence intervals in a single, unified diagram.

The Forestplot consists of:

Estimate points:These points or squares represent the effect (e.g. mean, odds ratio) of each study or subgroup.
Confidence intervals:The horizontal lines indicate the uncertainty of the estimate.
Vertical line:This represents the "no effect" point, e.g., an odds ratio of 1 or an effect size of 0.
Overall effect:A diamond at the bottom indicates the weighted mean of all studies, with the width of the diamond representing the confidence interval.

Example

A meta-analysis examines the effectiveness of a plaster on wound healing time in various studies.

Study A shows a significant reduction in wound healing time, with a confidence interval that lies entirely below the "no effect" line.
Study B has a wide confidence interval that includes both positive and negative effects, which suggests uncertainty in the results.
The overall effect (diamond) is also below the line, indicating a significant effectiveness of the patch.

6.1 Further Study

Analysis of heterogeneity:

Cochran's Q-test:Checks whether the variation between studies is greater than would be expected by chance.
I² statistic:Indicates the percentage of variability that is explained by heterogeneity. A high value (e.g., above 50%) suggests that a random-effects model is more appropriate.

Fixed effects vs. random effects models:

Fixed-effects model:Assumes that all studies measure the same true effect and that differences arise only by chance.
Random Effects Model:It takes into account that studies may have different populations and conditions, and allows for greater variability between studies.
1. Bayesian Forest Plots:
Bayesian approaches use prior knowledge to better model uncertainty. The forest plot could visualize posterior distributions and credible intervals, allowing for a deeper interpretation.

Interpretation in practice:

A forest plot can be used to assess the consistency of results. Studies whose confidence intervals do not cross the "no effect" line provide strong evidence. Divergent results from individual studies may indicate methodological differences or specific population effects.

7. What is linked data?

Linked data are measurements that are not independent of each other. This often occurs in clinical trials when, for example, the same patient is measured multiple times (e.g., before and after treatment) or when observations exist within pairs or groups (e.g., twins or devices tested on the same patient).

With related data, one measurement directly influences another. The most well-known examples are before-and-after measurements or paired samples. In such cases, it is important to use statistical methods that account for this dependency, otherwise incorrect conclusions may be drawn.

Typical scenario:Data collection before and after a patient's treatment. Since both measurements come from the same patient, they are not independent.

Example

A study is investigating the effectiveness of a new wound dressing in accelerating post-operative healing. Healing time is measured in the same patients before and after application of the dressing. Since both measurements are taken from the same patient, they are related. A simple comparison of means without considering this relationship would lead to biased results. Instead, a paired t-test should be used to correctly analyze the differences in healing times.

7.1 Further Study

Why is dependency important?Independent data adhere to the basic assumption of many statistical tests. However, with related data, this dependency violates the assumption. Therefore, a specific analysis is required to avoid biased results.
Suitable statistical methods:

Paired t-test:This test compares the means of two related groups by analyzing the differences between the pairs.
Wilcoxon signed-rank test:This is the non-parametric alternative when the data are not normally distributed.
Linear mixed models (LMM):These models are particularly useful in complex study designs with multiple time points or groups. They can analyze random effects (e.g., individual differences) and fixed effects (e.g., treatment) simultaneously.

Variance-covariance structure: In advanced models such as repeated-measures ANOVA, the relationship between measurements must be modeled correctly. Different assumptions about the structure (e.g., compound symmetry or autoregressive structures) influence the results.
Practical challenges:

Missing values:Linked data are particularly susceptible to bias when measurements are missing. Methods such as multiple imputation or maximum likelihood estimation can help minimize distortions.
Complexity:Analyzing related data often requires specialized software and knowledge of advanced statistical methods.

8. Difference between sensitivity and specificity

Sensitivity and specificity are fundamental measures for evaluating the quality of a diagnostic test. They describe how well a test is able to identify sick individuals and correctly exclude healthy individuals.

Sensitivity: The proportion of truly ill individuals correctly identified by the test (true positives). It measures the ability to avoid overlooking sick individuals.

Specificity: The proportion of truly healthy individuals who are correctly identified as healthy (true negatives). It describes how well the test can avoid false alarms.

Why is this important? A perfect test would have a sensitivity and specificity of 100%. In practice, however, compromises often have to be made, for example in mass screenings where a test with high sensitivity is preferred to avoid missing any sick people.

Example

A test for diagnosing a rare disease includes:

90% sensitivity:Of 100 actually sick patients, the test correctly identifies 90; 10 are incorrectly classified as healthy.
80% specificity:Of 100 healthy individuals, 80 are correctly identified as healthy; 20 are incorrectly classified as ill.

8.1 Further Study

Relationship with prevalence:

The positive and negative predictive values (PPV and NPV) depend directly on the prevalence of the disease. A test with high sensitivity could produce many false-positive results at low prevalence.

ROC curves and thresholds:

A receiver operating characteristic (ROC) curve shows how the sensitivity and specificity of a test change at different threshold values. The ideal threshold maximizes sensitivity and specificity while minimizing false positives and false negatives. The area under the ROC curve (AUC) is a measure of the overall performance of the test.

Trade-offs between sensitivity and specificity:

Tests with high sensitivity (e.g., screening tests) often have lower specificity and produce more false-positive results. Combined testing strategies (e.g., a sensitive screening test followed by a specific confirmatory test) can improve diagnostic accuracy.

Bayesian analysis:

Bayesian analyses allow the calculation of the probability that a patient is actually ill, based on a positive test result and known prevalence. This helps to better inform diagnostic decisions.

Practical applications:

Diagnostic tests such as COVID-19 antigen tests or mammography screenings.
Evaluation of new diagnostic devices or methods in clinical trials.

9. Conclusion

Statistical methods are indispensable tools in clinical research and medical device development. They enable precise data analysis, the quantification of uncertainties, and informed decision-making. From calculating confidence intervals and avoiding first- and second-order errors to interpreting boxplots and forestplots, statistics offers a wide range of techniques to improve the quality and validity of clinical studies. The targeted application of these methods allows us not only to demonstrate the efficacy and safety of medical devices but also to meet regulatory requirements and ultimately optimize patient care. In a world where data plays an increasingly important role, statistics remains an essential component of evidence-based medicine.

10. How we can help you

Do you already have some initial questions?

You can get a free initial consultation here: free initial consultation

Blog

From figures to findings - statistics in practice

Main menu