Ease of use meets evidence: The interface between usability and clinical evaluation

In this blog post, you'll learn how to systematically identify potential use-related hazards and capture them in risk management, accurately translate these use risks into clinical endpoints, and then support them with robust evidence using formative and summary usability studies and simulations. You'll learn how to seamlessly integrate the obtained usability data into your benefit-risk analysis and clinical evaluation report.

Abbreviations

MDR	Medical Device Regulation (EU Ordinance 2017/745)
Sota	State of the Art (state of the art)
CEP	Clinical evaluation plan
CERIUM	Clinical Evaluation Report
Pmcf	Post-Market Clinical Follow-up
Pms	Post-Market Surveillance
USAB	Usability

 

Underlying regulations and norms

 

EU Regulation 2017/745 (MDR)

En ISO 14971

IEC 62366-1

IEC 60601-1-6

1 Introduction

Clinical evaluation and usability (USAB) are still viewed as separate disciplines in many companies – different teams, different documents, different timelines.

However, the reality under the Medical Device Regulation (MDR, EU 2017/745) is different:
Clinical safety or clinical performance cannot be credibly demonstrated if the product is not fit for its intended use.

The MDR requires manufacturers to demonstrate that a medical device:

• is safe
• provides the intended service
• can be used safely and effectively in the hands of the intended user, in the intended application context

This makes it clear: Data from usability engineering are not a “nice-to-have,” but a central component of clinical evaluation and flow directly into the benefit-risk assessment.

2. Regulatory framework

The regulatory framework for linking usability engineering and clinical evaluation is based primarily on the MDR and the international standards ISO 14971, IEC 62366-1, and IEC 60601-1-6. According to Annex I of the MDR, medical devices must be designed to reduce usage-related risks to an acceptable minimum.

The ISO 14971 standard integrates use-related hazards into risk management: Every potential user error is recorded as a hazard, assessed, and subjected to risk control. Remaining residual risks must be documented and justified in the benefit-risk analysis within the Clinical Evaluation Report (CER).

The IEC 62366-1 standard defines the usability engineering process for all medical devices: starting with the creation of a use specification, through the formal analysis of all use-related hazards, to the summative evaluation. The data obtained (e.g., error rates, processing times) provide precise endpoints that are documented in the Clinical Evaluation Plan (CEP) and later evaluated in the Clinical Evaluation Review (CER).

IEC 60601-1-6 supplements these requirements for electrical medical devices with specific requirements for user interfaces, alarm and display design: For example, the readability of displays and the comprehensibility of alarm messages must be validated and documented.

3. What does usability mean?

In the context of medical devices, the term usability describes far more than just "user-friendly design".
According to IEC 62366-1, usability is the "property of the user interface that supports use and thus achieves effectiveness, efficiency and user satisfaction in the specified usage environment".

This definition clarifies that usability is not just about aesthetic aspects or intuitive operation, but about the safe, effective and error-free use of a medical device by the intended users, in the intended application scenarios and environments.

A usable product reduces the likelihood of use errors and thus directly contributes to patient safety and the fulfillment of regulatory requirements.

Conversely, poor usability and unclear user guidance can lead to risks that cannot be adequately compensated for by either product design or training.

3.1 Usability in the MDR

The MDR explicitly recognizes the importance of usability and enshrines it in several Essential Safety and Performance Requirements (GSPR):

• GSPR 5 – “Risks arising from user error should be avoided or minimized through design and construction.”
  → This means: Manufacturers must actively identify which user errors could occur and eliminate or reduce these risks through design decisions as early as the development phase.
• GSPR 14.2(a) – “Risks of injury must be minimized in conjunction with the physical characteristics of the product – including volume/pressure ratio, dimensions and, where applicable, ergonomic features.”
  → Ergonomics here is not an optional comfort factor, but a safety-relevant criterion that is directly incorporated into the product design.
• GSPR 14.6 – “Measuring, control or display devices shall be designed ergonomically, taking into account their intended purpose, the intended users and the environmental conditions.”
  → This includes, for example, the readability of displays, logical menu navigation, the design of control buttons or acoustic signals – everything must be adapted to the user and their environment.
• GSPR 22 – “Special consideration given to the abilities and limitations of lay users.”
  → Devices for home use or for patients themselves must be designed so that even persons without medical training can operate them safely and correctly.

3.2 Why is this important?

A product that meets all regulatory requirements but is complicated or confusing to use in practice will fail to deliver its clinical performance.
Usability is therefore not just a design aspect, but an integral part of safety and performance evaluation and directly impacts the benefit-risk analysis in clinical assessment.

In practice this means:

• Usability should be considered early in the development process (not at the end)
• Results from usability tests should be incorporated into risk management, IFU, labeling and clinical evaluation
• If changes are made to the user interface, risks must be reassessed and usability must be revalidated if necessary

4. The usability engineering process – more than just testing

Before linking usability and clinical evaluation can be successful, key terms and the underlying process must be clearly defined.

Use Specification:
The use specification comprehensively describes how and by whom a medical device is used. It defines user groups (e.g., nurses, physicians, lay users), application environments (hospital ward, home care, emergency response), and usage scenarios (routine injection, emergency response, long-term monitoring). A precise use specification serves as the basis for all further steps, as it defines the context in which risks can occur.

Use-Related Hazards:
A use-related hazard is a potential danger that can arise from the use of the product, not from technical malfunctions, but from misuse, operator error, or misunderstandings. Examples include confusing controls, inadequate cleaning, or unclear menu navigation. Each use-related hazard is identified in a formalized hazard analysis and documented in the risk management process.

Formative Evaluation:
In formative evaluation, prototypes or early product versions are iteratively tested with representative users. The goal is not definitive proof of safety, but rather the early identification and correction of usability issues. Common methods include task analysis, eye-tracking, think-aloud protocols, and structured observations. The insights gained are directly incorporated into design optimizations, ensuring that all the most serious use-related hazards are mitigated in advance in the final product.

Summative Evaluation
: Summative evaluation is the final, standards-compliant usability test. With the final product version, you conduct structured tests with typical users under realistic conditions and defined parameters. The collected key performance indicators (KPIs), such as error rates, processing times, and success rates for safety-relevant tasks, are quantitatively evaluated.

Process overview

1. Create use specifications
   to define user profiles, application environments and usage scenarios.
2. Use-Related Hazard Analysis
   Identification and documentation of potential misuses in the risk register (integration into ISO 14971 process).
3. Iterative formative evaluation:
   Early testing on prototypes, task analyses and usage observation for design optimization.
4. Implementation of mitigation measures
   , design adjustments, revised IFUs, training materials to minimize identified hazards.
5. Conduct summative evaluation:
   final tests with the final product, quantitative recording of the results.
6. Integration into the CEP/CER:
   Transfer of results into the Clinical Evaluation Plan and Clinical Evaluation Report, linking with benefit-risk analysis and risk management.

This structured process ensures that usability and clinical evaluation do not occur separately, but work in close coordination to ensure a safe, intuitive, and evidence-based medical device.

5. Interface: Usability → Clinical Evaluation

Transferring usability results into clinical evaluation ensures that risks typical of use are not only technically addressed but also validated. The process is divided into four consecutive steps:

5.1 Identification and prioritization of use-related hazards

In the first step, all potential misuses and operating errors are systematically recorded:

• from the use specification (e.g., turning on the device, making settings, reading parameters).
• In task analyses and workshops with representatives of the user groups (nursing staff, doctors, possibly laypersons), you identify use-related hazards such as incorrect button selection, confusion of modes, or unclear displays.
• Using the risk table according to ISO 14971, you assess each hazard with regard to severity, occurrence, and detectability. The prioritized risks form the basis for the clinical question.

5.2 Translation into endpoints

For each prioritized use-related risk, define one or more measurable endpoints:

• Error rate in safety-relevant tasks (e.g., percentage of users who do not correctly reset an alarm)
• Average processing time until a critical process is successfully completed (e.g., start injection)
• Number of ignored alarms within a defined observation period.
  These endpoints must be clearly operationalized (measurement method, number of subjects, test conditions) and include quantifiable acceptance criteria (e.g., error rate ≤ 2%).

5.3 Integration into benefit-risk analysis and CER

In the clinical evaluation, you link the usability endpoint results with the use-related risks documented in risk management, e.g., as:

• Tabular benefit-risk matrix: Each use-related hazard is compared to the measured endpoint, including quantitative results and assessment of the residual risk.
• Narrative evaluation: Explanation of how reducing the error rate or shortening processing times improves the clinical benefit-risk profile (e.g., faster therapy, lower patient burden).
• Claim derivation: Formulate precise claims based on the endpoint data (e.g., “Alarm reset successful in 98% of cases within 5 seconds”).

6. Example: Usability interface

To illustrate the procedure, let's look at a digital infrared ear thermometer:

1. ID

• Use-Related Hazard: False-negative temperature measurement due to improper probe alignment

2. Translation in endpoint

• Endpoint: Percentage of measurements whose deviation from the reference thermometer is ≤ ± 0.3 °C
• Acceptance criterion: ≥ 95% of the measurements must meet the criterion.

3. Evidence strategy

• Summative usability study:
  – n = 100 participants (mixed user groups to cover the entire patient population)
  – Measurement in real home environments
  – Documentation of deviations and the number of incorrectly placed probes
• Supplementary simulation:
  – Laboratory bench test to determine the dependence of measurement accuracy on insertion direction (± 5° steps)

4. Integration into benefit risk analysis

Use-Related Hazard	Endpoint	Result	Benefit-Risk Commentary
False-negative measurement due to incorrect probe alignment	≥ 95% of the measured values within ± 0.3 °C	96% fulfilled	Significantly reduces the risk of undetected fever

5. Narrative

"In the summative study, 96% of users achieved the required measurement accuracy, keeping the residual risk of undetected fever cases to a minimum. The supplementary bench tests confirm that even small deviations in probe alignment only marginally affect accuracy."

 

6. Claim derivation
   : “The digital infrared ear thermometer offers ≥ 95% measurement accuracy (± 0.3 °C) in real application environments.”

 This example illustrates how a use-related hazard is transformed into a precise clinical endpoint, which is validated through usability studies, documented in the CER, and finally converted into a quantifiable claim.

7. Conclusion

The consistent integration of usability into clinical evaluation not only increases user-friendliness, but is also essential for patient safety and regulatory compliance. By systematically identifying and prioritizing use-related hazards, deriving precise clinical endpoints, and planning and conducting formative and summative usability studies, you create robust evidence. The structured linking of this data with risk management and the benefit-risk analysis in the CER ensures seamless traceability and audit-proof documentation. This lays the foundation for medical devices that are both intuitive to use and clinically safe, meeting the requirements of users, auditors, and authorities.

8. How we can help you

We support you from the initial use specification to the final benefit-risk analysis, ensuring that your usability results flow seamlessly into the clinical evaluation. In joint workshops, we first develop your user profile, typical use cases, and identify all potential use-related hazards. We then plan and conduct both formative prototype tests and summative usability studies with representative user groups to generate reliable endpoint data.

We integrate this data directly into your CEP and CER. Together, we define precise endpoints, document the study results statistically, and link them to the corresponding risks in risk management.

In addition, based on your usability metrics, we formulate measurable, regulatory-compliant claims for your benefit-risk analysis and advise you on compliance with ISO 14971, IEC 62366-1, and IEC 60601-1-6 standards. This ensures that your medical devices are not only safe and intuitive to use, but also undergo clinically convincing evaluations and are approved quickly.

Want to know more? Contact us for a free initial consultation!

You can get a free initial consultation here: free initial consultation