Ease of use meets evidence: The interface between usability and clinical evaluation

In this blog post, you'll learn how to systematically identify potential use-related hazards and capture them in risk management, accurately translate these use risks into clinical endpoints, and then support them with robust evidence using formative and summary usability studies and simulations. You'll learn how to seamlessly integrate the obtained usability data into your benefit-risk analysis and clinical evaluation report.

Abbreviations

MDR	Medical Device Regulation (EU Ordinance 2017/745)
Sota	State of the Art (state of the art)
CEP	Clinical evaluation plan
CERIUM	Clinical Evaluation Report
Pmcf	Post-Market Clinical Follow-up
Pms	Post-Market Surveillance
USAB	Usability

 

Underlying regulations and norms

 

EU Regulation 2017/745 (MDR)

En ISO 14971

IEC 62366-1

IEC 60601-1-6

1 Introduction

Clinical evaluation and usability (USAB) are still viewed as separate disciplines in many companies – different teams, different documents, different timelines.

However, the reality under the Medical Device Regulation (MDR, EU 2017/745) is different:
Clinical safety or clinical performance cannot be credibly demonstrated if the device is not fit for use in the intended application.

The MDR requires manufacturers to demonstrate that a medical device:

• is safe
• provides the intended service
• can be used safely and effectively in the hands of the intended user, in the intended application context

This makes it clear: Data from usability engineering are not a “nice-to-have,” but a central component of clinical evaluation and flow directly into the benefit-risk assessment.

2. Regulatory framework

The regulatory framework for linking usability engineering and clinical evaluation is based primarily on the MDR and the international standards ISO 14971, IEC 62366-1, and IEC 60601-1-6. According to Annex I of the MDR, medical devices must be designed to reduce usage-related risks to an acceptable minimum.

ISO 14971 standard integrates use-related hazards into risk management: Every potential user error is recorded as a hazard, assessed, and subjected to risk control. Remaining residual risks must be documented and justified in the benefit-risk analysis in the Clinical Evaluation Report (CER).

IEC 62366-1 standard defines the usability engineering process for all medical devices: starting with the creation of a use specification, continuing with the formal analysis of all use-related hazards and culminating in the summative evaluation. The data obtained (e.g., error rates, processing times) provide precise endpoints that are defined in the Clinical Evaluation Plan (CEP) and later evaluated in the CER.

IEC 60601-1-6 supplements these specifications for electrical medical devices with specific requirements for user interfaces, alarm and display design: For example, the readability of displays and the comprehensibility of alarm messages must be validated and documented.

3. What does usability mean?

In the context of medical devices, the term usability describes much more than "user-friendly design."
According to IEC 62366-1, usability is "the property of the user interface that supports use and thus achieves effectiveness, efficiency, and user satisfaction in the specified usage environment."

This definition makes it clear that usability is not just about aesthetic aspects or intuitive operation, but about the safe , effective and error-free use of a medical device - by the intended users, in the intended application scenarios and environments.

A usable product reduces the likelihood of use errors and thus directly contributes to patient safety and the fulfillment of regulatory requirements.

Conversely, poor usability and unclear user guidance can lead to risks that cannot be adequately compensated for by either product design or training.

3.1 Usability in the MDR

The MDR explicitly recognizes the importance of usability and enshrines it in several Essential Safety and Performance Requirements (GSPR):

• GSPR 5 – “Risks due to user errors shall be avoided or minimized through design and construction.”
  → This means: Manufacturers must actively identify which user errors could occur and eliminate or reduce these risks through design decisions during the development phase.
• GSPR 14.2(a) – “Risks of injury shall be minimized in conjunction with the physical characteristics of the product – including volume/pressure ratio, dimensions and, where applicable, ergonomic features.”
  → Ergonomics is not an optional comfort factor here, but a safety-relevant criterion that flows directly into the product design.
• GSPR 14.6 – "Measuring, control, or display devices shall be designed ergonomically, taking into account their intended purpose, the intended users, and the environmental conditions."
  → This applies, for example, to the readability of displays, logical menu navigation, the design of control buttons, or acoustic signals – everything must be adapted to the user and their environment.
• GSPR 22 – “Particular consideration of the abilities and limitations of lay users.”
  → Devices for home use or for patients themselves must be designed so that even persons without medical training can use them safely and correctly.

3.2 Why is this important?

A product that meets all regulatory requirements but is complicated or unclear to use in practice fails to meet its clinical performance requirements.
Usability is therefore not merely a design consideration, but an integral component of the safety and performance assessment and directly impacts the benefit-risk analysis in clinical evaluation.

In practice this means:

• Usability should be considered early in the development process (not at the end)
• Results from usability tests should be incorporated into risk management, IFU, labeling and clinical evaluation
• If changes are made to the user interface, risks must be reassessed and usability must be revalidated if necessary

4. The usability engineering process – more than just testing

Before linking usability and clinical evaluation can be successful, key terms and the underlying process must be clearly defined.

Use Specification:
The use specification comprehensively describes how and by whom a medical device is used. It defines user groups (e.g., nurses, physicians, lay users), application environments (hospital ward, home care, emergency use), and usage scenarios (routine injection, emergency response, long-term monitoring). A precise use specification serves as the basis for all subsequent steps, as it defines the context in which risks can occur.

Use-Related Hazards
A use-related hazard is a potential hazard that can arise from the use of the product, not from technical malfunctions, but from misuse, operating errors, or misunderstandings. Examples include confusing controls, inadequate cleaning, or unclear menu navigation. Each use-related hazard is identified in a formalized hazard analysis and documented in risk management.

Formative Evaluation:
In formative evaluation, prototypes or early product versions are iteratively tested with representative users. The goal is not to definitively demonstrate security, but rather to identify and resolve usability vulnerabilities early on. Common methods include task analyses, eye-tracking, think-aloud protocols, and structured observations. The insights gained are directly incorporated into design optimizations, ensuring that all the most serious use-related hazards are mitigated in advance in the final product.

Summative Evaluation:
The summative evaluation is the final, standards-compliant usability test. With the final product version, you conduct structured tests with typical users under realistic conditions and defined boundary parameters. The resulting key metrics, such as error rates, processing times, and success rates for safety-relevant tasks, are quantitatively evaluated.

Process overview

1. Create use specifications
   to define user profiles, application environments and usage scenarios.
2. Use-Related Hazard Analysis
   Identification and documentation of potential misuses in the risk register (integration into ISO 14971 process).
3. Iterative formative evaluation
   Early testing of prototypes, task analyses and usage observation for design optimization.
4. Implementation of mitigation measures
   : design adjustments, revised IFUs, training materials to minimize identified hazards.
5. Conduct summative evaluation,
   final tests with the final product, quantitative recording of the results.
6. Integration into the CEP/CER
   Transfer of the results into the Clinical Evaluation Plan and Clinical Evaluation Report, linking with benefit-risk analysis and risk management.

This structured process ensures that usability and clinical evaluation do not occur separately, but work in close coordination to ensure a safe, intuitive, and evidence-based medical device.

5. Interface: Usability → Clinical Evaluation

Transferring usability results into clinical evaluation ensures that risks typical of use are not only technically addressed but also validated. The process is divided into four consecutive steps:

5.1 Identification and prioritization of use-related hazards

In the first step, all potential misuses and operating errors are systematically recorded:

• from the use specification (e.g. switching on the device, making settings, reading parameters).
• In task analyses and workshops with representatives of the user groups (nursing staff, doctors, and possibly laypeople), you identify use-related hazards such as incorrect button selection, confusion of modes, or unclear displays.
• Using the risk table according to ISO 14971, you evaluate each hazard in terms of severity, probability of occurrence, and detectability. The prioritized risks form the basis for the clinical question.

5.2 Translation into endpoints

For each prioritized use-related risk, define one or more measurable endpoints :

• Error rate for safety-relevant tasks (e.g., percentage of users who fail to reset an alarm correctly)
• Average processing time until a critical process is successfully completed (e.g., start injection)
• Number of unnoticed alarms within a defined observation period.
  These endpoints must be clearly operationalized (measurement method, number of subjects, test conditions) and contain quantifiable acceptance criteria (e.g. error rate ≤ 2%).

5.3 Integration into benefit-risk analysis and CER

In the clinical evaluation, you link the usability endpoint results with the use-related risks documented in risk management, e.g., as:

• Tabular benefit-risk matrix: Each use-related hazard is compared to the measured endpoint, including quantitative results and assessment of the residual risk.
• Narrative evaluation: Explanation of how reducing the error rate or shortening processing times improves the clinical benefit-risk profile (e.g., faster therapy, lower patient burden).
• Claim derivation: Formulate precise claims based on the endpoint data (e.g., “Alarm reset successful in 98% of cases within 5 seconds”).

6. Example: Usability interface

To illustrate the procedure, let's look at a digital infrared ear thermometer:

1. ID

• Use-Related Hazard: False-negative temperature measurement due to improper probe alignment

2. Translation in endpoint

• Endpoint: Percentage of measurements whose deviation from the reference thermometer is ≤ ± 0.3 °C
• Acceptance criterion: ≥ 95% of measurements must meet the criterion

3. Evidence strategy

• Summative usability study:
  – n = 100 subjects (mixed user groups to cover the entire patient population)
  – Measurement in real home environments
  – Documentation of deviations and the number of incorrectly placed probes
• Supplementary simulation:
  – Laboratory bench test to determine the dependence of measurement accuracy on insertion direction (± 5° steps)

4. Integration into benefit risk analysis

Use-Related Hazard	Endpoint	Result	Benefit-Risk Commentary
False-negative measurement due to incorrect probe alignment	≥ 95% of the measured values within ± 0.3 °C	96% fulfilled	Significantly reduces the risk of undetected fever

5. Narrative

"In the summative study, 96% of users achieved the required measurement accuracy, keeping the residual risk of undetected fever cases to a minimum. The supplementary bench tests confirm that even small deviations in probe alignment only marginally affect accuracy."

 

6. Claim derivation
   “The digital infrared ear thermometer offers ≥ 95% measurement accuracy (± 0.3 °C) in real-world application environments.”

 This example illustrates how a use-related hazard is transformed into a precise clinical endpoint, which is validated through usability studies, documented in the CER, and finally converted into a quantifiable claim.

7. Conclusion

The consistent integration of usability into clinical evaluation not only increases user-friendliness, but is also essential for patient safety and regulatory compliance. By systematically identifying and prioritizing use-related hazards, deriving precise clinical endpoints, and planning and conducting formative and summative usability studies, you create robust evidence. The structured linking of this data with risk management and the benefit-risk analysis in the CER ensures seamless traceability and audit-proof documentation. This lays the foundation for medical devices that are both intuitive to use and clinically safe, meeting the requirements of users, auditors, and authorities.

8. How we can help you

We support you from the initial use specification to the final benefit-risk analysis, ensuring that your usability results flow seamlessly into the clinical evaluation. In joint workshops, we first develop your user profile, typical use cases, and identify all potential use-related hazards. We then plan and conduct both formative prototype tests and summative usability studies with representative user groups to generate reliable endpoint data.

We integrate this data directly into your CEP and CER. Together, we define precise endpoints, document the study results statistically, and link them to the corresponding risks in risk management.

In addition, based on your usability metrics, we formulate measurable, regulatory-compliant claims for your benefit-risk analysis and advise you on compliance with ISO 14971, IEC 62366-1, and IEC 60601-1-6 standards. This ensures that your medical devices are not only safe and intuitive to use, but also undergo clinically convincing evaluations and are approved quickly.

Want to know more? Contact us for a free initial consultation!

You can get a free initial consultation here: free initial consultation