Keywords: Sample size planning, clinical study, clinical trial
Secondary keywords: Sample size, case number calculation
1 Introduction
Sample size planning plays a crucial role in the planning of a clinical trial. This determines how many participants need to be included to demonstrate a relevant effect – and thus ultimately the success or failure of the study. What considerations are involved in this process?
To demonstrate the efficacy of any clinical trial, such as PMCF or registration studies, hypotheses are tested against a primary endpoint. A hypothesis to be tested (called an alternative hypothesis) could be, for example, the superiority of a product over a standard therapy. Confirmation or rejection of a hypothesis is assessed based on collected data, and the results are then extrapolated to the entire population, i.e., the entire target group. For this to be meaningful, a sufficient number of observational data from the target group must be available. If there are too few observations, actual treatment effects cannot be demonstrated, and the study fails. On the other hand, a large sample size leads to high costs, is ethically difficult to justify, ties up resources, and extends the study duration.
Sample size planning determines the minimum number of patients or study participants to be included in order to demonstrate an actual effect. A number of preliminary considerations are crucial for this.
2. Reasons for case number planning
The goal of every confirmatory clinical trial is to statistically prove a hypothesis. If the sample size is too small, a difference actually existing between two treatment groups cannot be demonstrated. This results in a non-significant statistical test, even though effects do indeed exist.
On the other hand, data collection is very time-consuming, ties up personnel resources, and incurs costs for each additional patient included. Furthermore, recruiting too many patients can lead to even small, medically irrelevant effects being detected.
A sample size plan for a clinical trial thus ensures that
- An effect present in the target group is detected by the statistical test, i.e., the test delivers a significant result
- If the statistical test does not show a significant result, a sufficient sample size ensures that there is also no effect in the target group (population) with a sufficiently high degree of certainty.
The necessity of sample size planning during the planning phase of clinical trials is also legally mandated and reviewed by the ethics committee. Calculating the sample size is an essential part of both the clinical trial protocol and the statistical analysis plan.
For prospective study designs, sample size planning before the start of the study is essential, but even in pilot studies or retrospective studies, consideration should be given in advance to how high the minimum sample size must be.
Aspects of case number planning
Physicians, principal investigators, statisticians, and CROs work closely together on sample size planning. The starting point is always the primary endpoint and the hypothesis to be tested in the clinical trial.
3. Selection of statistical tests
The type of research question is crucial for selecting the appropriate statistical test. Different test procedures are required depending on whether the goal is to demonstrate the superiority or equivalence of a treatment. The scale level of the primary outcome variable also plays a decisive role. Different methods are used for nominal variables (yes/no, success/no success) than for ordinal variables (e.g., Likert scale) or continuous variables (e.g., visual analog scale (VAS), sum scores, etc.).
3.1 Effect size
The effect size indicates the relevant difference to be detected. Different measures are used depending on the test procedure employed. For continuous variables, the most well-known effect size is Cohen's d, which expresses the difference between two independent groups relative to their joint variance.
For categorical endpoints, the effect size W is used, which is calculated as the square root of the relative difference of the proportions.
According to Cohen (1988), the following rules of thumb roughly apply:
Effect size ≈ 0.2: small effect
Effect size ≈ 0.5: medium effect
Effect size ≈ 0.8: large effect
Determining the effect size requires the most precise possible preliminary information from the literature or our own pilot studies. The medically and practically relevant difference that can be demonstrated is also taken into account. A mean blood pressure reduction of just a few mmHg, i.e., a very small effect size, can be statistically proven with a sufficiently large sample size, but is practically irrelevant for both patients and physicians.
3.2 Significance level of the statistical test
The significance level α must be defined in advance and documented in the study protocol and the statistical analysis plan (SAP). The significance level indicates the probability of obtaining a statistically significant test result if no effect is actually present in the target group. A further distinction is made between one-sided and two-sided tests. One-sided tests examine superiority hypotheses. Two-sided tests, which compare the effects of two therapies, are more common. A significance level of α = 5% has become established, while α = 2.5% is often used for one-sided tests.
3.3 Power or Might
During the planning phase, the study's power is also determined. This refers to the probability that a statistical test will detect the actual difference, i.e., yield a significant p-value. The power of a study should therefore be as high as possible. Values between 80% and 90% are common. The higher the power of a study, the larger the resulting sample size.
4. Example from our NOVUSSTAT consulting practice
As part of a clinical trial, the improvement in quality of life, as measured by the score on the "Physical Functioning" scale of the SF-36 questionnaire, will be demonstrated after a 3-month therapy. The scale ranges from 0 to 100 points. The measurement instrument is well-documented and validated, and numerous publications exist using this instrument. The norm table from the German National Health Survey [1] shows that healthy individuals aged 40-70 years have a mean score of 80-90 with a standard deviation of approximately 20 points. For the study population, this physical functioning level at baseline (before therapy) is expected to be 50 points (standard deviation 25 points), as results from a pilot study have shown. After three months of therapy, the aim is to achieve an improvement in physical functioning of 30 points, so that the mean functional level after therapy corresponds to that of healthy individuals of the same age. A low correlation of 0.2 is expected (and confirmed by the data from the pilot study) between the first measurement before therapy and the second measurement after 3 months of therapy, due to the time interval.
If you enter these values into G*Power, a software for calculating sample size, you get the following result:
Fig. 1 Calculation of the effect size
Based on the given information and prior data, an effect size of 0.949, or approximately 1, is obtained. This information is now needed to calculate the minimum required sample size to detect an effect of d = 0.949.
For a normally distributed variable, a two-sided paired t-test can be used to confirm the finding. With a 5% significance level and a power of 90%, at least 14 observations are required (see Figure 2).
Fig. 2 Calculation of sample size for a two-sided paired t-test.
Taking into account a drop-out rate of 10%, at least 1.1*14 = 15.4, i.e. 16 patients, must be recruited.
In a subsequent sensitivity analysis, we examine how sensitive the sample size is to deviations from the assumptions. This can be achieved by varying the effect size within reasonable limits, or by using a non-parametric alternative to determine the sample size. Reducing the power reduces the required sample size.
A graphical sensitivity analysis can be seen in Figure 3.
Fig. 3 Sensitivity analysis: Sample size depending on effect size and study power
5. Sources/Literature
- Sample size planning in clinical trials
- Chow S, Shao J, Wang H. 2008. Sample Size Calculations in Clinical Research. 2nd Ed. Chapman & Hall/CRC Biostatistics Series.
- Bock J., Determining the sample size for biological experiments and controlled clinical trials. Oldenbourg 1998
6. What we can do for you
Before the start of a clinical trial, sample size planning is a crucial part of the preparation. Calculating the sample size ensures that the actual effect can be demonstrated. Professional sample size planning ensures that the sample size remains as small as possible. Sample size planning is tailored to the specific trial, taking into account the study design, the primary outcome variable, the hypothesis to be tested, and the required level of certainty.
Therefore, our study planning always includes sample size planning as a first step. The entire study concept is based on this. Subsequent planning (e.g., how many study centers are needed? How long will recruitment take? etc.) can then build upon this foundation.
We would like to take this opportunity to thank our partner Novustat for the guest contribution, as we believe that this topic is often underestimated.
About the author: "Dr. Robert Grünwald has been self-employed for 6 years with the statistics consultancy Novustat and, together with his team, primarily advises clients from the pharmaceutical, medical technology and industrial sectors on all questions relating to statistical analyses."
Statistics consultancy Novustat
8. Outlook
In one of the next blog posts, we will revisit the topic of "study types" and take a closer look at the approval study according to MDR Article 62.
9. How we can help you
At medXteam we clarify whether and if so which clinical trial needs to be carried out under what conditions and according to what requirements during the pre-study phase: In 3 steps we determine the correct and cost-effective strategy in relation to the clinical trial required in your case Data collection.
Do you already have some initial questions?
You can get a free initial consultation here: free initial consultation
[1] https://www.thieme.de/statics/dokumente/thieme/final/de/dokumente/zw_das-gesundheitswesen/gesu-suppl_klein.pdf