Answer

How many participants do you need when comparing several treatment arms?

Inspired by a question on Cross Validated · October 31, 2026

power analysisanovamultiple comparisons

The short answer

A study with several arms (say a control group and two treatments, or three competing treatments) is usually analysed with a set of specific comparisons, not only the overall one-way ANOVA. The F test tells you that the group means are not all equal; it does not tell you which treatment beat the control. So the sample size should be driven by the comparisons that answer your research question.

• Write down the comparisons you will report. Two treatments against one control gives two comparisons; all pairs among three groups gives three.
• Decide on the alpha for each comparison. If you correct for multiple testing (for example Bonferroni, with α = .05 split across the comparisons), use the corrected alpha in the power analysis.
• Run an ordinary two-group power analysis for one comparison at that alpha and with the smallest difference you care about, and give each group that size. Because every comparison uses the same effect size and the same alpha, one calculation covers all of them.
• Decide what "80% power" should mean. Power for each comparison on its own is not the same as power to find every true effect in the study, which is lower.

See it in R and Python

The example assumes a control group with mean 0 and two treatments with mean 0.5, all with a standard deviation of 1, so each treatment differs from the control by d = 0.5. It finds the per-group size for 80% power from the omnibus ANOVA, then from a single treatment-versus-control t test at α = .05 and at Bonferroni-corrected levels for two and three comparisons. Finally it simulates 10,000 studies to see how often one, at least one, and both treatments come out significant against the shared control. Both calculations use only base R (power.anova.test and power.t.test).

set.seed(467595)
means <- c(control = 0, t1 = 0.5, t2 = 0.5)  # assumed group means; within-group SD = 1

# 1. Omnibus one-way ANOVA: n per group for 80% power at alpha = .05
f <- sqrt(mean((means - mean(means))^2)) / 1  # Cohen's f
aov_n <- power.anova.test(groups = 3, between.var = var(means),
                          within.var = 1, power = 0.8)$n
round(c(cohens_f = f, anova_n_per_group = aov_n), 3)

# 2. One treatment vs control (d = 0.5): n per group for 80% power
n_for <- function(alpha) ceiling(power.t.test(delta = 0.5, sd = 1,
                                 power = 0.8, sig.level = alpha)$n)
c(alpha_.05 = n_for(0.05), alpha_.025 = n_for(0.05 / 2),
  alpha_.0167 = n_for(0.05 / 3))

# 3. Simulate both treatment-vs-control tests with a shared control group
sim <- function(n, alpha, reps = 10000) {
  res <- replicate(reps, {
    ctl <- rnorm(n, 0); y1 <- rnorm(n, 0.5); y2 <- rnorm(n, 0.5)
    p1 <- t.test(y1, ctl, var.equal = TRUE)$p.value
    p2 <- t.test(y2, ctl, var.equal = TRUE)$p.value
    c(t1 = p1 < alpha, at_least_one = p1 < alpha | p2 < alpha,
      both = p1 < alpha & p2 < alpha)
  })
  round(rowMeans(res), 3)
}
rbind(n64_alpha.05 = sim(64, 0.05), n78_alpha.025 = sim(78, 0.025))

import numpy as np
from scipy import stats, optimize

rng = np.random.default_rng(467595)
means = np.array([0, 0.5, 0.5])  # control, t1, t2; within-group SD = 1

# 1. Omnibus one-way ANOVA: n per group for 80% power (same formula as power.anova.test)
f = np.sqrt(np.mean((means - means.mean()) ** 2)) / 1  # Cohen's f
def anova_power(n, k=3, alpha=0.05):
    df1, df2 = k - 1, k * (n - 1)
    ncp = (k - 1) * n * np.var(means, ddof=1) / 1
    return stats.ncf.sf(stats.f.ppf(1 - alpha, df1, df2), df1, df2, ncp)
aov_n = optimize.brentq(lambda n: anova_power(n) - 0.8, 2, 1000)
print("cohens_f", round(f, 3), "anova_n_per_group", round(aov_n, 3))

# 2. One treatment vs control (d = 0.5): n per group for 80% power (as power.t.test)
def t_power(n, alpha, d=0.5):
    df = 2 * n - 2
    return stats.nct.sf(stats.t.ppf(1 - alpha / 2, df), df, d * np.sqrt(n / 2))
for alpha in (0.05, 0.05 / 2, 0.05 / 3):
    n = optimize.brentq(lambda n: t_power(n, alpha) - 0.8, 2, 1000)
    print(f"alpha {alpha:.4f}: n = {int(np.ceil(n))}")

# 3. Simulate both treatment-vs-control tests with a shared control group
def sim(n, alpha, reps=10000):
    ctl = rng.normal(0, 1, (reps, n))
    y1, y2 = rng.normal(0.5, 1, (reps, n)), rng.normal(0.5, 1, (reps, n))
    s1 = stats.ttest_ind(y1, ctl, axis=1).pvalue < alpha
    s2 = stats.ttest_ind(y2, ctl, axis=1).pvalue < alpha
    return {"t1": s1.mean(), "at_least_one": (s1 | s2).mean(), "both": (s1 & s2).mean()}
for n, alpha in ((64, 0.05), (78, 0.025)):
    print(f"n{n}_alpha{alpha}", {k: round(float(v), 3) for k, v in sim(n, alpha).items()})

         cohens_f anova_n_per_group 
            0.236            58.818 
  alpha_.05  alpha_.025 alpha_.0167 
         64          78          86 
                 t1 at_least_one  both
n64_alpha.05  0.806        0.915 0.693
n78_alpha.025 0.799        0.920 0.690

The figures come from one seeded run of the R code. The sample-size calculations involve no randomness, and Python reproduces them exactly. Python's random numbers differ from R's, so its simulated rates differ slightly (for example 69.7% instead of 69.3% for both treatments at 64 per group), but they show the same pattern.

What the numbers show

• The ANOVA calculation asks for fewer people. For this pattern of means (Cohen's f = 0.236), the F test reaches 80% power with 59 per group (177 in total). But a significant F only says the three means are not all equal. It does not guarantee that either treatment-versus-control comparison will be significant.
• One comparison at α = .05 needs 64 per group (192 in total). The simulation confirms that each treatment then beats the control about 80% of the time (80.6%).
• A Bonferroni correction raises the requirement. Splitting α = .05 across two comparisons (α = .025 each) needs 78 per group (234 in total); across three comparisons (α ≈ .0167 each) it needs 86 per group (258 in total). At 78 per group with α = .025, each comparison again had about 80% power (79.9%).
• Power to find both effects is much lower. Even with 80% power for each comparison, both treatments came out significant in only 69.3% of simulated studies at 64 per group, and 69.0% at 78 per group with the correction. At least one was significant about 92% of the time. If your conclusion needs every arm to succeed, plan for that joint power directly, which takes a larger sample.

The joint power is a little higher than you would get by multiplying 0.806 by itself (about 0.65), because the two comparisons share a control group: when the control mean happens to come out low, both treatments look better at the same time. This correlation is also why an exact many-to-one method such as Dunnett's test is slightly less conservative than Bonferroni for treatment-versus-control comparisons, so a Bonferroni-based sample size errs on the safe side.

Choices that change the answer

• Whether to correct at all. If each treatment answers a separate question (two different drugs, each with its own claim), many researchers test each at α = .05. If a single claim rests on any one arm "working", the chance of a false claim grows with the number of arms, and a correction is expected.
• Decide in advance. Choose the comparisons and the correction before collecting data and say which you used. See When should you use a Bonferroni correction? and Planned contrasts vs post hoc tests.
• Unequal effect sizes. If you expect one treatment to have a smaller effect than another, size the study for the smallest difference that matters, since that comparison needs the most people.
• A bigger control group. When several treatments are each compared with one control, the control is used in every comparison. A common rule of thumb is to make the control group about √k times the size of each treatment group, where k is the number of treatments, which gives a bit more power for the same total sample.
• Covariates. If the analysis will adjust for a strong baseline predictor (as in ANCOVA or a regression with a pre-test), the residual standard deviation shrinks and fewer people are needed. Use the expected residual standard deviation in the calculation, and only if you are confident about it.

This page is about planning a study with several arms. For the separate question of whether a two-group t test has a minimum sample size, see Does a t test need a minimum sample size?. To read the F test itself, see How to interpret the F statistic in ANOVA. For more plain-language guides to planning and reporting analyses, see the DASS blog.

How to report a power analysis in APA style (7th edition)

Report an a priori power analysis in the method section, naming every input so a reader can repeat it. Using the example above:

"An a priori power analysis was conducted for the two planned comparisons of each treatment with the control. To detect a difference of d = 0.50 with 80% power in a two-sided independent-samples t test at a Bonferroni-adjusted α = .025, 78 participants were required per group (N = 234). A simulation of 10,000 studies with this design indicated a power of .69 to detect both treatment effects."

Name the software or function you used (here, power.t.test in R), state where the assumed effect size came from (earlier studies, a pilot, or the smallest difference of practical importance), and say whether the planned sample was reached.

Related tools and guides

• Power and sample size calculators
• ANOVA power and sample size calculator
• APA 7 formatter for ANOVA
• Which statistical test should I use?
• Does a t test need a minimum sample size?
• When should you use a Bonferroni correction?
• Planned contrasts vs post hoc tests
• Power of a test (Wikipedia)

More answered questions

• Significant ANOVA but no significant pairwise comparisons: how can that happen?
• Planned contrasts vs post hoc tests: why do they give different p-values?
• Why does a t test use the t distribution and ANOVA the F distribution?

Written by AskStats with AI assistance. This is general information, not advice for your specific data or study. When the results matter, check your approach with a qualified statistician.