Tag

multiple comparisons

8 answered questions tagged “multiple comparisons”. Browse and search all answers, or ask your own.

Searches the multiple comparisons answers on this page. Search all answers

What counts as a family when you correct for multiple comparisons?

How to decide which tests share one familywise error correction, with a simulation of per-outcome versus pooled Holm corrections.

Read the answer →

Which interaction terms belong in a regression or mixed model?

Which terms must accompany a three-way interaction, whether the others are required, and how choosing interactions from the data inflates false positives.

Read the answer →

Dunn's test after Kruskal-Wallis: which p-value adjustment?

Whether pairwise Dunn tests need adjusting after Kruskal-Wallis rejects, and how Holm, Šidák and Benjamini-Hochberg differ.

Read the answer →

Is an F-test gate needed before Tukey's HSD?

Whether the overall F test must reject before follow-up tests between groups, what that gate does to false positives and power, and when F still helps.

Read the answer →

How many participants do you need when comparing several treatment arms?

Plan a multi-arm study around the comparisons you will report, not the ANOVA F test, and see how a multiplicity correction changes the numbers.

Read the answer →

Significant ANOVA but no significant pairwise comparisons: how can that happen?

Why a one-way ANOVA F test can reject while every pairwise or Tukey comparison does not (and the reverse), and how far apart their p values can get.

Read the answer →

Planned contrasts vs post hoc tests: why do they give different p-values?

Why one comparison gets different p-values as a planned contrast, a Tukey test or a Bonferroni test, and when to use each.

Read the answer →

Bonferroni correction: when should you use it, and what are the alternatives?

What a Bonferroni correction protects against, why it costs so much power, and how Holm and Benjamini-Hochberg compare.

Read the answer →