Tag
multiple comparisons
8 answered questions tagged “multiple comparisons”. Browse and search all answers, or ask your own.
Searches the multiple comparisons answers on this page. Search all answers
What counts as a family when you correct for multiple comparisons?
How to decide which tests share one familywise error correction, with a simulation of per-outcome versus pooled Holm corrections.
Read the answer →Which interaction terms belong in a regression or mixed model?
Which terms must accompany a three-way interaction, whether the others are required, and how choosing interactions from the data inflates false positives.
Read the answer →Dunn's test after Kruskal-Wallis: which p-value adjustment?
Whether pairwise Dunn tests need adjusting after Kruskal-Wallis rejects, and how Holm, Šidák and Benjamini-Hochberg differ.
Read the answer →Is an F-test gate needed before Tukey's HSD?
Whether the overall F test must reject before follow-up tests between groups, what that gate does to false positives and power, and when F still helps.
Read the answer →How many participants do you need when comparing several treatment arms?
Plan a multi-arm study around the comparisons you will report, not the ANOVA F test, and see how a multiplicity correction changes the numbers.
Read the answer →Significant ANOVA but no significant pairwise comparisons: how can that happen?
Why a one-way ANOVA F test can reject while every pairwise or Tukey comparison does not (and the reverse), and how far apart their p values can get.
Read the answer →Planned contrasts vs post hoc tests: why do they give different p-values?
Why one comparison gets different p-values as a planned contrast, a Tukey test or a Bonferroni test, and when to use each.
Read the answer →Bonferroni correction: when should you use it, and what are the alternatives?
What a Bonferroni correction protects against, why it costs so much power, and how Holm and Benjamini-Hochberg compare.
Read the answer →