Answer

How do you compare medians when the groups differ in shape and spread?

Inspired by a question on Cross Validated · October 17, 2026

nonparametrichypothesis testingbootstrap

The short answer

A common workflow is to run a Kolmogorov-Smirnov test to see whether two samples have the same shape, use Mann-Whitney when they do, and fall back on Welch's t test when they do not. Each step answers a slightly different question from the one people usually have in mind, which is: is the typical value higher in one group?

• **Welch's t test** compares means. With skewed data the mean and median can tell different stories.
• Mann-Whitney (the Wilcoxon rank-sum test) compares the chance that a random score from one group exceeds a random score from the other with 50%. Only when the two distributions have the same shape and differ by a pure shift is that the same as comparing medians.
• Kolmogorov-Smirnov picks up any difference at all between the distributions. Using it as a gate in front of another test is a two-stage procedure whose overall error rate is unknown, and with small groups it will miss most shape differences anyway.

If medians really are the quantity you care about, use a method built for medians. This page complements t test or Mann-Whitney in small samples, which is about choosing between those two tests; here the focus is what to do when neither answers a question about medians.

Methods that actually compare medians

• A bootstrap confidence interval for the difference in medians. Resample each group with replacement, record the difference in medians each time, and take the middle 95% of those differences. It gives an estimate in the outcome's own units, which is usually more useful than a p value. See Why does bootstrapping work? for the idea.
• Mood's median test. Find the median of both groups pooled, count how many scores in each group fall above it, and test the resulting 2 x 2 table with Fisher's exact test (or a chi-square test in large samples). It is simple and makes no assumption about shape, but it throws away a lot of information, so its power is low.
• Median regression. Quantile regression at the 50th percentile, for example rq() in the R package quantreg, models the conditional median and can add covariates. With a single group indicator it estimates the same difference in medians.

None of these is perfect. Mood's test is not guaranteed to hold exactly 5% when the groups have very different spreads near the median, and bootstrap intervals for a median behave poorly in tiny samples because a sample median can take only a few distinct values.

See it in R and Python

The code builds two populations with exactly the same median, 0: one normal and one right-skewed (a lognormal shifted down by 1). It then simulates 2,000 studies with 100 per group and records how often each test gives p < .05. Because the medians are equal, a test of medians should reject about 5% of the time. Finally it analyses one study with 40 and 25 observations and a true median difference of 0.5.

set.seed(176785)

# Two populations with the SAME median (0) but different shapes and spreads
gen_x <- function(n) rnorm(n)              # symmetric
gen_y <- function(n) rlnorm(n) - 1         # right-skewed
round(c(median_y = qlnorm(0.5) - 1, mean_y = exp(0.5) - 1), 4)
mean(gen_y(1e6) > gen_x(1e6))              # P(Y > X): not 0.5

# Mood's median test: counts above the pooled median, Fisher's exact test
mood_p <- function(x, y) {
  m <- median(c(x, y))
  tab <- rbind(c(sum(x <= m), sum(x > m)), c(sum(y <= m), sum(y > m)))
  fisher.test(tab)$p.value
}

# 2,000 studies with 100 per group; medians are equal in every one
res <- replicate(2000, {
  x <- gen_x(100); y <- gen_y(100)
  c(welch_t      = t.test(x, y)$p.value < .05,
    mann_whitney = wilcox.test(x, y)$p.value < .05,
    mood_median  = mood_p(x, y) < .05)
})
round(rowMeans(res), 3)                    # share of studies with p < .05

# One study, unequal sizes, true median difference of 0.5
x <- gen_x(40); y <- gen_y(25) + 0.5
round(c(median_x = median(x), median_y = median(y),
        diff = median(y) - median(x)), 2)
boot <- replicate(5000, median(sample(y, replace = TRUE)) -
                        median(sample(x, replace = TRUE)))
round(quantile(boot, c(0.025, 0.975)), 2)  # percentile bootstrap 95% CI
round(c(p_mood = mood_p(x, y)), 4)

import numpy as np
from scipy import stats

rng = np.random.default_rng(176785)

# Two populations with the SAME median (0) but different shapes and spreads
gen_x = lambda n: rng.normal(0, 1, n)              # symmetric
gen_y = lambda n: rng.lognormal(0, 1, n) - 1       # right-skewed
print("median_y", stats.lognorm.median(1) - 1, "mean_y", round(np.exp(0.5) - 1, 4))
print("P(Y > X)", np.mean(gen_y(10**6) > gen_x(10**6)))   # not 0.5

# Mood's median test: counts above the pooled median, Fisher's exact test
def mood_p(x, y):
    m = np.median(np.concatenate([x, y]))
    tab = [[np.sum(x <= m), np.sum(x > m)], [np.sum(y <= m), np.sum(y > m)]]
    return stats.fisher_exact(tab).pvalue

# 2,000 studies with 100 per group; medians are equal in every one
res = []
for _ in range(2000):
    x, y = gen_x(100), gen_y(100)
    res.append([stats.ttest_ind(x, y, equal_var=False).pvalue < .05,
                stats.mannwhitneyu(x, y).pvalue < .05,
                mood_p(x, y) < .05])
print("welch, mann-whitney, mood:", np.round(np.mean(res, axis=0), 3))

# One study, unequal sizes, true median difference of 0.5
x, y = gen_x(40), gen_y(25) + 0.5
print("medians", round(np.median(x), 2), round(np.median(y), 2),
      "diff", round(np.median(y) - np.median(x), 2))
bx = np.median(rng.choice(x, (5000, len(x))), axis=1)
by = np.median(rng.choice(y, (5000, len(y))), axis=1)
print("95% CI", np.round(np.quantile(by - bx, [0.025, 0.975]), 2))  # percentile bootstrap
print("p_mood", round(mood_p(x, y), 4))

The figures below come from one seeded run of the R code. Python's random numbers differ from R's, so its figures differ slightly but show the same pattern in the simulation.

• The setup. Both medians are 0, but the skewed group's mean is 0.6487, and a random skewed score beats a random normal score 56.3% of the time rather than 50%.
• **Welch's t test** rejected in 87.1% of studies. That is correct behaviour for a test of means, since the means really differ, but it says nothing about medians.
• Mann-Whitney rejected in 36.7% of studies, more than seven times the nominal 5%. It is detecting the 56% versus 50% imbalance, not a difference in medians.
• Mood's median test rejected in 4.3% of studies, close to the 5% a test of equal medians should give. Python's run showed the same ordering: .879, .359 and .036.

In the single study, the sample medians were 0.21 (n = 40) and 1.29 (n = 25), a difference of 1.07, with a percentile bootstrap 95% CI of [0.11, 2.26] from 5,000 resamples. Mood's median test gave p = .005. The true difference was 0.5, which lies inside the interval; the wide interval shows how imprecise a median is with 25 skewed observations. Python's sample from the same design happened to show a smaller difference (0.23, 95% CI [-0.17, 1.03], Mood p = .208), another reminder of how much one sample can vary.

How small can the groups be?

There is no hard minimum, but very small groups make any comparison of medians weak. With 4 observations in one group and 10 in the other, the most extreme result Mood's test can produce, all 4 on the same side of the pooled median, has a two-sided Fisher p value of 70/1001, about .070, so it can never reach .05. A bootstrap interval with 4 observations is equally unreliable, because the group's median can take only a handful of values.

With groups that small, report the raw values or a dot plot, give the medians, and treat any test as descriptive. If a firm conclusion matters, plan a larger sample; a power analysis before collecting data is the honest fix. To match a design to a test, try the test chooser, and for more plain-language guides, see the DASS blog.

How to report a difference in medians in APA style (7th edition)

Give each group's median and size, the difference in medians with its confidence interval and how it was obtained, then the test. With the single study above:

"The median was higher in group Y (Mdn = 1.29, n = 25) than in group X (Mdn = 0.21, n = 40), a difference of 1.07, 95% CI [0.11, 2.26] (percentile bootstrap, 5,000 resamples). Mood's median test, using Fisher's exact test on counts above the pooled median, was significant, p = .005."

In the method section, say that medians were compared because the outcome was skewed and the groups differed in spread, name the bootstrap method and number of resamples, and give the random seed. If you also report Mann-Whitney, describe it as a test of whether one group tends to score higher, not as a test of medians.

Related tools and guides

• APA 7 formatter for nonparametric tests
• Check a reported p-value
• t test or Mann-Whitney in small samples: how do you choose?
• Effect size and confidence interval for a Mann-Whitney test
• Why does bootstrapping work?
• Welch vs Student t test: should you just always use Welch?
• Is a normality test worth running on your data?

More answered questions

• Kolmogorov-Smirnov, t test or Mann-Whitney: which detects a change from a baseline?
• What are the assumptions of the Wilcoxon signed-rank test?
• Significant ANOVA but no significant pairwise comparisons: how can that happen?

Written by AskStats with AI assistance. This is general information, not advice for your specific data or study. When the results matter, check your approach with a qualified statistician.