Answer

Chi-square test with expected counts below 5: what should you do?

Inspired by a question on Cross Validated · October 19, 2026

chi-squarehypothesis testing

The short answer

The Pearson chi-square test compares observed counts in a contingency table with the counts expected if the two variables were independent. Its p value comes from the chi-square distribution, which is a large-sample approximation. When the expected counts are small, the real sampling distribution of the statistic is lumpy and can drift from that smooth curve, so software warns you.

The warning does not mean the data are unusable. It means the p value should come from a method that does not rely on the approximation. For a 2x2 table that is Fisher's exact test. For bigger tables it is the Freeman-Halton extension of Fisher's test (R's fisher.test() handles any r x c table) or a Monte Carlo p value, which simulates thousands of tables with the same row and column totals. Both give valid p values however sparse the table is.

What the rule of 5 is really about

• It concerns expected counts. An observed zero is fine. What matters is each cell's row total times column total, divided by N.
• It is a guideline, not a law. Cochran's widely quoted version allows up to 20% of cells with expected counts below 5, as long as none is below 1. Many later studies found it cautious: the uncorrected Pearson test often keeps close to its stated error rate with smaller expected counts than that.
• Sparse cells can still hurt. With very small expected counts, a single cell can dominate the statistic, and the approximate p value can be noticeably smaller or larger than the exact one. In the example below it was less than half of it.
• Yates's continuity correction is not the fix. It makes the 2x2 test much too conservative, as the simulation below shows. Note that Python's chi2_contingency() applies it to 2x2 tables by default, while R's chisq.test() also does unless you set correct = FALSE.

Options for 2x2 and larger tables

1. Report an exact or Monte Carlo p value. This is the default fix. Keep the table and the chi-square statistic as a description of the departure from independence, and take the p value from Fisher's test or from simulation. Most packages offer one of these; in SPSS, for example, the Crosstabs dialog has an Exact option when the Exact Tests module is installed.
2. Merge categories only for substantive reasons. Combining rare categories (say, two small age bands) raises the expected counts, but it changes the question. Do it only when the merged category is meaningful in its own right, and decide before looking at the test result.
3. Use the order when categories are ordered. If both variables are ordinal, a test for a linear trend uses one degree of freedom instead of many and is usually more powerful than the omnibus test. With one ordinal outcome and two groups, a rank test can also work; see comparing an ordinal scale between two groups.
4. Do not drop rows or columns to silence the warning. Removing sparse categories quietly changes the population you are describing and can hide the very pattern you are testing.

This page is about whether the p value can be trusted. The related question of which tail of the distribution the test uses, and why its p value is never halved, is covered in Is the chi-square test one-tailed or two-tailed?.

See it in R and Python

The code analyses a 2x2 table of reported side effects under two treatments (15 people each) and a sparse 3x4 table of preferred study format by year of study (N = 55). It then computes, exactly rather than by simulation, how often each 2x2 method gives p < .05 when the null hypothesis is true, with 15 per group and a true rate of 0.1, 0.2 or 0.5.

set.seed(44622)

# 1. A small 2x2 table: side effect reported under two treatments (15 people each)
tab <- matrix(c(7, 8,
                1, 14), nrow = 2, byrow = TRUE,
              dimnames = list(group = c("A", "B"), effect = c("yes", "no")))
suppressWarnings(chisq.test(tab)$expected)
p1 <- c(pearson = suppressWarnings(chisq.test(tab, correct = FALSE)$p.value),
        yates   = suppressWarnings(chisq.test(tab)$p.value),
        fisher  = fisher.test(tab)$p.value,
        monte_carlo = chisq.test(tab, simulate.p.value = TRUE, B = 10000)$p.value)
round(p1, 4)
ft <- fisher.test(tab)   # conditional odds ratio with its exact 95% CI
round(c(odds_ratio = unname(ft$estimate), lower = ft$conf.int[1], upper = ft$conf.int[2]), 2)

# 2. A sparse 3x4 table: preferred study format (columns) by year (rows)
big <- matrix(c(9, 6, 3, 1,
                4, 7, 5, 2,
                1, 3, 6, 8), nrow = 3, byrow = TRUE)
e <- suppressWarnings(chisq.test(big)$expected)
round(c(n = sum(big), min_expected = min(e), share_below_5 = mean(e < 5)), 3)
ct <- suppressWarnings(chisq.test(big))
p2 <- c(chisq = unname(ct$statistic), pearson = ct$p.value,
        fisher_exact = fisher.test(big)$p.value,
        monte_carlo = chisq.test(big, simulate.p.value = TRUE, B = 10000)$p.value)
round(p2, 4)

# 3. Exact false-positive rates (%) for a 2x2 table with 15 per group, null true
rates <- function(prob, n = 15) {
  out <- c(pearson = 0, yates = 0, fisher = 0)
  for (a in 0:n) for (b in 0:n) {
    if (a + b == 0 || a + b == 2 * n) next   # one column empty: no test possible
    t2 <- matrix(c(a, n - a, b, n - b), 2, byrow = TRUE)
    p <- c(suppressWarnings(chisq.test(t2, correct = FALSE)$p.value),
           suppressWarnings(chisq.test(t2)$p.value),
           fisher.test(t2)$p.value)
    out <- out + dbinom(a, n, prob) * dbinom(b, n, prob) * (p < 0.05)
  }
  out
}
round(100 * rbind(p_0.1 = rates(0.1), p_0.2 = rates(0.2), p_0.5 = rates(0.5)), 2)

import numpy as np
from scipy import stats

rng = np.random.default_rng(44622)

def mc_pvalue(tab, B, rng):
    # Monte Carlo p value with both margins fixed (shuffle column labels)
    r, c = tab.shape
    rows = np.repeat(np.repeat(np.arange(r), c), tab.ravel())
    cols = np.repeat(np.tile(np.arange(c), r), tab.ravel())
    expected = np.outer(tab.sum(1), tab.sum(0)) / tab.sum()
    stat = lambda t: ((t - expected) ** 2 / expected).sum()
    obs = stat(tab)
    sims = [stat(np.bincount(rows * c + rng.permutation(cols), minlength=r * c).reshape(r, c))
            for _ in range(B)]
    return (1 + np.sum(np.array(sims) >= obs * (1 - 64 * np.finfo(float).eps))) / (B + 1)

# 1. A small 2x2 table: side effect reported under two treatments (15 people each)
tab = np.array([[7, 8],
                [1, 14]])
print(stats.contingency.expected_freq(tab))
print({"pearson": round(stats.chi2_contingency(tab, correction=False)[1], 4),
       "yates": round(stats.chi2_contingency(tab)[1], 4),
       "fisher": round(stats.fisher_exact(tab)[1], 4),
       "monte_carlo": round(mc_pvalue(tab, 10000, rng), 4)})
orr = stats.contingency.odds_ratio(tab)   # conditional odds ratio with its exact 95% CI
ci = orr.confidence_interval()
print({"odds_ratio": round(orr.statistic, 2), "lower": round(ci.low, 2), "upper": round(ci.high, 2)})

# 2. A sparse 3x4 table: preferred study format (columns) by year (rows)
big = np.array([[9, 6, 3, 1],
                [4, 7, 5, 2],
                [1, 3, 6, 8]])
e = stats.contingency.expected_freq(big)
print({"n": big.sum(), "min_expected": round(e.min(), 3), "share_below_5": round(np.mean(e < 5), 3)})
chi2, p, dof, _ = stats.chi2_contingency(big)
print({"chisq": round(chi2, 4), "pearson": round(p, 4),
       "monte_carlo": round(mc_pvalue(big, 10000, rng), 4)})   # scipy has no exact r x c Fisher test

# 3. Exact false-positive rates (%) for a 2x2 table with 15 per group, null true
def rates(prob, n=15):
    out = np.zeros(3)
    for a in range(n + 1):
        for b in range(n + 1):
            if a + b == 0 or a + b == 2 * n:   # one column empty: no test possible
                continue
            t2 = np.array([[a, n - a], [b, n - b]])
            p = [stats.chi2_contingency(t2, correction=False)[1],
                 stats.chi2_contingency(t2)[1],
                 stats.fisher_exact(t2)[1]]
            out += stats.binom.pmf(a, n, prob) * stats.binom.pmf(b, n, prob) * (np.array(p) < 0.05)
    return out

for prob in (0.1, 0.2, 0.5):
    print("p =", prob, dict(zip(["pearson", "yates", "fisher"], np.round(100 * rates(prob), 2))))

The figures below come from one seeded run of the R code. Everything except the two Monte Carlo p values is exact, and Python reproduces it (its upper odds-ratio limit is 590.42 rather than 588.26, a numerical difference in a very wide interval). Python's random numbers differ from R's, so its Monte Carlo p values differ slightly (.0345 and .0073) but tell the same story. scipy has no exact test for tables larger than 2x2, so Python gives only the Monte Carlo version there.

• The 2x2 table. 7 of 15 people (46.7%) reported the side effect under treatment A and 1 of 15 (6.7%) under B. Both expected counts in the yes column are 4, so two of the four cells (50%) fall below 5.
• Four p values for one table. The uncorrected Pearson test gave p = .0132, Yates's correction .0390, Fisher's exact test .0352 and the Monte Carlo test .0374. The two methods that avoid the approximation agree with each other, and the approximate p value is less than half of theirs.
• Effect size. Fisher's conditional odds ratio was 11.26, 95% CI [1.13, 588.26]: clearly above 1, but with 30 people it could be modest or enormous.
• The 3x4 table. With N = 55, 75% of the expected counts were below 5 (the smallest was 3.6), far beyond Cochran's guideline. The statistic was 17.37 on 6 degrees of freedom; the approximate p value was .0080, the exact Freeman-Halton p value .0112 and the Monte Carlo p value .0072.
• False-positive rates with the null true (15 per group). For true rates of 0.1, 0.2 and 0.5, the uncorrected Pearson test rejected 2.46%, 4.96% and 4.56% of the time; Fisher's test 0.55%, 1.67% and 1.65%; Yates's correction 0.11%, 0.95% and 1.65%. None exceeded 5%, but Fisher and Yates were very conservative, which costs power.

So the practical message is twofold. An exact or Monte Carlo p value is always valid and is the safe thing to report when cells are sparse. And a warning about small expected counts is not, on its own, a reason to distrust a chi-square result that is far from the .05 boundary.

How to report Fisher's exact test in APA style (7th edition)

Fisher's exact test has no test statistic to report, so give the counts or percentages, the p value and, for a 2x2 table, an effect size such as the odds ratio with its confidence interval. Name the method so readers know why there is no χ² value. Using the example above:

"Side effects were reported more often under Treatment A (7 of 15, 46.7%) than under Treatment B (1 of 15, 6.7%), Fisher's exact test, p = .035, odds ratio = 11.26, 95% CI [1.13, 588.26]."

For a larger table, keep the chi-square statistic and say where the p value came from: "Preferred study format differed by year of study, χ²(6, N = 55) = 17.37, p = .011 (Fisher-Freeman-Halton exact test, used because 75% of expected counts were below 5)." For a Monte Carlo p value, state the number of simulated tables (here 10,000). Put the full table of counts in a table, and mention the method in the analysis section. Not sure which test fits your design? Try the test chooser, or see the DASS blog.

Related tools and guides

• Chi-square power and sample size calculator
• APA 7 formatter for chi-square tests
• Check a reported p-value
• Which statistical test should I use?
• Is the chi-square test one-tailed or two-tailed?
• Comparing an ordinal scale between two groups
• Fisher's exact test (Wikipedia)

More answered questions

• Is the chi-square test one-tailed or two-tailed?
• How do you compare medians when the groups differ in shape and spread?
• Kolmogorov-Smirnov, t test or Mann-Whitney: which detects a change from a baseline?

Written by AskStats with AI assistance. This is general information, not advice for your specific data or study. When the results matter, check your approach with a qualified statistician.