Answer

Can a p-value be exactly zero? Reporting p < .001

Inspired by a question on Cross Validated ·

p-valueshypothesis testing

The short answer

No. A p-value is a probability computed under the null hypothesis, and for a sensible test it is always greater than zero, however small. When software shows 0 or "< 2.2e-16", it is hitting a limit of computer arithmetic or of a resampling procedure, not reporting an impossible result. A permutation or bootstrap p-value should be computed as (r + 1)/(B + 1), which can never be zero. In a write-up, report such values as p < .001.

Why the answer is no

The p-value is the probability, assuming the null hypothesis is true, of a test statistic at least as extreme as the one observed. For a test based on a continuous distribution (t, z, F, chi-square), any finite statistic leaves some probability in the tail, so the p-value is positive. For an exact test on discrete data it is at least the probability of the observed table or arrangement itself, which is also positive. A literal zero would mean the data were impossible under the null, and the null model behind standard tests never rules data out completely.

So a printed zero always comes from how the number was computed or displayed. There are three usual culprits.

Where the zeros come from

For the resampling case the standard fix is to count the observed data as one of the possible arrangements and use (r + 1)/(B + 1). This is never zero and keeps the test valid; see Phipson and Smyth, Permutation p-values should never be zero.

See it in R and Python

The code compares two simulated groups of 1,000 and 700 values whose means differ by half a standard deviation. It runs a Welch t test, a permutation test with 2,000 shuffles, and then shows what happens to a z statistic of 40.

R

set.seed(653127)

# Two large groups that differ by half a standard deviation
a <- rnorm(1000, mean = 0,   sd = 1)
b <- rnorm(700,  mean = 0.5, sd = 1)

# 1. A Welch t test: R prints "< 2.2e-16", but the stored value is not zero
tt <- t.test(b, a)
tt$p.value
format.pval(tt$p.value)

# 2. A permutation test with 2,000 shuffles of the group labels
obs <- mean(b) - mean(a)
pooled <- c(a, b)
B <- 2000
perm <- replicate(B, {
  i <- sample(length(pooled), length(b))
  mean(pooled[i]) - mean(pooled[-i])
})
r <- sum(abs(perm) >= abs(obs))     # shuffles at least as extreme as the data
c(exceed = r, naive_p = r / B, corrected_p = (r + 1) / (B + 1))

# 3. Floating-point limits: a z of 40 gives a p-value too small to store
c(p_two_sided = 2 * pnorm(-40),
  log10_p = (log(2) + pnorm(-40, log.p = TRUE)) / log(10))

Python

import math
import numpy as np
from scipy import stats

rng = np.random.default_rng(653127)

# Two large groups that differ by half a standard deviation
a = rng.normal(0, 1, 1000)
b = rng.normal(0.5, 1, 700)

# 1. A Welch t test: the p-value is tiny but not zero
tt = stats.ttest_ind(b, a, equal_var=False)
print("t-test p:", float(tt.pvalue))

# 2. A permutation test with 2,000 shuffles of the group labels
obs = b.mean() - a.mean()
pooled = np.concatenate([a, b])
B = 2000
perm = np.empty(B)
for k in range(B):
    idx = rng.permutation(len(pooled))
    perm[k] = pooled[idx[:len(b)]].mean() - pooled[idx[len(b):]].mean()
r = int(np.sum(np.abs(perm) >= abs(obs)))  # shuffles at least as extreme as the data
print({"exceed": r, "naive_p": r / B, "corrected_p": round((r + 1) / (B + 1), 6)})

# 3. Floating-point limits: a z of 40 gives a p-value too small to store
print({"p_two_sided": float(2 * stats.norm.sf(40)),
       "log10_p": round(float((math.log(2) + stats.norm.logsf(40)) / math.log(10)), 2)})

Output from one seeded run of the R code:

[1] 3.606661e-19
[1] "< 2.22e-16"
      exceed      naive_p  corrected_p 
0.0000000000 0.0000000000 0.0004997501 
p_two_sided     log10_p 
      0.000    -349.136

What the numbers show

What to do in practice

How to report a very small p-value in APA style (7th edition)

APA style gives exact p-values to two or three decimals, except that values below .001 are reported as p < .001. There is no leading zero, because a p-value cannot exceed 1. Using the R example:

"The second group scored higher than the first, Welch's t-test, p < .001." In a full report, add the t value with its degrees of freedom, the mean difference and its 95% CI, and an effect size such as Cohen's d.

For the permutation test: "A permutation test with 2,000 random relabellings gave p < .001 (no relabelling produced a difference as large as the observed one)."

Related tools and guides

More answered questions

Working with your own data?

General answers only go so far. Send us your situation and we'll reply by email within two business days.

Ask your question

Written by AskStats with AI assistance. This is general information, not advice for your specific data or study. When the results matter, check your approach with a qualified statistician.