Answer

Why can a CI for Cohen's d include zero when the t test is significant?

Inspired by a question on Cross Validated · October 14, 2026

effect sizet testconfidence intervals

The short answer

A paired t test and a standardised effect size can seem to disagree: the 95% confidence interval for the mean difference sits clearly above zero, yet the 95% interval reported for Cohen's d stretches below it. The data have not changed, so the two intervals are measuring the uncertainty in different ways, and the interval for d is usually the one at fault.

Most effect size calculators that ask for two means, two standard deviations and two sample sizes assume independent groups. In a within-subjects design the same people appear in both conditions, and the paired test gains its power by looking only at each person's difference score. A formula for separate groups ignores that correlation, so it treats a precise paired comparison as a noisy between-groups one. The fix is to choose a version of d and an interval that belong to the paired design.

Two versions of d for repeated measures

With repeated measures there is more than one sensible standard deviation to divide by, and each gives a different d (Lakens, 2013, gives a clear overview):

• ***dav: divide by the average SD of the two conditions.* dav = mean difference / √((sA² + sB²) / 2). This is on the same scale as d from a between-subjects study, so it is the better choice for comparing with other research or for meta-analysis. Its sampling variability depends on the correlation between the conditions, so an interval for it must take that correlation into account.
• ***dz: divide by the SD of the difference scores.* dz = mean difference / sdiff. It is tied directly to the test, because the paired t statistic equals dz × √n. It is the natural input for a power analysis of a paired design.
• How they relate. When both conditions have the same SD and the correlation between them is r, sdiff = s × √(2(1 − r)), so dz = dav / √(2(1 − r)). When r is above .5, dz is larger than dav; when r is below .5 it is smaller. Neither is wrong, but they answer different questions, so always say which one you report.

Because dz is just a rescaled t statistic, its exact confidence interval comes from the noncentral t distribution: find the noncentrality values under which the observed t would sit at the 97.5th and 2.5th percentiles, and divide them by √n. At a noncentrality of zero this distribution is the ordinary central t, so the interval's lower limit is above zero exactly when the two-sided p is below .05. The effect size interval and the test can then never disagree.

See it in R and Python

The code simulates 28 people measured under two conditions, with a strong correlation between conditions because some people simply score higher than others. It runs the paired t test and then computes three effect size intervals: dav with the independent-groups formula (the mistake), dz with the exact noncentral t interval, and dav with a percentile bootstrap that resamples whole pairs.

set.seed(87068)

# Paired data: 28 people measured under conditions A and B
n <- 28
person <- rnorm(n, 0, 8)                   # stable differences between people
a <- 54 + person + rnorm(n, 0, 6)
b <- 50 + person + rnorm(n, 0, 6)
diffs <- a - b
round(c(M_A = mean(a), SD_A = sd(a), M_B = mean(b), SD_B = sd(b),
        r_AB = cor(a, b), M_diff = mean(diffs), SD_diff = sd(diffs)), 2)

# 1. Paired t test: CI for the mean difference
tt <- t.test(a, b, paired = TRUE)
round(c(t = unname(tt$statistic), df = unname(tt$parameter), p = tt$p.value,
        lower = tt$conf.int[1], upper = tt$conf.int[2]), 4)

# 2. d_av (mean difference / average SD) with an independent-groups CI
d_av <- mean(diffs) / sqrt((var(a) + var(b)) / 2)
se_ind <- sqrt(2 / n + d_av^2 / (4 * n))   # treats A and B as separate groups of n
round(c(d_av = d_av, lower = d_av - 1.96 * se_ind, upper = d_av + 1.96 * se_ind), 4)

# 3. d_z (mean difference / SD of differences) with an exact noncentral t CI
d_z <- mean(diffs) / sd(diffs)
t_obs <- d_z * sqrt(n)                     # identical to the paired t statistic
ncp_at <- function(prob) uniroot(function(ncp) pt(t_obs, n - 1, ncp) - prob,
                                 t_obs + c(-6, 6), tol = 1e-10)$root
round(c(d_z = d_z, lower = ncp_at(.975) / sqrt(n), upper = ncp_at(.025) / sqrt(n)), 4)

# 4. d_av with a percentile bootstrap CI that resamples whole pairs
B <- 5000
idx <- matrix(sample.int(n, B * n, replace = TRUE), nrow = B)
A <- matrix(a[idx], nrow = B); Bm <- matrix(b[idx], nrow = B)
sd_rows <- function(m) sqrt(rowSums((m - rowMeans(m))^2) / (n - 1))
d_boot <- (rowMeans(A) - rowMeans(Bm)) / sqrt((sd_rows(A)^2 + sd_rows(Bm)^2) / 2)
round(quantile(d_boot, c(.025, .975)), 4)

import numpy as np
from scipy import stats, optimize

rng = np.random.default_rng(87068)

# Paired data: 28 people measured under conditions A and B
n = 28
person = rng.normal(0, 8, n)                 # stable differences between people
a = 54 + person + rng.normal(0, 6, n)
b = 50 + person + rng.normal(0, 6, n)
diffs = a - b
print("M_A %.2f SD_A %.2f M_B %.2f SD_B %.2f r_AB %.2f M_diff %.2f SD_diff %.2f" % (
    a.mean(), a.std(ddof=1), b.mean(), b.std(ddof=1),
    np.corrcoef(a, b)[0, 1], diffs.mean(), diffs.std(ddof=1)))

# 1. Paired t test: CI for the mean difference
tt = stats.ttest_rel(a, b)
ci = tt.confidence_interval(0.95)
print("t %.3f df %d p %.3f lower %.3f upper %.3f" % (tt.statistic, tt.df, tt.pvalue, ci.low, ci.high))

# 2. d_av (mean difference / average SD) with an independent-groups CI
d_av = diffs.mean() / np.sqrt((a.var(ddof=1) + b.var(ddof=1)) / 2)
se_ind = np.sqrt(2 / n + d_av**2 / (4 * n))  # treats A and B as separate groups of n
print("d_av %.3f lower %.3f upper %.3f" % (d_av, d_av - 1.96 * se_ind, d_av + 1.96 * se_ind))

# 3. d_z (mean difference / SD of differences) with an exact noncentral t CI
def dz_ci(d_z, n):
    t_obs = d_z * np.sqrt(n)                 # identical to the paired t statistic
    ncp_at = lambda prob: optimize.brentq(
        lambda ncp: stats.nct.cdf(t_obs, n - 1, ncp) - prob, t_obs - 6, t_obs + 6, xtol=1e-10)
    return ncp_at(.975) / np.sqrt(n), ncp_at(.025) / np.sqrt(n)

d_z = diffs.mean() / diffs.std(ddof=1)
print("d_z %.3f lower %.3f upper %.3f" % ((d_z,) + dz_ci(d_z, n)))
# The same function applied to d_z from the R run (no randomness involved)
print("R's d_z: lower %.3f upper %.3f" % dz_ci(5.23 / 8.15, n))

# 4. d_av with a percentile bootstrap CI that resamples whole pairs
B = 5000
idx = rng.integers(0, n, size=(B, n))
A, Bm = a[idx], b[idx]
d_boot = (A.mean(axis=1) - Bm.mean(axis=1)) / np.sqrt((A.var(axis=1, ddof=1) + Bm.var(axis=1, ddof=1)) / 2)
print("bootstrap 2.5%% %.3f 97.5%% %.3f" % tuple(np.quantile(d_boot, [.025, .975])))

The figures below come from one seeded run of the R code. Python's random numbers differ from R's, so its sample is different (its mean difference is 6.06 rather than 5.23) and its figures differ, but they show the same pattern: the independent-groups interval is far wider than the two intervals that respect the pairing. Given R's dz, the Python function returns the same exact interval, [0.229, 1.044], because that step involves no randomness.

• The data. Condition A: M = 56.22, SD = 10.94. Condition B: M = 50.99, SD = 11.29. The two conditions correlate at r = .73, and the difference scores have M = 5.23, SD = 8.15.
• The paired t test. t(27) = 3.39, p = .002, and the 95% CI for the mean difference is [2.07, 8.39], well clear of zero.
• ***dav with the independent-groups interval.* dav = 0.47 with a 95% interval of [−0.06, 1.00]. It includes zero, and it is about 1.06 units wide.
• ***dz with the exact interval.* dz = 0.64, 95% CI [0.23, 1.04]. It excludes zero, matching the test. dz is larger than dav because the correlation of .73 is above .5.
• ***dav with the paired bootstrap.* The same dav of 0.47 gets a percentile bootstrap interval of [0.20, 0.80], about 0.59 units wide: roughly half the width of the independent-groups interval, and above zero.

So the conflict was never about the data. The independent-groups formula charged the effect size for between-person variability that the paired design had already removed.

Which one should you report?

• **Report dz** when you want the effect size to match the paired test, or when it will feed a power analysis for a similar within-subjects study. Use the noncentral t interval, which needs only t, n and a root finder.
• **Report dav** when readers will compare it with between-subjects studies or include it in a meta-analysis. Give it an interval that accounts for the pairing: a bootstrap that resamples whole pairs, as above, or an analytic approximation for dependent designs such as that of Algina and Keselman (2003).
• Match the interval to the d. Do not compute dav and attach an interval designed for independent groups, and do not label dz as plain "Cohen's d" without saying which SD it uses. Readers often compare d values across studies, and dz from a strongly correlated design can look much larger than a between-subjects d for the same effect.
• Do not judge the effect by whether its interval crosses zero. The test answers whether there is a difference; the effect size interval tells you how big it plausibly is. In the example, both intervals that respect the pairing start at about 0.2 and end between 0.8 and 1.0 SD, a wide range that a sample of 28 cannot narrow further.

For help choosing between paired and independent analyses, see Paired vs independent t test; for what the size labels mean, see How to interpret Cohen's d. That page explains what a value of d says about two groups; this one is about getting the interval right in a paired design. For what a 95% interval does and does not promise in general, see What does 95% confidence mean?; this page deals only with why two intervals from the same data can seem to disagree. More plain-language guides are on the DASS blog.

How to report Cohen's d and its confidence interval in APA style (7th edition)

Report the paired test, the mean difference with its interval, and the effect size with its interval, and name the version of d. Keep the leading zero for d, which can exceed 1. Using the example above:

"Scores were higher in condition A (M = 56.22, SD = 10.94) than in condition B (M = 50.99, SD = 11.29), a mean difference of 5.23, 95% CI [2.07, 8.39], t(27) = 3.39, p = .002, dz = 0.64, 95% CI [0.23, 1.04]."

If you report dav instead, write "dav = 0.47, 95% CI [0.20, 0.80]" and say in the method section how the interval was obtained, for example "Confidence intervals for dav were obtained with a percentile bootstrap (5,000 resamples of participants)." For dz, a suitable line is "Effect sizes are dz (mean difference divided by the standard deviation of the differences), with exact confidence intervals from the noncentral t distribution."

Related tools and guides

• t-test power and sample size calculator
• APA 7 formatter for t tests
• How do you interpret Cohen's d?
• Paired vs independent t test: which should you use?
• What does 95% confidence mean?
• Confidence interval for eta squared
• Effect size (Wikipedia)

More answered questions

• Confidence interval for eta squared: does the Fisher z shortcut work?
• Mann-Whitney effect size: is z/√N right, and which CI should you report?
• What should you do after rejecting the null hypothesis?

Written by AskStats with AI assistance. This is general information, not advice for your specific data or study. When the results matter, check your approach with a qualified statistician.