AskStats

Answer

What are degrees of freedom in statistics, really?

Inspired by a question on Cross Validated ·

hypothesis testingt teststandard deviation

The short answer

Degrees of freedom (df) are the number of observations minus the number of constraints your calculation has already placed on them, usually one per parameter you estimated first. For a sample variance that is n - 1, because the deviations around the sample mean must add up to zero. The common textbook definitions (values free to vary, information left over, dimension of the space the residuals live in) are three views of the same count.

The short version

Every time you estimate something from the data and then reuse that estimate inside another calculation, you tie the data down a little. Degrees of freedom keep track of how much freedom is left. Start with the number of observations, subtract one for each quantity you had to estimate along the way, and what remains is the df of the next statistic.

The count matters because it controls how much you can trust a variance estimate. A variance built from 3 df is very noisy; one built from 300 df is nearly exact. Distributions such as t, chi-square and F are indexed by df for exactly this reason: they describe how uncertain the variance part of a test statistic is.

Why the sample variance divides by n - 1

Take four scores: 4, 7, 9 and 12. Their mean is 8, so the deviations are -4, -1, 1 and 4. Deviations around a sample mean always total zero, because that is how the mean is defined. So once you know any three deviations, the fourth is forced: it must be whatever makes the sum zero. Only three of the four can take any value they like.

That is the first meaning of df, values that can vary freely. It also explains the second meaning, information left over. The sample mean sits closer to the data than the true mean does, so squared deviations around it are too small on average. Dividing their sum by n - 1 instead of n exactly offsets this, which makes the sample variance an unbiased estimate of the population variance.

The geometric view, and why all three definitions agree

Write the n observations as one point (a vector) in n-dimensional space. Subtracting the mean splits that vector into two perpendicular pieces: a part along the direction (1, 1, ..., 1), which carries the mean, and a residual part. The residual vector is forced to have components that sum to zero, which confines it to a flat subspace with n - 1 dimensions. The df is the dimension of the subspace in which the leftover variation can move.

The same picture covers regression. With p coefficients (including the intercept), least squares makes the residuals perpendicular to each of the p predictor columns. That is p linear constraints, so the residuals live in a space of n - p dimensions and the residual df is n - p. Counting free values, counting estimated parameters and counting dimensions give the same answer because they describe the same set of linear constraints.

Some familiar cases:

In smoothing splines, ridge regression and similar methods, the fit is shrunk rather than constrained, so software reports an effective df (the trace of the matrix that maps the observed outcomes to the fitted values). It need not be a whole number, but it still measures how much flexibility the fit has used.

See it in R and Python

The code checks each idea: the zero-sum constraint, the bias of dividing by n in 100,000 samples of size 5 from a population with variance 4, the three constraints on regression residuals, how the t critical value grows as df shrinks, and the df of a one-sample t test.

R

set.seed(1692)

# 1. Deviations from the sample mean always sum to zero
x <- c(4, 7, 9, 12)
d <- x - mean(x)
d
sum(d)                    # 0, so the last deviation is fixed by the first three
-sum(d[1:3])              # equals d[4]

# 2. Dividing by n - 1 instead of n removes the bias in the variance
sims <- replicate(100000, {
  s <- rnorm(5, mean = 50, sd = 2)          # true variance = 4
  c(n_minus_1 = var(s), n = var(s) * 4 / 5)
})
round(rowMeans(sims), 3)

# 3. Regression: each estimated coefficient uses up one df
n  <- 20
x1 <- rnorm(n); x2 <- rnorm(n)
y  <- 1 + 0.5 * x1 - 0.3 * x2 + rnorm(n)
fit <- lm(y ~ x1 + x2)
df.residual(fit)                                  # 20 - 3 = 17
X <- model.matrix(fit)
round(crossprod(X, resid(fit)), 10)               # 3 constraints on the residuals

# 4. Fewer df means heavier tails and wider critical values
round(sapply(c(df2 = 2, df5 = 5, df10 = 10, df30 = 30, df1000 = 1000),
             function(k) qt(0.975, k)), 3)
round(qnorm(0.975), 3)

# 5. A one-sample t test on 12 observations has 11 df
scores <- round(rnorm(12, mean = 104, sd = 10), 1)
tt <- t.test(scores, mu = 100)
round(c(mean = mean(scores), sd = sd(scores), t = unname(tt$statistic),
        df = unname(tt$parameter), p = tt$p.value), 3)
round(tt$conf.int, 2)

Python

import numpy as np
from scipy import stats

rng = np.random.default_rng(1692)

# 1. Deviations from the sample mean always sum to zero
x = np.array([4, 7, 9, 12])
d = x - x.mean()
print(d, d.sum())               # 0, so the last deviation is fixed by the first three
print(-d[:3].sum())             # equals d[3]

# 2. Dividing by n - 1 instead of n removes the bias in the variance
s = rng.normal(50, 2, size=(100_000, 5))       # true variance = 4
print("n-1:", round(s.var(axis=1, ddof=1).mean(), 3), " n:", round(s.var(axis=1, ddof=0).mean(), 3))

# 3. Regression: each estimated coefficient uses up one df
n = 20
x1, x2 = rng.normal(size=n), rng.normal(size=n)
y = 1 + 0.5 * x1 - 0.3 * x2 + rng.normal(size=n)
X = np.column_stack([np.ones(n), x1, x2])
beta, *_ = np.linalg.lstsq(X, y, rcond=None)
resid = y - X @ beta
print("residual df:", n - X.shape[1])          # 20 - 3 = 17
print(np.round(X.T @ resid, 10))               # 3 constraints on the residuals

# 4. Fewer df means heavier tails and wider critical values
print({k: round(stats.t.ppf(0.975, k), 3) for k in [2, 5, 10, 30, 1000]})
print(round(stats.norm.ppf(0.975), 3))

# 5. A one-sample t test on 12 observations has 11 df
scores = np.round(rng.normal(104, 10, 12), 1)
tt = stats.ttest_1samp(scores, 100)
ci = tt.confidence_interval(0.95)
print("mean", round(scores.mean(), 3), "sd", round(scores.std(ddof=1), 3),
      "t", round(tt.statistic, 3), "df", tt.df, "p", round(tt.pvalue, 3))
print("95% CI", round(ci.low, 2), round(ci.high, 2))

The figures below come from one seeded run of the R code. Python uses a different random number generator, so its simulated figures differ slightly, but the pattern is the same; the parts with no randomness (the four scores, the residual df and the critical values) match exactly.

Common mistakes

How to report degrees of freedom in APA style (7th edition)

In APA style, df go in parentheses right after the test statistic's symbol, with no label. Using the one-sample test above:

"Scores (M = 96.59, SD = 10.66) did not differ significantly from the reference value of 100, t(11) = -1.11, p = .292, 95% CI [89.82, 103.37]."

Other tests follow the same pattern with placeholders: F(df1, df2) = x.xx for ANOVA and regression, for example F(2, 17) = x.xx, and χ²(df, N = xxx) = x.xx for a chi-square test. For Welch's test, give the fractional df as computed, for example t(x.xx) = x.xx. In a table, df can go in their own column or in a note.

Related tools and guides

More answered questions

Working with your own data?

General answers only go so far. Send us your situation and we'll reply by email within two business days.

Ask your question

Written by AskStats with AI assistance. This is general information, not advice for your specific data or study. When the results matter, check your approach with a qualified statistician.