AskStats

Answer

Factor analysis vs PCA: what is the difference?

Inspired by a question on Cross Validated ·

pcafactor analysis

The short answer

PCA is a data-reduction tool: it rewrites your variables as weighted sums that capture as much total variance as possible, error included. Factor analysis is a model: it assumes hidden factors cause the correlations and splits each variable's variance into a shared and a unique part. Use PCA to compress data, and factor analysis to measure a construct such as anxiety.

Two different directions of explanation

The cleanest way to see the difference is to ask which way the arrows point.

Because of this, the two methods treat each variable's variance differently. PCA tries to account for all of it, including random measurement error. Factor analysis divides it into the communality (the share explained by the common factors) and the uniqueness (specific variance plus error), and only tries to explain the communality. In practice this means PCA loadings are usually larger than factor loadings for the same data, most of all for weak or noisy items.

What each method assumes

See it in R and Python

This simulation builds four questionnaire items from a single hidden trait, with known true loadings of 0.8, 0.7, 0.6 and 0.4. It then compares the loadings from a one-factor maximum likelihood factor analysis (base R's factanal()) with the loadings of the first principal component, and checks how well each one-dimensional summary reproduces the observed correlations. The Python tab uses NumPy and scikit-learn's FactorAnalysis.

R

set.seed(1576)
n <- 500
# One latent factor drives four items with different true loadings
lambda <- c(0.8, 0.7, 0.6, 0.4)
f <- rnorm(n)
X <- sapply(lambda, function(l) l * f + rnorm(n, sd = sqrt(1 - l^2)))
colnames(X) <- c("item1", "item2", "item3", "item4")
R <- cor(X)

# PCA: loadings of the first component = eigenvector * sqrt(eigenvalue)
e <- eigen(R)
pc_load <- abs(e$vectors[, 1]) * sqrt(e$values[1])

# Factor analysis (maximum likelihood, base R)
fa <- factanal(X, factors = 1)
fa_load <- abs(as.vector(fa$loadings))

round(cbind(true = lambda, factor = fa_load, pca = pc_load), 3)
round(e$values[1] / ncol(X), 3)          # share of total variance on PC1
round(fa$uniquenesses, 3)                # variance unique to each item

# How well does each one-dimensional model rebuild the correlations?
off <- lower.tri(R)
fit_fa  <- tcrossprod(fa_load)
fit_pca <- tcrossprod(pc_load)
round(mean(R[off]), 3)                   # average observed correlation
round(mean(fit_fa[off]), 3)              # average implied by factor model
round(mean(fit_pca[off]), 3)             # average implied by PC1
round(max(abs(R[off] - fit_fa[off])), 3) # worst error, factor model
round(max(abs(R[off] - fit_pca[off])), 3)# worst error, PC1

Python

import numpy as np
from sklearn.decomposition import FactorAnalysis

rng = np.random.default_rng(1576)
n = 500
# One latent factor drives four items with different true loadings
lam = np.array([0.8, 0.7, 0.6, 0.4])
f = rng.normal(0, 1, n)
X = np.column_stack([l * f + rng.normal(0, np.sqrt(1 - l**2), n) for l in lam])
R = np.corrcoef(X, rowvar=False)

# PCA: loadings of the first component = eigenvector * sqrt(eigenvalue)
values, vectors = np.linalg.eigh(R)
values, vectors = values[::-1], vectors[:, ::-1]   # largest first
pc_load = np.abs(vectors[:, 0]) * np.sqrt(values[0])

# Factor analysis (maximum likelihood) on standardized items
Z = (X - X.mean(axis=0)) / X.std(axis=0)
fa = FactorAnalysis(n_components=1).fit(Z)
fa_load = np.abs(fa.components_[0])

print(np.round(np.column_stack([lam, fa_load, pc_load]), 3))  # true, factor, pca
print(round(values[0] / X.shape[1], 3))          # share of total variance on PC1
print(np.round(fa.noise_variance_, 3))           # variance unique to each item

# How well does each one-dimensional model rebuild the correlations?
off = np.tril_indices(4, -1)
fit_fa = np.outer(fa_load, fa_load)
fit_pca = np.outer(pc_load, pc_load)
print(round(R[off].mean(), 3), round(fit_fa[off].mean(), 3), round(fit_pca[off].mean(), 3))
print(round(np.abs(R[off] - fit_fa[off]).max(), 3), round(np.abs(R[off] - fit_pca[off]).max(), 3))

The figures below come from one seeded run of the R code. Python uses a different random number generator, so its figures differ slightly, but they show the same pattern.

The gap shrinks when items are reliable and there are many of them per factor, because then the unique variances are small relative to the shared variance. That is why the two methods often give similar pictures in large, clean datasets, and why the choice matters most for short scales with mixed-quality items.

Which one should you use?

How to report a factor analysis in APA style (7th edition)

In the method section, name the extraction method (for example maximum likelihood or principal axis factoring, not just 'factor analysis'), the rotation if any, and how you chose the number of factors. Using the example above, with placeholders for the fit test that the code does not print:

"A maximum likelihood factor analysis of the four items (N = 500) supported a single factor, χ²(2) = x.xx, p = .xxx. Standardized loadings ranged from .43 to .76, and uniquenesses ranged from .42 to .82."

If you ran PCA instead, call the results components rather than factors and report the percentage of variance explained (here, 54.5% for the first component). Put the loadings in a table with one row per item and one column per factor, and add communalities or uniquenesses as a final column.

Related tools and guides

More answered questions

Working with your own data?

General answers only go so far. Send us your situation and we'll reply by email within two business days.

Ask your question

Written by AskStats with AI assistance. This is general information, not advice for your specific data or study. When the results matter, check your approach with a qualified statistician.