Answer

Does k-means make assumptions? Its drawbacks explained

Inspired by a question on Cross Validated · October 21, 2026

clusteringnormalization

The criterion is the assumption

It is true that k-means is an optimization problem: pick k centres and assign every point to its nearest centre so that the total squared Euclidean distance from points to their centres is as small as possible. Nothing in that recipe mentions normal distributions or variances. But choosing that particular objective is itself a modelling choice. It defines what a 'good' cluster means, and k-means will always find the partition that is best by that definition, whether or not it matches the structure you care about.

Two consequences follow directly from the objective. First, each point goes to the nearest centre, so the boundary between two clusters is the straight line (or plane) halfway between their centres. Second, distance is plain Euclidean distance, so every direction and every variable counts equally. Put another way, k-means is what you get from a mixture of normal distributions in which every cluster has the same round (spherical) spread and the same share of the data, and each point is assigned outright to one cluster. That is where the familiar list of 'assumptions' comes from: they are the conditions under which minimising this objective recovers the true groups.

Where each drawback comes from

• Round clusters. Squared distance treats a step in any direction the same. If a group is long and thin, points at its two ends are far apart, and the objective drops more by cutting the group across its long axis than by separating it from a parallel neighbour. K-means cuts in the wrong direction.
• Equal spread. The boundary sits halfway between centres. When a wide, diffuse group sits next to a tight one, the halfway line runs through the wide group, so its outer points are handed to the tight cluster.
• Similar sizes. The objective adds up squared distances over points, so a big group carries more weight than a small one. Splitting a large group often lowers the total more than respecting a small, distinct one, and the boundary ignores how common each group is.
• Comparable units. Euclidean distance mixes variables directly, so the one with the largest numeric spread dominates: income in dollars rather than thousands can change the clusters completely. So variables are usually standardized first, though that is a choice too, since it gives noisy variables full weight (see centering vs standardizing).

None of these makes k-means 'fail' in the sense of an error message. It always returns k groups. The danger is that the groups look convincing even when they are artefacts of the objective.

Practical limits beyond the shape of clusters

• You must choose k. The within-cluster total always falls as k grows, so it cannot pick k by itself. Elbow plots, silhouette widths and the gap statistic help, but they are guides, not tests.
• Local optima. The usual algorithms only find a local minimum that depends on the starting centres. R's kmeans() uses one random start unless you set nstart; use 25 or more and keep the best solution.
• Outliers. Means are pulled by extreme points, so one far-off point can grab a cluster of its own or drag a centre. k-medoids (PAM) is more resistant.
• Numeric data only. Means and Euclidean distances make little sense for categorical variables; use methods built for mixed data instead.
• It always finds clusters. Even data with no group structure are split into k pieces, so check whether the clusters are stable across resamples and meaningful in context.

When the round, equal-spread picture does not fit, Gaussian mixture models allow clusters with their own shapes, sizes and proportions; density-based methods such as DBSCAN find irregular shapes; and hierarchical clustering lets you choose the linkage to suit the data. If you only want to reduce many variables to a few before clustering, see our page on principal component analysis.

See it in R and Python

The example builds three data sets where the true groups are known: two long parallel bands, a big diffuse group next to a small tight one, and two age groups measured alongside an unrelated variable in dollars. It runs k-means with k = 2 and 25 random starts and reports the share of points placed in their true group (0.5 is no better than chance for two equal groups). The R version uses base R; the Python tab uses scikit-learn's KMeans.

set.seed(133656)
# Share of points put in the right group (two clusters, either label order)
agree <- function(cl, truth) {
  tab <- table(cl, truth)
  round(max(sum(diag(tab)), sum(tab) - sum(diag(tab))) / sum(tab), 3)
}

# 1. Long, thin clusters: two parallel bands 2 units apart
n <- 300
x1 <- cbind(rnorm(2 * n, 0, 5), c(rnorm(n, 0, 0.3), rnorm(n, 2, 0.3)))
g1 <- rep(1:2, each = n)
k1 <- kmeans(x1, centers = 2, nstart = 25)
cat("bands: agreement =", agree(k1$cluster, g1), "\n")

# 2. Unequal sizes and spreads: a big diffuse group next to a small tight one
x2 <- rbind(cbind(rnorm(500, 0, 1.5), rnorm(500, 0, 1.5)),
            cbind(rnorm(25, 4, 0.3), rnorm(25, 0, 0.3)))
g2 <- rep(1:2, c(500, 25))
k2 <- kmeans(x2, centers = 2, nstart = 25)
print(table(true = g2, kmeans = k2$cluster))
cat("unequal: agreement =", agree(k2$cluster, g2), "\n")

# 3. Units: groups differ on age; an unrelated variable is in dollars
age <- c(rnorm(n, 30, 4), rnorm(n, 50, 4))
dollars <- rnorm(2 * n, 40000, 8000)
x3 <- cbind(age, dollars)
g3 <- rep(1:2, each = n)
k3_raw <- kmeans(x3, centers = 2, nstart = 25)
k3_std <- kmeans(scale(x3), centers = 2, nstart = 25)
cat("raw units: agreement =", agree(k3_raw$cluster, g3), "\n")
cat("standardized: agreement =", agree(k3_std$cluster, g3), "\n")
cat("standardized: between_SS / total_SS =",
    round(k3_std$betweenss / k3_std$totss, 3), "\n")
cat("cluster sizes:", sort(k3_std$size), "\n")

import numpy as np
import pandas as pd
from sklearn.cluster import KMeans

rng = np.random.default_rng(133656)

def agree(cl, truth):
    # Share of points put in the right group (two clusters, either label order)
    same = np.mean(cl == truth)
    return round(float(max(same, 1 - same)), 3)

def km(x):
    return KMeans(n_clusters=2, n_init=25, random_state=1).fit(x)

# 1. Long, thin clusters: two parallel bands 2 units apart
n = 300
x1 = np.column_stack([rng.normal(0, 5, 2 * n),
                      np.concatenate([rng.normal(0, 0.3, n), rng.normal(2, 0.3, n)])])
g1 = np.repeat([0, 1], n)
print("bands: agreement =", agree(km(x1).labels_, g1))

# 2. Unequal sizes and spreads: a big diffuse group next to a small tight one
x2 = np.vstack([rng.normal(0, 1.5, (500, 2)),
                np.column_stack([rng.normal(4, 0.3, 25), rng.normal(0, 0.3, 25)])])
g2 = np.repeat([0, 1], [500, 25])
k2 = km(x2)
print(pd.crosstab(pd.Series(g2, name="true"), pd.Series(k2.labels_, name="kmeans")))
print("unequal: agreement =", agree(k2.labels_, g2))

# 3. Units: groups differ on age; an unrelated variable is in dollars
age = np.concatenate([rng.normal(30, 4, n), rng.normal(50, 4, n)])
dollars = rng.normal(40000, 8000, 2 * n)
x3 = np.column_stack([age, dollars])
g3 = np.repeat([0, 1], n)
z3 = (x3 - x3.mean(axis=0)) / x3.std(axis=0, ddof=1)   # same as R's scale()
k3_raw, k3_std = km(x3), km(z3)
print("raw units: agreement =", agree(k3_raw.labels_, g3))
print("standardized: agreement =", agree(k3_std.labels_, g3))
total_ss = np.sum((z3 - z3.mean(axis=0)) ** 2)
print("standardized: between_SS / total_SS =",
      round(float(1 - k3_std.inertia_ / total_ss), 3))
print("cluster sizes:", sorted(np.bincount(k3_std.labels_)))

The figures below come from one seeded run of the R code. Python uses a different random number generator, so its figures differ slightly, but they show the same pattern.

• Long, thin bands. Agreement is 0.52, essentially chance. K-means cut both bands across the middle instead of separating them, even though the gap between the bands is clear to the eye.
• Unequal groups. 212 of the 500 points in the big diffuse group were put in the same cluster as the 25 points of the small tight group, so agreement is only 0.596.
• Units. On the raw scale, agreement is 0.543, because the dollar variable has by far the largest spread and the clusters split on it. After standardizing both variables, agreement is 0.993, the two clusters have 300 points each, and they account for 43.5% of the total sum of squares.

In all three cases k-means reached its objective; the objective simply did not describe these groups. Plot your data (or its first few principal components) before trusting a clustering, and if you are choosing a method for a group comparison afterwards, try the test chooser.

How to report k-means clustering in APA style (7th edition)

In the method section, say which variables were used and how they were scaled, how k was chosen, which algorithm and software you used, and how many random starts. In the results, give the cluster sizes and the share of variance explained, and put the cluster centres in a table. Using the standardized example above:

"Age and income were standardized (z scores) and entered into a k-means cluster analysis with k = 2 (Hartigan-Wong algorithm, R kmeans, 25 random starts). The two clusters (n = 300 each) accounted for 43.5% of the total sum of squares."

Add a sentence on how k was chosen (for example, average silhouette width for k = 2 to 6) and on any stability check. A table with the mean of each variable per cluster, in original units, makes the clusters interpretable.

Related tools and guides

• Which statistical test should I use?
• PCA, eigenvectors and eigenvalues explained
• Centering vs standardizing predictors
• K-means clustering (Wikipedia)

More answered questions

• PCA, eigenvectors and eigenvalues explained in plain language
• Centering vs standardizing predictors in regression: when to do which?
• How do you normalize data to a 0-1 range?

Written by AskStats with AI assistance. This is general information, not advice for your specific data or study. When the results matter, check your approach with a qualified statistician.