Answer

What is AUC? The area under the ROC curve, explained

Inspired by a question on Cross Validated · September 25, 2026

classificationroc curvelogistic regression

Start with the ROC curve

Many models do not output a yes/no label directly. A logistic regression, a risk score or a lab biomarker gives each person a number, and you turn that number into a decision by picking a cut-off: above it you predict "has the condition", below it "does not". Every cut-off gives a different trade-off. A low cut-off catches most true cases (high sensitivity, also called the true positive rate) but raises many false alarms (a high false positive rate, which is 1 minus specificity). A high cut-off does the opposite.

The ROC curve plots the true positive rate against the false positive rate for every possible cut-off, from the strictest (bottom-left corner, nobody flagged) to the most lenient (top-right corner, everybody flagged). The name comes from signal detection work on radar during the Second World War, where operators had to separate real signals from noise. A model that separates the two groups well bends sharply toward the top-left corner; a useless model follows the diagonal line.

What the area means

The AUC is the area under that curve, so it runs from 0 to 1. Its most useful reading is as a probability: pick one person who has the outcome and one who does not, at random. The AUC is the chance that the model gives the first person the higher score (ties count as half). This is why AUC is also called the concordance or c-statistic in logistic regression output, and why it is exactly the Mann-Whitney U statistic divided by the number of case-control pairs.

• 0.5 means the scores carry no information about who has the outcome; the ROC curve sits on the diagonal.
• 1.0 means every case scores above every non-case, so some cut-off separates them perfectly.
• Below 0.5 means the scores point the wrong way; flipping their sign would give 1 minus the AUC.
• A widely quoted rule of thumb from Hosmer and Lemeshow's logistic regression textbook calls 0.7 to 0.8 acceptable, 0.8 to 0.9 excellent and 0.9 or more outstanding discrimination. Treat these as rough labels; what counts as good depends on the field and on what the model is for.
• Some software reports the Gini coefficient instead, which is 2 × AUC − 1 and runs from −1 to 1.

Because AUC depends only on how the scores rank people, any transformation that keeps the order (probabilities, log-odds, or the raw score that fed the model) gives the same AUC. It also does not depend on any single cut-off, which is its main appeal, and it is not affected by how common the outcome is in the way plain accuracy is.

Compute it in R and Python

This simulation gives 300 people a risk score, generates a yes/no outcome from a logistic model, fits the model with base R's glm(), and then computes the AUC three ways: as the area from the ROC points, as the share of case-control pairs ranked correctly, and from the Mann-Whitney statistic. It adds a standard error from the Hanley-McNeil formula. The Python tab does the same with scikit-learn and SciPy. For real work in R, the pROC package draws ROC curves and gives DeLong confidence intervals and tests for comparing two AUCs.

set.seed(4417)
n <- 300
score <- rnorm(n)                                   # a risk score or biomarker
y <- rbinom(n, 1, plogis(-1 + 1.2 * score))         # 1 = has the condition
fit <- glm(y ~ score, family = binomial)
p <- fitted(fit)                                    # predicted probabilities
c(cases = sum(y), controls = sum(1 - y))

# ROC curve: sensitivity and 1 - specificity at every cut-off
cuts <- c(Inf, sort(unique(p), decreasing = TRUE))
tpr <- sapply(cuts, function(k) mean(p[y == 1] >= k))
fpr <- sapply(cuts, function(k) mean(p[y == 0] >= k))
auc_trap <- sum(diff(fpr) * (head(tpr, -1) + tail(tpr, -1)) / 2)

# Same number as a probability: a random case outranks a random control
pairs <- outer(p[y == 1], p[y == 0], "-")
auc_pairs <- mean(pairs > 0) + 0.5 * mean(pairs == 0)

# Same number again from the Mann-Whitney statistic
W <- wilcox.test(p[y == 1], p[y == 0], exact = FALSE)$statistic
auc_mw <- unname(W) / (sum(y) * sum(1 - y))
round(c(trapezoid = auc_trap, pairs = auc_pairs, mann_whitney = auc_mw), 4)

# Only the ranking matters: the raw score gives the same AUC
pr <- outer(score[y == 1], score[y == 0], "-")
round(mean(pr > 0) + 0.5 * mean(pr == 0), 4)

# Hanley-McNeil standard error and a 95% CI
A <- auc_pairs; n1 <- sum(y); n0 <- sum(1 - y)
Q1 <- A / (2 - A); Q2 <- 2 * A^2 / (1 + A)
se <- sqrt((A * (1 - A) + (n1 - 1) * (Q1 - A^2) + (n0 - 1) * (Q2 - A^2)) / (n1 * n0))
round(c(AUC = A, SE = se, lower = A - 1.96 * se, upper = A + 1.96 * se), 3)

# Accuracy at a 0.5 cut-off, for contrast
round(mean((p >= 0.5) == y), 3)

import numpy as np
from scipy import stats
from scipy.special import expit
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score, roc_curve

rng = np.random.default_rng(4417)
n = 300
score = rng.normal(size=n)                            # a risk score or biomarker
y = rng.binomial(1, expit(-1 + 1.2 * score))          # 1 = has the condition
fit = LogisticRegression(penalty=None).fit(score.reshape(-1, 1), y)
p = fit.predict_proba(score.reshape(-1, 1))[:, 1]     # predicted probabilities
print("cases", y.sum(), "controls", n - y.sum())

# ROC curve: sensitivity and 1 - specificity at every cut-off
fpr, tpr, _ = roc_curve(y, p)
auc_trap = np.trapz(tpr, fpr)

# Same number as a probability: a random case outranks a random control
pairs = p[y == 1][:, None] - p[y == 0][None, :]
auc_pairs = np.mean(pairs > 0) + 0.5 * np.mean(pairs == 0)

# Same number again from the Mann-Whitney statistic
U = stats.mannwhitneyu(p[y == 1], p[y == 0]).statistic
auc_mw = U / (y.sum() * (n - y.sum()))
print("trapezoid", round(auc_trap, 4), "pairs", round(auc_pairs, 4),
      "mann_whitney", round(auc_mw, 4), "sklearn", round(roc_auc_score(y, p), 4))

# Only the ranking matters: the raw score gives the same AUC
print("raw score AUC", round(roc_auc_score(y, score), 4))

# Hanley-McNeil standard error and a 95% CI
A, n1, n0 = auc_pairs, y.sum(), n - y.sum()
Q1, Q2 = A / (2 - A), 2 * A**2 / (1 + A)
se = np.sqrt((A * (1 - A) + (n1 - 1) * (Q1 - A**2) + (n0 - 1) * (Q2 - A**2)) / (n1 * n0))
print("AUC", round(A, 3), "SE", round(se, 3), "CI", np.round([A - 1.96 * se, A + 1.96 * se], 3))

# Accuracy at a 0.5 cut-off, for contrast
print("accuracy", round(np.mean((p >= 0.5) == y), 3))

The figures below come from one seeded run of the R code. Python uses a different random number generator, so its figures differ slightly (for example, an AUC of 0.764 instead of 0.739), but the pattern is the same: all methods agree exactly.

• Three routes, one number. The sample has 93 cases and 207 controls. The trapezoid area under the ROC points, the share of correctly ranked case-control pairs, and the Mann-Whitney statistic all give an AUC of 0.739.
• Only the order matters. Using the raw score instead of the fitted probabilities gives the same 0.739, because the logistic model only rescales the score without changing its order.
• Uncertainty. The Hanley-McNeil standard error is 0.033, giving a 95% CI of 0.675 to 0.803.
• Why accuracy can mislead. Accuracy at a 0.5 cut-off is 0.733, barely above the 0.69 you would get by predicting "no" for everyone (207 of 300). AUC is not fooled by the imbalance in the same way.

What AUC does not tell you

• Whether the probabilities are right. AUC measures discrimination (ranking), not calibration. Multiply every predicted probability by one half and the AUC is unchanged, yet the predictions are badly off. Check calibration separately, for example with a calibration plot.
• Which cut-off to use. AUC averages over all cut-offs, including ones nobody would use. Choose a working threshold from the costs of false positives and false negatives, and report sensitivity and specificity at that point.
• How the model does on new data. An AUC computed on the data used to fit the model is optimistic. Use cross-validation, a bootstrap correction, or a separate test set.
• Performance on rare outcomes. When positives are very rare, a high AUC can hide many false alarms among the people you flag. A precision-recall curve is often more informative there.

One more source of confusion: in pharmacology, AUC usually means the area under a drug's concentration-time curve, a measure of total exposure. It is the same geometric idea applied to a different curve, so check which one a paper means.

How to report AUC in APA style (7th edition)

In the method section, say what the scores were (for example, predicted probabilities from a logistic regression), whether the AUC was computed on the training data, a test set or by cross-validation, and how the confidence interval was obtained (for example, DeLong or Hanley-McNeil). Because AUC cannot exceed 1, write it without a leading zero. Using the example above:

"The risk score discriminated between participants with and without the condition (93 cases, 207 controls), AUC = .74, 95% CI [.68, .80]."

If you chose a cut-off, add a sentence such as "At a cut-off of x.xx, sensitivity was .xx and specificity was .xx." Include the ROC curve as a figure when the AUC is a main result, and when comparing two models on the same people, report both AUCs with a test for correlated AUCs such as DeLong's.

Related tools and guides

• Which statistical test should I use?
• Logit vs probit: which should you use?
• Receiver operating characteristic (Wikipedia)
• pROC package for ROC analysis (CRAN)

More answered questions

• Logit vs probit: what is the difference, and which should you use?

Written by AskStats with AI assistance. This is general information, not advice for your specific data or study. When the results matter, check your approach with a qualified statistician.