Answers
Statistics questions, answered
Plain-language answers to statistics questions people are asking. Have your own? Ask us.
Build a raw-score-to-T-score crosswalk table for a shortened scale
Convert summed scores on a 6-item short form to the full-scale T-score metric with IRT and the Lord-Wingersky algorithm, in R and Python.
Read the guide →Reviewer 2 criticized your shortened survey scale? How to defend it
The post-hoc evidence to put in a revise-and-resubmit when a reviewer questions an abbreviated scale: CFA fit, McDonald's omega, local independence and IRT precision.
Read the guide →How to shorten a validated survey scale without losing validity
Why picking the items with the highest Cronbach's alpha backfires, and how item response theory and automated test assembly build a defensible short form.
Read the guide →What does it mean to control for a variable in regression?
How adding a covariate to a regression model 'holds it constant', why this replaces sorting people into subgroups, and what controlling can and cannot fix.
Read the answer →SVD and PCA: how the singular value decomposition does the same job
An SVD of the centered data matrix is PCA: V gives the axes, U times D the scores, and squared singular values over n - 1 the variances.
Read the answer →When can you drop the intercept from a linear regression?
Why a no-intercept model reports a higher R-squared, how forcing the line through zero biases the slope, and when leaving out the constant is justified.
Read the answer →Where does a curve level off? Estimating a plateau point
Polynomials cannot show where a curve flattens out. Fit a model with a built-in plateau, define the onset, and give it a confidence interval.
Read the answer →Which interaction terms belong in a regression or mixed model?
Which terms must accompany a three-way interaction, whether the others are required, and how choosing interactions from the data inflates false positives.
Read the answer →Dunn's test after Kruskal-Wallis: which p-value adjustment?
Whether pairwise Dunn tests need adjusting after Kruskal-Wallis rejects, and how Holm, Šidák and Benjamini-Hochberg differ.
Read the answer →Precision-recall or ROC: which should you use for rare outcomes?
Precision drops as positives get rarer; sensitivity and specificity do not. When precision-recall beats ROC, and how to report average precision.
Read the answer →Is an F-test gate needed before Tukey's HSD?
Whether the overall F test must reject before follow-up tests between groups, what that gate does to false positives and power, and when F still helps.
Read the answer →Pearson or Spearman correlation: which should you use?
What Pearson's r and Spearman's rho each measure, when they disagree (curved relationships and outliers), and how to pick and report one.
Read the answer →How many participants do you need when comparing several treatment arms?
Plan a multi-arm study around the comparisons you will report, not the ANOVA F test, and see how a multiplicity correction changes the numbers.
Read the answer →Is stepwise regression a good way to choose predictors?
Automatic stepwise selection finds patterns in pure noise and leaves p-values and R-squared too optimistic. Here is why, and what to do instead.
Read the answer →Bagging, boosting and stacking: how do these ensemble methods differ?
How bagging, boosting and stacking each combine many models, what problem each one fixes, and a side-by-side comparison on simulated data in R and Python.
Read the answer →Mean, median or quantile: which point prediction should a model give?
Why the loss you will be judged on, not the likelihood you fitted, decides whether a point prediction should be the conditional mean, the median or a quantile.
Read the answer →Bayesian vs frequentist statistics in plain English
Frequentists treat probability as long-run frequency; Bayesians treat it as degree of belief. Here is what that changes, with one example done both ways.
Read the answer →Why Euclidean distance breaks down in high dimensions
Why distances between points become nearly equal as features pile up, and how to tell if clustering on 100 features is still trustworthy.
Read the answer →Can you run a chi-square test on weighted survey estimates?
A chi-square test on weighted survey totals treats millions of estimated people as real observations. Why that fails and what to do instead.
Read the answer →Chi-square goodness of fit vs test of independence: what is the difference?
Both tests use the same Pearson formula. What differs is the question, where the expected counts come from, and so the degrees of freedom.
Read the answer →Does k-means make assumptions? Its drawbacks explained
Why k-means, though 'just an optimization', behaves as if clusters were round, equally spread and equally sized, and what happens when they are not.
Read the answer →Why does a t test use the t distribution and ANOVA the F distribution?
How the t, F and chi-square reference distributions are derived from the way each test statistic is built, and what to do when no formula exists.
Read the answer →Where should histogram bin edges go when your data are rounded?
When values are recorded to a fixed unit, put histogram bin edges halfway between the possible values. Here is why, and how counts differ.
Read the answer →Skewed data with zeros: is log(x + 1) OK, and what else works?
Why log(x + c) gives results that hinge on an arbitrary constant, and better options for skewed data with zeros: log-link GLMs, two-part models and asinh.
Read the answer →Chi-square test with expected counts below 5: what should you do?
What the rule of 5 really checks, when sparse cells matter, and how Fisher's exact test or a Monte Carlo p value fixes 2x2 and larger tables.
Read the answer →How do you test whether event counts vary by month?
A chi-square goodness-of-fit test weighted by month length, its blind spots, and a Poisson regression that tests one seasonal wave on 2 df.
Read the answer →How do you compare medians when the groups differ in shape and spread?
Why Mann-Whitney is not a test of medians when shapes differ, and how Mood's median test and a bootstrap interval compare medians directly.
Read the answer →Kolmogorov-Smirnov, t test or Mann-Whitney: which detects a change from a baseline?
What each two-sample test detects, why Kolmogorov-Smirnov misses changes in spread with small samples, and how to compare against a baseline.
Read the answer →How do you interpret the ICC when random effects are nested?
With readings nested in stations nested in sites, one model gives several ICCs. Which one answers which question, with R and Python examples.
Read the answer →Significant ANOVA but no significant pairwise comparisons: how can that happen?
Why a one-way ANOVA F test can reject while every pairwise or Tukey comparison does not (and the reverse), and how far apart their p values can get.
Read the answer →t tests vs the F test in regression: what is the difference?
Why regression output gives a t test for each coefficient and one overall F test, when the two agree exactly, and why they can point in opposite directions.
Read the answer →Factor analysis vs PCA: what is the difference?
PCA compresses all the variance in your variables into components; factor analysis models only the shared variance as coming from hidden factors.
Read the answer →Why can a CI for Cohen's d include zero when the t test is significant?
Why an effect size interval for Cohen's d can cross zero while the mean difference interval does not, and which d and interval agree with the test.
Read the answer →Regression to the mean vs the gambler's fallacy: what is the difference?
Why expecting an extreme score to be followed by a more ordinary one is sound, while expecting a streak to be balanced out is a fallacy.
Read the answer →Why does the Welch t statistic only approximately follow a t distribution?
Why a sum of two weighted chi-square variances breaks the exact t distribution, and how the Satterthwaite fractional df approximate it.
Read the answer →False positive risk: how likely is a significant finding to be wrong?
Why low power raises the share of significant findings that are false, inflates the real effects that get through, and what one researcher can do.
Read the answer →RMSE vs standard deviation: what does comparing them tell you?
Why the standard deviation of the outcome is the RMSE of a model that always predicts the mean, and how the ratio of the two gives R-squared.
Read the answer →Confidence interval for eta squared: does the Fisher z shortcut work?
Why the Fisher z shortcut gives unreliable intervals for eta squared, and how to get an exact one from F and its degrees of freedom.
Read the answer →Mann-Whitney effect size: is z/√N right, and which CI should you report?
Why z/√N is a shaky effect size for Mann-Whitney, why Fisher's z interval does not suit it, and which effect sizes and intervals to report instead.
Read the answer →Should you check normality of the outcome variable or of the residuals?
Linear models assume normal errors, not a normal outcome. A simulation shows how checking Y instead of the residuals misleads in both directions.
Read the answer →Training, validation and test sets: what is each one for?
What training, validation and test data are each for, why a validation score is too optimistic, and when cross-validation replaces it.
Read the answer →What should you do after rejecting the null hypothesis?
A significant result is a starting point, not a conclusion. Estimate the effect, check that it is robust, and replicate before you treat it as established.
Read the answer →Why is accuracy a poor way to judge a classification model?
Why the share of correct labels misleads with rare outcomes, ignores probabilities and costs, and which measures (Brier score, log loss, AUC) to use instead.
Read the answer →How to interpret the F statistic and p value in a one-way ANOVA
What the F ratio in an ANOVA table measures, how it maps onto the p value and the critical F, and what a significant F does and does not tell you.
Read the answer →Planned contrasts vs post hoc tests: why do they give different p-values?
Why one comparison gets different p-values as a planned contrast, a Tukey test or a Bonferroni test, and when to use each.
Read the answer →Bonferroni correction: when should you use it, and what are the alternatives?
What a Bonferroni correction protects against, why it costs so much power, and how Holm and Benjamini-Hochberg compare.
Read the answer →Why does almost everything become statistically significant with a large sample?
Large samples do not bias a test toward rejecting a true null; they detect tiny real differences. How to tell significant from important.
Read the answer →What does r mean when it is reported next to a t test, and how do you turn it into d?
Why papers report an r with a t test, how it is computed from t, what r squared means, how to convert r to Cohen's d, and how to report it in APA style.
Read the answer →t test or Mann-Whitney in small samples: how do you choose?
Why a normality pre-check is a weak way to pick a t test or a rank test with small n, what a power simulation shows, and where permutation tests fit.
Read the answer →Paired vs independent t test: which should you use?
How to tell whether your data are paired, why pairing can turn a hopeless comparison into a clear one, and what goes wrong when you pick the wrong test.
Read the answer →How are alpha and beta (Type I and Type II error rates) related?
Why lowering alpha raises beta, why the trade-off is not linear or a fixed ratio, and what a Bonferroni correction does to power.
Read the answer →How do you compare a short ordinal rating scale between two groups?
t test, Mann-Whitney or chi-square for a 1-4 rating item: what each one tests, how they compare in a simulation, and how to report the result.
Read the answer →Standard error vs standard deviation: what is the difference?
The SD describes how individual values vary; the standard error describes how a sample mean varies. How to tell which one you need, totals included.
Read the answer →Is the chi-square test one-tailed or two-tailed?
Why a chi-square test reads only the right end of its curve yet detects differences in any direction, and why its p value is never halved.
Read the answer →t test vs ANOVA with two groups: are the assumptions really different?
Why a pooled t test and a one-way ANOVA on two groups give the same p value, and why their normality assumptions are the same thing stated two ways.
Read the answer →Why does the central limit theorem work? An intuitive explanation
Averages look normal because averaging erases every feature of the original shape except its center and spread. The idea, without heavy math.
Read the answer →Is a non-significant result from a large study evidence for the null?
When failing to reject the null hypothesis tells you something, why the p value alone cannot show it, and how equivalence tests make the case.
Read the answer →How do you interpret Cohen's d, and are 0.2, 0.5 and 0.8 a real standard?
What Cohen's d says about group overlap, where the 0.2/0.5/0.8 cutoffs come from, why fields differ, and how to report d in APA style.
Read the answer →Does a t test need a minimum sample size?
Why no fixed minimum n makes a t test valid, what small samples really cost (power and robustness), and how to justify your sample size.
Read the answer →Welch vs Student t test: should you just always use Welch?
Why Welch's t test is the safer default for two independent groups, what it costs when variances are equal, and where Mann-Whitney fits.
Read the answer →Fixed vs random effects in mixed models: what is the difference?
A fixed effect is a parameter you estimate for its own sake; a random effect treats groups as a sample from a wider population. An example in R and Python.
Read the answer →One-tailed vs two-tailed tests: why not just test the direction the data point to?
Why a two-tailed test is the default, why choosing the tail after seeing the data doubles your false-positive rate, and when a one-tailed test is legitimate.
Read the answer →PCA, eigenvectors and eigenvalues explained in plain language
PCA finds the directions your data varies most. Eigenvectors are those directions; eigenvalues say how much variance each one carries.
Read the answer →What goes wrong if you sort x and y separately before a regression?
Sorting each variable on its own breaks the pairs, so the fit looks excellent whatever the real relationship is. It predicts new data worse than the mean.
Read the answer →How do you interpret regression output in R?
A line-by-line guide to R's linear model summary: residuals, estimates, standard errors, t values, p-values, R-squared and the F test.
Read the answer →Log-transforming data: when it helps, and what it changes
Logs turn multiplicative, right-skewed data into additive, roughly symmetric data. Here is when that helps and how results translate back.
Read the answer →What do p values and t values actually mean?
A t value is an effect measured in standard errors; the p value turns it into a tail probability under the null. What both mean, in plain language.
Read the answer →What is AUC? The area under the ROC curve, explained
AUC is the area under the ROC curve: the chance a model ranks a random positive case above a random negative one. How to read, compute and report it.
Read the answer →What are the assumptions of the Wilcoxon signed-rank test?
Independent pairs, rankable differences and symmetry: what each assumption means, and why a non-significant symmetry test proves little.
Read the answer →Centering vs standardizing predictors in regression: when to do which?
Centering and standardizing never change how well a regression fits. They change what the coefficients mean. Here is when each one helps.
Read the answer →Is R-squared useful or misleading?
R-squared depends on how spread out your predictors are, can look great for the wrong model, and never falls when you add terms. Here is when to trust it.
Read the answer →Logit vs probit: what is the difference, and which should you use?
Logit and probit models give almost the same fitted probabilities. They differ in coefficient scale and interpretation. Here is how to choose.
Read the answer →What are degrees of freedom in statistics, really?
Degrees of freedom count how much of your data is left for estimating variability once other quantities are estimated. The idea in plain language.
Read the answer →AIC vs BIC: which model selection criterion should you use?
AIC and BIC answer different questions. AIC aims for the best predictions; BIC aims to find the true model. Here is how to choose between them.
Read the answer →What is the beta distribution, and what is it used for?
The beta distribution describes uncertainty about a proportion or rate. Here is what its horizontal axis means and how its two parameters shape it.
Read the answer →Why does bootstrapping work? A plain-language explanation
Resampling your own data sounds like cheating, yet the bootstrap gives honest standard errors. Here is the idea in everyday terms.
Read the answer →What does a normal Q-Q plot's shape actually tell you?
A normal Q-Q plot compares your data to a normal distribution. The shape of the departure from the line tells you what kind of departure you have.
Read the answer →How do you normalize data to a 0-1 range?
Min-max scaling rescales any set of numbers so the smallest value becomes 0 and the largest becomes 1, using one formula.
Read the answer →Is there a 95% probability that your confidence interval covers the true mean?
The 95% describes the method that builds the interval, not the one interval you computed. Here is why that distinction matters, and when it does not.
Read the answer →Interrupted time series vs event study vs difference-in-differences
All three estimate the effect of an intervention, but each builds its counterfactual differently. Here is how to tell them apart.
Read the answer →Is a normality test worth running on your data?
Normality tests answer the wrong question: with lots of data they flag harmless quirks, with little data they miss real problems. Here is what to do.
Read the answer →What does kurtosis actually tell you about your data?
Kurtosis describes how heavy a distribution's tails are, not how peaked it is. Here is how to read it, in plain language.
Read the answer →Where does the 1/sqrt(n) margin of error come from?
The rule that a poll has a margin of error of about 1/sqrt(n) is a short derivation, not a guess. Here it is step by step.
Read the answer →Why does standard deviation square the differences?
Squaring deviations makes variances add up, ties into least squares and the normal distribution, and keeps the math clean.
Read the answer →