Tag
regression
22 answered questions tagged “regression”. Browse and search all answers, or ask your own.
Searches the regression answers on this page. Search all answers
Can you test the exclusion restriction of an instrumental variable?
Why data cannot confirm an instrument's exclusion restriction, and how placebo tests and sensitivity analysis help instead, with an R simulation.
Read the answer →How do you simulate a prediction interval from a fitted regression model?
A simulated prediction interval must combine estimation error and individual scatter. The recipe for one new case and for a group average, in R and Python.
Read the answer →What does it mean to control for a variable in regression?
How adding a covariate to a regression model 'holds it constant', why this replaces sorting people into subgroups, and what controlling can and cannot fix.
Read the answer →When can you drop the intercept from a linear regression?
Why a no-intercept model reports a higher R-squared, how forcing the line through zero biases the slope, and when leaving out the constant is justified.
Read the answer →Where does a curve level off? Estimating a plateau point
Polynomials cannot show where a curve flattens out. Fit a model with a built-in plateau, define the onset, and give it a confidence interval.
Read the answer →Which interaction terms belong in a regression or mixed model?
Which terms must accompany a three-way interaction, whether the others are required, and how choosing interactions from the data inflates false positives.
Read the answer →Is stepwise regression a good way to choose predictors?
Automatic stepwise selection finds patterns in pure noise and leaves p-values and R-squared too optimistic. Here is why, and what to do instead.
Read the answer →Bagging, boosting and stacking: how do these ensemble methods differ?
How bagging, boosting and stacking each combine many models, what problem each one fixes, and a side-by-side comparison on simulated data in R and Python.
Read the answer →Mean, median or quantile: which point prediction should a model give?
Why the loss you will be judged on, not the likelihood you fitted, decides whether a point prediction should be the conditional mean, the median or a quantile.
Read the answer →Skewed data with zeros: is log(x + 1) OK, and what else works?
Why log(x + c) gives results that hinge on an arbitrary constant, and better options for skewed data with zeros: log-link GLMs, two-part models and asinh.
Read the answer →t tests vs the F test in regression: what is the difference?
Why regression output gives a t test for each coefficient and one overall F test, when the two agree exactly, and why they can point in opposite directions.
Read the answer →Regression to the mean vs the gambler's fallacy: what is the difference?
Why expecting an extreme score to be followed by a more ordinary one is sound, while expecting a streak to be balanced out is a fallacy.
Read the answer →RMSE vs standard deviation: what does comparing them tell you?
Why the standard deviation of the outcome is the RMSE of a model that always predicts the mean, and how the ratio of the two gives R-squared.
Read the answer →Should you check normality of the outcome variable or of the residuals?
Linear models assume normal errors, not a normal outcome. A simulation shows how checking Y instead of the residuals misleads in both directions.
Read the answer →Training, validation and test sets: what is each one for?
What training, validation and test data are each for, why a validation score is too optimistic, and when cross-validation replaces it.
Read the answer →Fixed vs random effects in mixed models: what is the difference?
A fixed effect is a parameter you estimate for its own sake; a random effect treats groups as a sample from a wider population. An example in R and Python.
Read the answer →What goes wrong if you sort x and y separately before a regression?
Sorting each variable on its own breaks the pairs, so the fit looks excellent whatever the real relationship is. It predicts new data worse than the mean.
Read the answer →How do you interpret regression output in R?
A line-by-line guide to R's linear model summary: residuals, estimates, standard errors, t values, p-values, R-squared and the F test.
Read the answer →Centering vs standardizing predictors in regression: when to do which?
Centering and standardizing never change how well a regression fits. They change what the coefficients mean. Here is when each one helps.
Read the answer →Is R-squared useful or misleading?
R-squared depends on how spread out your predictors are, can look great for the wrong model, and never falls when you add terms. Here is when to trust it.
Read the answer →Logit vs probit: what is the difference, and which should you use?
Logit and probit models give almost the same fitted probabilities. They differ in coefficient scale and interpretation. Here is how to choose.
Read the answer →AIC vs BIC: which model selection criterion should you use?
AIC and BIC answer different questions. AIC aims for the best predictions; BIC aims to find the true model. Here is how to choose between them.
Read the answer →