Multiple linear regression

No description yet. This curated deck is ready for daily study.

Total cards: 24•Content version: v4•Public deck•Updated: 8/1/2026

Card preview (24)

Front (Question)

disadvantage of doing regressions separately?

Back (Answer)

ignores potential synergy effect–>lead to misleading results

Front (Question)

RSE?

Back (Answer)

sqrt(1/(n-p-1) * RSS). n-(p+1) denominator is the degrees of freedom

Front (Question)

why does R squared increase when non-zero inputs are added?

Back (Answer)

RSS will decrease when choosing another (non-zero) parameter to estimate logic for finding a new set of coefficient based on minimising RSS

Front (Question)

what does adjusted R squared do?

Back (Answer)

adds a penalisation factor to account for the number of predictors included in the model

Front (Question)

formula of adjusted R square?

Back (Answer)

1-(n-1)/(n-1-p)*RSS/TSS. ALWAYS SMALLER THAN R SQAURE(can be neg)

Front (Question)

what is the null hypothesis H0?

Back (Answer)

all regression coefficients are 0 simultaneously

Front (Question)

formula of F stats?

Back (Answer)

((TSS-RSS)/P)/(RSS/(n-1-p))

Front (Question)

when no relation what is F stat?

Back (Answer)

Front (Question)

when to reject null hypothesis?

Back (Answer)

p-value<0.05

#10

Front (Question)

how does forward selection work?

Back (Answer)

• start with null model with intercept but no predictor • successively include most informative variable (lowest RSS, highest R square) • stop when stopping rule is reached (all variables have p-value<0.05)

#11

Front (Question)

how does backward elimination work?

Back (Answer)

• start with full model with intercept and all predictors • successively remove least informative variable (highest RSS, lowest R squared) • stop when stopping rule is reached (all variables have p-value<0.05)

#12

Front (Question)

how does cross validation work?

Back (Answer)

• split dataset into training and testing set • train model using training set • validate fitted model using testing set

#13

Front (Question)

how is validation error rate assessed?

Back (Answer)

mean squared error (1/n*RSS)

#14

Front (Question)

process of leave one out CV?

Back (Answer)

1,fit training data (obs=n-1) into a model • validate the model using testing set (obs=1) • compute to test MSE for first round • repeat 1-3 for n times to obtain n MSEs • construct LOOCV estimate as avg for MSEs

#15

Front (Question)

K-fold CV?

Back (Answer)

• randomly split observations into k groups • fit training data (obs=n-n1) into a model • validate the model using testing set (obs=n1) • compute to test MSE for first round • repeat 2-4 for k times to obtain k MSEs • construct K-FOLD CV estimate as avg of k MSEs

#16

Front (Question)

lm for multiple linear regression?

Back (Answer)

lm_fit=lm(y~var1+var2,data=Boston) summary(lm_fit)

#17

Front (Question)

get model with all variables?

Back (Answer)

lm_fit1=lm(y~. , data=Boston)

#18

Front (Question)

remove one or two variables?

Back (Answer)

lm_fit2=lm(y~. -var1 , data=Boston) lm_fit3=lm(y~.-var1 -var2, data=Boston)

#19

Front (Question)

get null set?

Back (Answer)

lm_fit4=lm(y~1,data=Boston)

#20

Front (Question)

get correlation for all inputs?

Back (Answer)

round(cor(Boston),2)

#21

Front (Question)

how to visualise the pair-wise correlation matrix?

Back (Answer)

install.packages(‘corrplot’) library(corrplot) cor_matrix=round(cor(Boston), 2) corrplot(cor_matrix, type = “upper”, order = ‘alphabet’, tl.col = “black”, tl.srt = 45, tl.cex = 0.9, method = “circle”)

#22

Front (Question)

for variables with high correlation, how to remove additive assumption?

Back (Answer)

lm_non_add=lm(y~var1*var2, data=Boston)

#23

Front (Question)

scatterplot with linear assumption?

Back (Answer)

attach(Boston) plot(var1,y,pch=16,col=’gray50’)

#24

Front (Question)

for polynomial?

Back (Answer)

lm_nonlinear=lm(y~poly(var1,2),data=Boston)