Linear Regression

Machine Learning: Start with linear regression

“a very simple approach for supervised learning”

“widely used statistical learning method”

House Prices

Square feet | Price

\(y = Mx + b\)

Written as \(y = \beta_0 + \beta_1x\)

y is called response or target x predictor

Once we know betas we can predict value of response

Minimize the total error from each point to our line.

Bend the line to fit better

Square feet | # bedrooms | Price

cs229 Example

Simple Linear one variable

“We typically assume that the error term is independent of X.”

Multiple Linear Regression


We usually have more than one predictor.

\(Y = \beta_0 + \beta_1 X_1 + \cdots + \beta_p X_p + \epsilon\)

Estimating the Coefficients


\(\hat{y} = \hat{\beta}_0 + \beta_1 X_1 + \cdots + \beta_p X_p + \epsilon\)


Choose \(\beta\) to \(\beta_p\) that minimizes RSS p73