Linear Regression

To find the best fitting line for some points, we find the squared error and try to minimize it
Let’s assume that the best fitting line is $y=mx+b$
Squared error which we minimize will be:

$$ MSE = \frac{1}{n}\sum_{i=1}^{N}(y_i-(mx_i + b))^2 $$

where $(x_i, y_i)$ is the $i^{th}$ observation and $mx_i +b$ is the predicted value of $y$

Interesting fact: Mean of x and mean of y will be a point on this best fitting line

Question: What % of the variation in y is described by the variation in x?

Alternate question: If we fit a line on the data points, how good is the fit of that line?

Total variation in y is the distance of each value of y from the mean of y. It is also called SST.

$$ SST = \sum(y_i - \bar{y})^2 $$