Visualizing the Line of Best Fit
Understanding Least-Squares Regression
Let’s look at the method of least squares from another perspective. Imagine that you’ve plotted some data using a scatterplot, and that you fit a line for the mean of Y through the data. Let’s lock this line in place, and attach springs between the data points and the line.
Some of the data points are further from the mean line, so these springs are stretched more than others. The springs that are stretched the furthest exert the greatest force on the line.
What if we unlock this mean line, and let it rotate freely around the mean of Y? The forces on the springs balance, rotating the line. The line rotates until the overall force on the line is minimized.
The are some cool physics at play, involving the relationship between force and the energy needed to pull a spring a given distance. It turns out that minimizing the overall energy in the springs is equivalent to fitting a regression line using the method of least squares.