5 Assessing the Fit of a Regression

\(R^2\), SSTO, SSR, and SSE…

Not all regressions are created equally as the three plots below show. Sometimes the dots are a clustered very tightly to the line. At other times, the dots spread out fairly dramatically from the line.

A common way to measure the fit of a regression is with correlation. While this can be a useful measurement, there is greater insight in using the square of the correlation, called \(R^2\). Before you can understand \(R^2\), you must understand three important “sums of squares”.

(Read more about sums…)

A sum is just a fancy word for adding things together. \[ 1 + 2 + 3 + 4 + 5 + 6 = 21 \]

Long sums get tedious to write out by hand. So we use the symbol \(\Sigma\) to denote the word “sum”. Further, we use a subscript \(\Sigma_{i=1}\) to state what value the sum is beginning with, and a superscript \(\Sigma_{i=1}^6\) to state the value we are ending at. This gives \[ \sum_{i=1}^6 i = 1 + 2 + 3 + 4 + 5 + 6 = 21 \]

Test your knowledge, do you see why the answer is 6 to the sum below? \[ \sum_{i=1}^3 i = 6 \]

Computing sums in R is fairly easy. Type the following codes in your R Console.

sum(1:6) #gives the answer of 21

sum(1:3) #gives the answer of 6

However, sums really become useful when used with a data set.

Each row of a data set represents an “individual’s” data. We can reference each individual with a row number. In the data below, individual 3, denoted by \(i=3\), has a speed of 7 and a dist of 4.

pander(head(cbind(Individual = 1:50, cars), 6), emphasize.strong.rows=3)

Individual	speed	dist
1	4	2
2	4	10
3	7	4
4	7	22
5	8	16
6	9	10

To compute the sum of the speed column, use sum(speed). If we divided this sum by 6, we would get the mean of speed mean(speed). In fact, the two most used statistics mean(...) and sd(...) both use sums. Take a moment to review the formulas for mean and standard deviation. It is strongly recommended that you study the Explanation tab for both as well. We’ll wait. See you back here shortly.

…

Welcome back.

Suppose we let X = speed and Y = dist. Then \(X_3 = 7\) and \(Y_3 = 4\) because we are accessing row 3 of both the \(X\) (or speed) column and \(Y\) (or dist) column. (Remember from the above discussion that for individual #3, the speed was 7 and the dist was 4.) Further, sum(speed) would be written mathematically as \(\sum_{i=1}^6 X_i\) and sum(dist) would be written as \(\sum_{i=1}^6 Y_i\).

Sum of Squared Errors	Sum of Squares Regression	Total Sum of Squares
\(\text{SSE} = \sum_{i=1}^n \left(Y_i - \hat{Y}_i\right)^2\)	\(\text{SSR} = \sum_{i=1}^n \left(\hat{Y}_i - \bar{Y}\right)^2\)	\(\text{SSTO} = \sum_{i=1}^n \left(Y_i - \bar{Y}\right)^2\)
Measures how much the residuals deviate from the line.	Measures how much the regression line deviates from the average y-value.	Measures how much the y-values deviate from the average y-value.
Equals SSTO - SSR	Equals SSTO - SSE	Equals SSE + SSR
sum( (Y - mylm$fit)^2 )	sum( (mylm$fit - mean(Y))^2 )	sum( (Y - mean(Y))^2 )

---

It is important to remember that SSE and SSR split up SSTO, so that \[ \text{SSTO} = \text{SSE} + \text{SSR} \] This implies that if SSE is large (close to SSTO) then SSR is small (close to zero) and visa versa. The following three graphics demonstrate how this works.

The above graphs reveal that the idea of correlation is tightly linked with sums of squares. In fact, the correlation squared is equal to SSR/SSTO. And this fraction, SSR/SSTO is called \(R^2\) (“r-squared”).

R-Squared (\(R^2\)) \[ \underbrace{R^2 = \frac{SSR}{SSTO} = 1 - \frac{SSE}{SSTO}}_\text{Interpretation: Proportion of variation in Y explained by the regression.} \]

The smallest \(R^2\) can be is zero, and the largest it can be is 1. This is because \(SSR\) must be between 0 and SSTO, inclusive.