Unit 5 · Topic 5.5 · about 25 minutes

Least-Squares Regression

Get the least-squares regression line from technology and interpret its slope, y-intercept and coefficient of determination in context.

Predict first

Three students each draw a line through the same scatterplot, and each claims to have the best one. Which rule do statisticians use to settle the argument?

What least squares means

Every line drawn through a scatterplot leaves residuals. Squaring them does two jobs. Every miss counts as positive, so misses above the line cannot cancel misses below it. And a big miss counts far more than a small one. Out of all possible lines, exactly one makes the sum of the squared residuals as small as it can be. That line is the least-squares regression line, often shortened to LSRL.

Jordan recorded the most pull-ups he could do in each of his first five weeks of training: 2, 5, 4, 7 and 9. Line A connects his first week to his fifth. Line B is the least-squares line.

Line A: $\hat{y} = 0.25 + 1.75 x$ . Line B: $\hat{y} = 0.6 + 1.6 x$ .
Week, $x$	Pull-ups, $y$	Line A residual	Line A squared	Line B residual	Line B squared
1	2	$0$	$0$	$- 0.2$	$0.04$
2	5	$1.25$	$1.5625$	$1.2$	$1.44$
3	4	$- 1.5$	$2.25$	$- 1.4$	$1.96$
4	7	$- 0.25$	$0.0625$	$0$	$0$
5	9	$0$	$0$	$0.4$	$0.16$
Sum of squares			$3.875$		$3.60$

Line A hits two of the points exactly and still loses, 3.875 to 3.60. No line through these five points does better than 3.60. That is what the least-squares line guarantees.

You will not find the least-squares line by trying lines one at a time, or by hand at all. A calculator or software computes the slope $b$ , the y-intercept $a$ and the correlation $r$ from the data.

One fact about the line is worth knowing without technology: it always passes through the point $(\overline{x}, \overline{y})$ . Jordan's averages are $\overline{x} = 3$ weeks and $\overline{y} = 5.4$ pull-ups, and Line B gives $0.6 + 1.6 (3) = 5.4$ exactly.

Jordan's five weeks with the least-squares line

The line passes through $(3, 5.4)$ , the point of averages, even though no week landed there.

Reading regression output

Software prints the least-squares line as a small table. This output comes from 15 apartments in one city, with floor area in square feet used to predict monthly rent in dollars.

Regression output: monthly rent against size for 15 apartments
Predictor	Coef
Constant	462.14
Size	1.2148

Under the table, the output adds S = 174.61 and R-Sq = 80.8%.

The Constant row holds the y-intercept, $a = 462.14$ .
The row named for the explanatory variable holds the slope, $b = 1.2148$ .
R-Sq is $r^{2}$ , the coefficient of determination, written as a percent. It is the proportion of the variation in the response variable that is explained by the linear relationship with the explanatory variable.

So the model is $\hat{rent} = 462.14 + 1.2148 (size)$ . Output usually prints more than you need. S is not part of this course, so find the two coefficients and R-Sq and leave the rest.

Rent and size for 15 apartments, with the least-squares line

Sizes run from 450 to 1,400 square feet. A size of 0 is far off the left edge of the data.

Worked exampleInterpreting the rent model

Use the output above. Interpret the slope, the y-intercept and $r^{2}$ in context, and find the correlation $r$ .

Slope. For each additional square foot of floor area, the predicted monthly rent increases by about 1.21 dollars. A square foot is small, so scaling up can help: 100 more square feet goes with about 121 dollars more in predicted rent.
Y-intercept. An apartment with 0 square feet has a predicted rent of 462.14 dollars. That has no reasonable meaning here. The apartments ranged from 450 to 1,400 square feet, so a size of 0 is far outside the data, and an apartment with no floor area does not exist.
$r^{2}$ . About 80.8% of the variation in monthly rent is explained by the linear relationship with floor area. The other 19.2% is variation the line does not account for, from things like location and the age of the building.
Correlation. $r = \sqrt{0.808} \approx 0.899$ . It is positive because the slope is positive; both describe the same upward direction.

Answer.

Slope: about 1.21 dollars more in predicted rent for each additional square foot. Intercept: no sensible meaning, since 0 square feet is far outside the data. $r^{2} = 0.808$ : about 80.8% of the variation in rent is explained by the linear relationship with size. $r \approx 0.899$ .

Lab

Regression Playground

Set Show to Squared residuals. Each residual becomes a square, and the least-squares line makes their total area as small as possible. Then switch Show to My line vs least squares and drag the ends of your line to try to beat it. You can get close, but you cannot win.

Open the full Regression Playground lab

Check your understanding

A model predicts a car's highway gas mileage (miles per gallon) from its weight (thousands of pounds). For this model, $r^{2} = 0.72$ . Which is the correct interpretation?

The least-squares line for predicting how many minutes a phone's battery lasts from its screen brightness setting (percent) is $\hat{y} = 640 - 3.8 x$ . Which is the correct interpretation of the slope?

For 40 students, the mean number of hours spent studying for a test is 6.2 and the mean test score is 81.5. The least-squares line for predicting score from hours studied has slope 2.4. What score does the line predict for a student who studied 6.2 hours?

A teacher used data from 35 students to predict the final course grade (percent) from the number of absences. Computer output for the least-squares regression is shown.

\begin{matrix} Predictor & Coef \\ Constant & 98.6 \\ Absences & - 2.35 \\ S = 6.41 & R-Sq = 49.0 % \end{matrix}

What final grade does the model predict for a student with 6 absences?

Each model below was fit to the data described. For which model does the y-intercept have a reasonable interpretation in context?

Course alignment, for teachers

AP Statistics topic 5.5, Unit 5: Regression Analysis.

Skill 3.B: Calculate summary statistics, relative positions of points within a distribution, and predicted responses.
Skill 4.D: Interpret statistical calculations and results to assess meaning or a claim.