Unit 3 · Topic 3.6 · about 20 minutes

p-Values

Find the p-value of a test for a population proportion, from the normal model or from a simulation, and interpret it in context as a probability computed assuming the null hypothesis is true.

Predict first

A candy maker says 25% of its candies are red. A small bag you open has 4 red candies out of 20. A big jar has 80 red out of 400. Both are 20% red. Which result gives you more reason to doubt the company's claim?

The null distribution

A significance test starts by assuming the null hypothesis is true. Even in that imagined world, the test statistic changes from sample to sample, and its distribution there is called the null distribution. For a one-sample z-test for a proportion, the test statistic is the standardized score of $\hat{p}$ in that world,

z = \frac{\hat{p} - p_{0}}{\sqrt{\frac{p_{0} (1 - p_{0})}{n}}},

and when the conditions are met, its null distribution is the standard normal distribution. 3.7 Carrying Out a Test for a Population Proportion covers this calculation in full.

The p-value is the probability, computed assuming $H_{0}$ is true, of getting a test statistic as extreme as the one observed or more extreme. Extreme means in the direction of the alternative hypothesis, so the alternative decides which area you find.

Which area is the p-value, when $x$ is the observed value of the test statistic
Alternative hypothesis	p-value
$H_{a} : p > p_{0}$	$P (z \geq x)$ , the area at or above $x$
$H_{a} : p < p_{0}$	$P (z \leq x)$ , the area at or below $x$
$H_{a} : p \neq p_{0}$	$P (z \leq - \| x \|) + P (z \geq \| x \|)$ , the area in both tails beyond $\| x \|$

Worked exampleDog owners: finding and interpreting a p-value

A news report says that 40% of adults in a state own a dog. A pet food company believes the proportion is higher. In a random sample of 600 of the state's adults, 264 own a dog, so $\hat{p} = 0.44$ . The company tests $H_{0} : p = 0.40$ against $H_{a} : p > 0.40$ , where $p$ is the proportion of all adults in the state who own a dog. The conditions are met. Find and interpret the p-value.

Place the result in the null distribution. If $p = 0.40$ , then $\hat{p}$ has mean 0.40 and standard deviation $\sqrt{\frac{0.40 (0.60)}{600}} = 0.02$ . The test statistic is $z = \frac{0.44 - 0.40}{0.02} = 2.00$ .
Pick the tail. The alternative is $p > 0.40$ , so a result as extreme or more extreme means a test statistic at or above 2.00.
Find the area. p-value $= P (z \geq 2.00) \approx 0.0228$ .
Interpret. Assuming that 40% of all adults in the state own a dog, there is about a 0.023 probability of getting a sample proportion of 0.44 or higher in a random sample of 600 adults.
Weigh the evidence. A result this high would be unusual if the report were right, so the data provide evidence that more than 40% of the state's adults own a dog.

Answer.

The p-value is about 0.023. It describes samples in a world where the report is right. It is not the probability that the report is right.

Null distribution of $\hat{p}$ , assuming $p = 0.40$ and $n = 600$

Read on the z-scale under the axis, the same curve is the null distribution of the test statistic. The shaded area at or above $\hat{p} = 0.44$ , which is $z = 2$ , is the p-value, about 0.023.

When the null distribution is simulated

You can also get a p-value without any formula. Simulate the null distribution: generate many samples in a world where $H_{0}$ is true, compute the statistic for each one, and find the proportion of simulated values that are as extreme as the observed value or more extreme.

A student suspects a coin is unfair. She flips it 40 times and gets 25 heads, so $\hat{p} = 0.625$ . Her hypotheses are $H_{0} : p = 0.5$ and $H_{a} : p \neq 0.5$ , where $p$ is the proportion of all flips of this coin that land heads. A computer flipped a fair coin 40 times, recorded the proportion of heads, and repeated that 100 times.

Proportion of heads in 100 simulated sets of 40 flips of a fair coin

Five simulated proportions are at or above 0.625. Six are at or below 0.375, which is the same distance below 0.5.

Reading the simulation

The alternative is two-sided, so a result counts as extreme when it lands at least as far from 0.5 as 0.625 does, in either direction: at or above 0.625, or at or below 0.375. That is $5 + 6 = 11$ of the 100 simulated values, so the simulated p-value is about 0.11. With a one-sided alternative you would count one tail only: values at or above the observed one for $>$ , at or below it for $<$ .

Assuming the coin is fair, about 11% of sets of 40 flips give a proportion of heads at least as far from 0.5 as the student's 0.625. That is not unusual, so her flips do not provide convincing statistical evidence that the coin is unfair. They do not show that the coin is fair, either. A coin that lands heads 55% of the time could easily have produced the same 25 heads.

What small and not small mean

A small p-value says the observed result would be unusual if $H_{0}$ were true, so it provides evidence for $H_{a}$ . The lower the p-value, the more convincing the statistical evidence for $H_{a}$ . The dog owners' 0.023 is fairly convincing. A p-value of 0.0004 would be far more so.

A p-value that is not small says the result would not be unusual under $H_{0}$ . It does not provide convincing statistical evidence for $H_{a}$ , and it does not provide evidence that $H_{0}$ is true. Where to draw the line between small and not small is a decision made before the data are collected, and 3.7 Carrying Out a Test for a Population Proportion shows how it is made.

Lab

Is This Coin Fair?

Play several rounds with only a few flips and several with many. Before each reveal, look at the p-value the lab reports and ask how unusual your result would be for a fair coin. Notice that a biased coin tends to give smaller p-values when you flip it more, because the same lopsided proportion gets harder to explain by chance.

Open the full Is This Coin Fair? lab

Check your understanding

A city says that 70% of its residents recycle. A researcher suspects the proportion is lower. In a random sample of 300 residents, 195 recycle, so $\hat{p} = 0.65$ . For the test of $H_{0} : p = 0.70$ against $H_{a} : p < 0.70$ , the p-value is 0.029. Which is a correct interpretation of the p-value?

A test of $H_{0} : p = 0.30$ against $H_{a} : p \neq 0.30$ produces a test statistic of $z = 1.80$ . Which is the p-value?

To test $H_{0} : p = 0.30$ against $H_{a} : p > 0.30$ , a researcher simulated 200 random samples of the same size assuming $p = 0.30$ . Her actual sample gave $\hat{p} = 0.38$ . Of the 200 simulated sample proportions, 9 were at or above 0.38 and 3 were at or below 0.22. What is the simulated p-value?

A test of $H_{0} : p = 0.5$ against $H_{a} : p > 0.5$ , where $p$ is the proportion of a city's voters who favor a new park, gives a p-value of 0.41. Which statement is correct?

A test of $H_{0} : p = 0.60$ against $H_{a} : p < 0.60$ gives a test statistic of $z = - 1.52$ . Find the p-value. Round to four decimal places.

Your answer

Course alignment, for teachers

AP Statistics topic 3.6, Unit 3: Inference for Categorical Data: Proportions.

Skill 4.F: Interpret results of statistical inference methods.