Hypothesis Testing

The Hypotheses

• Null hypothesis ($H_0$): the default claim (usually "no effect" or "no difference").
• Alternative hypothesis ($H_a$): the claim we're looking for evidence to support.
  • Two-sided: $H_a: \mu \neq \mu_0$
  • One-sided: $H_a: \mu > \mu_0$ or $H_a: \mu < \mu_0$

Test Statistic

Measures how far the sample statistic is from the hypothesized parameter in standard error units.

For a proportion: $z = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}$

For a mean: $t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}}$ with $df = n-1$.

P-Value

The probability of observing data as extreme as (or more extreme than) the sample, assuming $H_0$ is true.

• Small p-value → evidence against $H_0$.
• If $p \leq \alpha$: reject $H_0$.
• If $p > \alpha$: fail to reject $H_0$.

Significance Level ($\alpha$)

Common values: $\alpha = 0.05$ or $\alpha = 0.01$.

Type I and Type II Errors

Power = $1 - \beta$ = probability of correctly rejecting a false $H_0$.

Increase power by: increasing $n$, increasing $\alpha$, increasing effect size.

Four-Step Process (AP)

1. State hypotheses and define parameters.
2. Plan: identify the test, check conditions.
3. Do: calculate test statistic and p-value.
4. Conclude: make a decision in context.

Problem: A company claims its light bulbs last 1000 hours on average. A sample of 36 bulbs has $\bar{x} = 985$, $s = 40$. Test at $\alpha = 0.05$.

Solution:

$H_0: \mu = 1000$, $H_a: \mu < 1000$ (one-sided).

$t = \frac{985 - 1000}{40/\sqrt{36}} = \frac{-15}{6.67} = -2.25$

$df = 35$. From $t$-table, p-value $\approx 0.015$.

Since $0.015 < 0.05$, reject $H_0$. There is convincing evidence the mean lifetime is less than 1000 hours.

• Set up $H_0$ and $H_a$; calculate test statistic and p-value.
• Reject $H_0$ if p-value ≤ $\alpha$; fail to reject otherwise.
• Type I error = false positive; Type II error = false negative.
• Power = $1 - \beta$; increased by larger $n$, larger effect, or larger $\alpha$.
• Follow the 4-step process: State, Plan, Do, Conclude.