Regression

Scatterplots

Describe: direction (positive/negative), form (linear/nonlinear), strength (weak/moderate/strong), unusual features (outliers).

Correlation ($r$)

$r = \frac{1}{n-1}\sum\left(\frac{x_i - \bar{x}}{s_x}\right)\left(\frac{y_i - \bar{y}}{s_y}\right)$

• $-1 \leq r \leq 1$.
• Measures linear association only.
• $r$ is unitless and unaffected by changes in units.

Least-Squares Regression Line (LSRL)

$\hat{y} = a + bx$ where $b = r \cdot \frac{s_y}{s_x}$ and the line passes through $(\bar{x}, \bar{y})$.

Coefficient of Determination ($r^2$)

$r^2$ = proportion of variability in $y$ explained by the linear relationship with $x$.

Residuals

$\text{residual} = y - \hat{y}$

• Residual plot should show random scatter (no pattern).
• Patterns indicate the model is not appropriate.

Influential Points

• Outlier: large residual.
• High leverage: extreme $x$-value.
• Influential: removing it substantially changes the regression line.

Inference for Slope

$t = \frac{b - 0}{SE_b}, \quad df = n - 2$

CI for $\beta$: $b \pm t^* \cdot SE_b$.

$H_0: \beta = 0$ (no linear relationship).

Conditions (LINE):

1. Linear relationship (check residual plot).
2. Independence of observations.
3. Normal distribution of residuals.
4. Equal variance (constant spread in residual plot).

Transformations

If the relationship is nonlinear, transform one or both variables (e.g., $\ln y$ vs. $x$ for exponential, $\ln y$ vs. $\ln x$ for power).

Problem: From computer output: $b = 2.5$, $SE_b = 0.8$, $n = 20$. Test if the slope is significantly different from zero.

Solution:

$H_0: \beta = 0$, $H_a: \beta \neq 0$.

$t = 2.5/0.8 = 3.125$, $df = 18$.

p-value $\approx 0.006$. At $\alpha = 0.05$, reject $H_0$. There is convincing evidence of a linear relationship.

• LSRL: $\hat{y} = a + bx$; $b = r(s_y/s_x)$; passes through $(\bar{x}, \bar{y})$.
• $r$ measures linear association; $r^2$ tells the explained proportion.
• Residual plots check model fit; patterns = bad fit.
• Inference for slope: $t = b/SE_b$; check LINE conditions.
• Transform data for nonlinear relationships.