STA258 Lecture 17

• Bonus Question:
  • The largest win in Winter Olymics Hockey History was in 1924 in Chamonix, France. Canada (33) - Switzerland (0)
• New Topic
  • Analysis of Variance (ANOVA)
  • If we have an experiment and three treatments: A, B, C.
  • Analysis of Variance is concerned with $2$ or more groups, rather than just having two groups like in the two-sample t-test.
• Example:
  • Suppose we observe a numerical response value under several different conditions.
  • Under context of ANOVA, this is referred to as treatments.
  • We want to determine whether all population means are equal.
  • ANOVA Decomposes the total variability in the data into two components:
    • Variablity between groups (treatments)
    • Variability within groups (treatments)
  • $\begin{array}{cc}\text{A}& \text{B}\\ 2& 3\\ 4& 5\\ 6& 7\end{array}$
  • But if we have another group
  • $\begin{array}{ccc}\text{A}& \text{B}& C\\ 2& 3& 8\\ 4& 5& 9\\ 6& 7& 10\end{array}$
  • Then we use ANOVA
• Example:
  • Assume we have $k$ groups.
  • In group $i$, we observer the following:
    • $Y_{i_1},Y_{i_2},\dots ,Y_{i_{n_i}}$ for $i=1,2,\dots ,k$
  • $i$ indexes the group (treatment / population).
  • $j$ indexes the observation within the group $i$.
  • $n_i$ is the sample size for group $i$.
  • $n=\sum _{i=1}^k{n}_i$
  • If we want to sum observations for a certain treatment. We can denote it as:
    • $Y_{i\ast }=\sum _{j=1}^{n_i}{Y}_{ij}$
  • Average along each example column before
    • ${\overline{Y}}_{i\ast }=\frac{1}{n_i}\sum _{j=1}^{n_i}{Y}_{ij}=\frac{Y_{i\ast }}{n_i}$
  • Assumptions:
    • All observations are independent
    • Each group is normally distributed
    • Equal variances
      • ${\sigma }_1^2={\sigma }_2^2=\cdots ={\sigma }_k^2={\sigma }^2$
  • Means for each $k$ populations is unknown. That's of interest.
  • $Y_{ij}\sim N(\mu ,{\sigma }^2)$ for $i=1,2,\dots ,k$ and $j=1,2,\dots ,n_i$
  • $H_0:{\mu }_1={\mu }_2=\cdots ={\mu }_k$
  • $H_a:\text{not all of }{\mu }_1,\dots ,{\mu }_k\text{ are equal}$
  • If we have a group with $(\genfrac{}{}{0ex}{}{k}{2})$ we can perform pairwise t-tests to compare the means of each pair of groups.
  • However, this approach can lead to an increased risk of Type I errors due to multiple comparisons.
  • ANOVA provides a more robust method for testing the overall equality of means across all groups simultaneously, without inflating the Type I error rate.
  • Doing $(\genfrac{}{}{0ex}{}{5}{2})$ is way to difficult by hand.
• Fundamental Idea behind ANOVA:
  • Compositions of partitions of variability in the data.
  • The total variability in response is quantified by the Total Sum of Squares (SST):
    • $SST=\sum _{i=1}^k\sum _{j=1}^{n_i}(Y_{ij}-{\overline{Y}}_{\ast \ast }{)}^2$
    • ${\overline{Y}}_{\ast \ast }=\frac{1}{n}\sum _{i=1}^k\sum _{j=1}^{n_i}{Y}_{ij}$ is the overall mean.
    • $TSS=SST+S\mathrm{SE}[X]$
    • $SST$ is the sum of sqaures for treatments.
      • Measures variability between groups.
    • $SSE$ is the sum of squares for error
      • Measures variability within groups.
    • $Y_{ij}-\overline{Y}=(Y_{ij}-{\overline{Y}}_{i\ast })+({\overline{Y}}_{i\ast }-\overline{Y})⟹(Y_{ij}-\overline{Y}{)}^2$
    • $(Y_{ij}-\overline{Y}{)}^2=(Y_{ij}-{\overline{Y}}_{i\ast }{)}^2+({\overline{Y}}_{i\ast }-\overline{Y}{)}^2+2(Y_{ij}-{\overline{Y}}_{i\ast })({\overline{Y}}_{i\ast }-\overline{Y})$
    • $\sum _{i=1}^k\sum _{j=1}^{n_i}(Y_{ij}-\overline{Y}{)}^2+2\sum _{i=1}^k\sum _{j=1}^{n_i}(Y_{ij}-{\overline{Y}}_{i\ast })({\overline{Y}}_{i\ast }-\overline{Y})$
    • The middle term doesn't depend on $j$
    • Since it's just a constant then, multiply by $n$
    • $\sum _{i=1}^k\sum _{j=1}^{n_i}(Y_{ij}-{\overline{Y}}_{i\ast }{)}^2+\sum _{i=1}^k\sum _{j=1}^{n_i}({\overline{Y}}_{i\ast }-\overline{Y}{)}^2+2\sum _{i=1}^k\sum _{j=1}^{n_i}(Y_{ij}-{\overline{Y}}_{i\ast })({\overline{Y}}_{i\ast }-\overline{Y})$
    • $\sum _{i=1}^k\sum _{j=1}^{n_i}(Y_{ij}-{\overline{Y}}_{i\ast }{)}^2+\sum _{i=1}^k{n}_i({\overline{Y}}_{i\ast }-\overline{Y}{)}^2+2\sum _{i=1}^k\sum _{j=1}^{n_i}(Y_{ij}-{\overline{Y}}_{i\ast })({\overline{Y}}_{i\ast }-\overline{Y})$
    • $\sum _{j=1}^{n_i}(Y_{ij}-{\overline{Y}}_{i\ast })=\sum _{j=1}^{n_i}{Y}_{ij}-n_i{\overline{Y}}_{i\ast }$
    • We have the Total SS = SSE + SST
    • $SSE=\sum _{i=1}^k\sum _{j=1}^{n_i}(Y_{ij}-{\overline{Y}}_{i\ast }{)}^2$
      • This measures the variability within groups (treatments) around their own group mean.
    • We know $S_i^2=\frac{1}{n_i-1}\sum _{i=1}^{n_i}(Y_{ij}-{\overline{Y}}_{i\ast }{)}^2$
      • $SSE=\sum _{i=1}^k(n_i-1)S_i^2$
      • Pooled variation for group. Sum of Squares.
    • $SST=\sum _{i=1}^k{n}_i({\overline{Y}}_{i\ast }-\overline{Y}{)}^2$
      • Measures the variability between groups (treatments) around the overall mean.
    • Total Degress of Freedom:
      • $n-1$
    • Treatement Degrees of Freedom:
      • $k-1$
    • Error Degrees of Freedom:
      • $n-k$
  • Mean Square for Treatment:
    • $MST=\frac{SST}{k-1}$
  • Mean Square for Error:
    • $MSE=\frac{SSE}{n-k}$
  • Divide each respective sum of squares by their degrees of freedom to get the mean squares.
  • $\mathrm{E}[\mathrm{MSE}]=\frac{1}{n-k}\mathrm{E}[SSE]=\frac{1}{n-k}\mathrm{E}\left[\frac{\sum _{i=1}^k(n_i-1)S_i^2}{n_1+n_2+\cdots +n_k-k}\right]={\sigma }^2$
    • By the properties of the pooled variance
  • $S_p^2=\frac{(n_1-1)S_1^2+(n_2-1)S_2^2}{n_1-n_2-2}$
  • It's similar to the pooled variance.
• Under $H_0:MST\approx MSE$
• Under $H_a:MST>MSE$
• $F+\frac{MST}{MSE}\sim F_{(k-1,n-k)}$
• $H_0$