STA260 Lecture 14

• $=\frac{1}{\theta }\frac{\mathrm{\Gamma }(n-1)}{\mathrm{\Gamma }(n)\mathrm{\Gamma }(1)}{x}^{n-1}(1-x{)}^{1-1}$ for $0<x<1$
• Transformation:
  • $\left|J\right|=\left|\frac{\text{differential}x}{\text{differential}y}\right|=\theta$
  • $Y=\frac{x}{\theta }$
  • $x=\theta y$
  • $0<y<1$
• So $f_Y(y)=f_X(g^{-1}(y))\left|J\right|$
  • $f_{X_{(n)}}=\left(\frac{n}{{\theta }^n}\right)x^{n-1}$
  • $g^{-1}(y)=\theta y$
  • $f_y(y)=f_X(g^{-1}(y))\left|J\right|$
  • $f_y(y)=\frac{n}{{\theta }^n}(g^{-1}(y){)}^{n-1}\theta$
  • $f_y(y)=\frac{n}{{\theta }^n}(\theta y{)}^{n-1}\theta$
  • $f_y(y)=\frac{n(\theta y{)}^{n-1}}{{\theta }^n}$
  • $f_y(y)=n(\theta y{)}^{n-1-n}\theta$
  • $f_y(y)=n(\theta y{)}^{-1}\theta$
  • $f_y(y)=(ny{)}^{n-1}0<y<1$
  • Beta dist has pdf: $\frac{\mathrm{\Gamma }(\alpha +\beta )}{\mathrm{\Gamma }(\alpha )\mathrm{\Gamma }(\beta )}{x}^{\alpha -1}(1-x{)}^{\beta -1}$ for $0<x<1$
  • $f_y(y)=(ny{)}^{n-1}(1-y{)}^{1-1}0<y<1$
  • $\alpha =n$
  • $\beta =1$
  • #tk practice this derivation from scratch
  • You can show that $\frac{\mathrm{\Gamma }(n+1)}{\mathrm{\Gamma }(n)\mathrm{\Gamma }(1)}=n$
  • $f_Y(y)=\frac{\mathrm{\Gamma }(n+1)}{\mathrm{\Gamma }(n)\mathrm{\Gamma }(1)}n{y}^{n-1}(1-y{)}^{1-1}0<y<1$
  • $Y\sim \mathrm{Beta}(\alpha =n,\beta =1)$
  • Since $X\sim \theta Y$
  • $X\sim \theta \mathrm{Beta}(\alpha =n,\beta =1)$
  • There is a general form:
    • $X\sim U(0,1)$
    • $Y=X^{\frac{1}{\alpha }}$
    • $Y\sim \mathrm{Beta}(\alpha ,1)$
      • #tk make flashcard of this shortcut.
• Mean Squared Error:
  • $\mathrm{MSE}[\hat{\theta }]=\mathrm{E}[(\hat{\theta }-\theta {)}^2]$
  • $\mathrm{MSE}[\hat{\theta }]=\mathrm{Var}(\hat{\theta })+b(\hat{\theta }{)}^2$
    • This is the variance bias tradeoff.
    • You can have a low variance estimator with a high bias, or a low bias estimator with a high variance.
  • Universally used to measure goodness of fit of an estimator.
  • Example:
    • Let $X_1,\dots ,X_n$ be a random sample from $N(\mu ,2)$
    • Show that $\overline{X}$ is an unbiased estimator of $\mu$
      • We already know $\mathrm{E}[\overline{X}]=\mu$
        • $\mathrm{E}[\overline{X}]=\mathrm{E}\left[\frac{1}{n}\sum _{i=1}^n{X}_i\right]$
        • $=\frac{1}{n}\sum _{i=1}^n\mathrm{E}[X_i]$
        • $=\frac{1}{n}\sum _{i=1}^n\mu$
        • $=\mu$
      • By CLT $\overline{X}$ is normally distributed with mean $\mu$ and variance $\frac{2}{n}$
    • Show ${\overline{X}}^2$ is a biased estimator of ${\mu }^2$
      • $\mathrm{E}[{\overline{X}}^2]\ne {\mu }^2$
      • or
      • $\mathrm{E}[{\overline{X}}^2]={\mu }^2+b({\overline{X}}^2)$
        • $\mathrm{Var}(\overline{X})=\mathrm{E}[{\overline{X}}^2]+\mathrm{E}[\overline{X}{]}^2$
        • $\mathrm{E}[{\overline{X}}^2]=\mathrm{E}[\overline{X}{]}^2+\mathrm{Var}(\overline{X})$
      • $\mathrm{Var}(\overline{X})+\mathrm{E}[\overline{X}{]}^2={\mu }^2+b({\overline{X}}^2)$
        • By CLT we can get the variance.
      • $\frac{2}{n}+{\mu }^2={\mu }^2+b({\overline{X}}^2)$
      • $\frac{2}{n}=b({\overline{X}}^2)$
      • As $n\to \mathrm{\infty }$ our bias shrinks.
    • Find the $\mathrm{MSE}[{\overline{X}}^2]$
      • $\mathrm{MSE}[\hat{\theta }]=\mathrm{Var}(\hat{\theta })+b(\hat{\theta }{)}^2$
      • $\mathrm{Var}({\overline{X}}^2)=\mathrm{E}[{\overline{X}}^4]+\mathrm{E}[{\overline{X}}^2{]}^2$
      • $\mathrm{E}[{\overline{X}}^2]=\mathrm{Var}(\overline{X})+\mathrm{E}[\overline{X}{]}^2$
      • $\mathrm{Var}({\overline{X}}^2)=\mathrm{E}[{\overline{X}}^4]+[\mathrm{Var}(\overline{X})+\mathrm{E}[\overline{X}{]}^2{]}^2$
      • $\mathrm{Var}({\overline{X}}^2)=\mathrm{E}[{\overline{X}}^4]+\mathrm{Var}(\overline{X}{)}^2+\mathrm{Var}(\overline{X})\mathrm{E}[\overline{X}{]}^2+\mathrm{E}[\overline{X}{]}^4$
      • $\hat{\theta }={\overline{X}}^2$
      • $\theta ={\mu }^2$
      • $\mathrm{MSE}[\hat{\theta }]=\mathrm{E}[({\overline{X}}^2-{\mu }^2{)}^2]=\mathrm{Var}({\overline{X}}^2)+b({\overline{X}}^2{)}^2$
      • We know $b({\overline{X}}^2{)}^2={\left(\frac{2}{n}\right)}^2=\frac{4}{n^2}$
      • We can add 0
        • #tk Remember this trick and context on how to use this.
        • #tk an old final question
      • $\mathrm{Var}({\overline{X}}^2)=\mathrm{Var}\left({\left[\frac{(\overline{X}-\mu +\mu )\frac{\sigma }{\sqrt{n}}}{\frac{\sigma }{\sqrt{n}}}\right]}^2\right)$
      • $\mathrm{Var}({\overline{X}}^2)=\mathrm{Var}\left({\left[\frac{(\overline{X}-\mu +\mu )}{\frac{\sigma }{\sqrt{n}}}\right]}^2\frac{{\sigma }^2}{n}\right)$
      • $\mathrm{Var}({\overline{X}}^2)=\frac{{\sigma }^4}{n^2}\mathrm{Var}\left({\left[\frac{(\overline{X}-\mu +\mu )}{\frac{\sigma }{\sqrt{n}}}\right]}^2\right)$
      • $\mathrm{Var}({\overline{X}}^2)=\frac{{\sigma }^4}{n^2}\mathrm{Var}\left({[\frac{\overline{X}-\mu }{\frac{\sigma }{\sqrt{n}}}+\frac{\mu }{\frac{\sigma }{\sqrt{n}}}]}^2\right)$
      • $\mathrm{Var}({\overline{X}}^2)=\frac{{\sigma }^4}{n^2}\mathrm{Var}\left({[Z^2+\frac{\mu }{\frac{\sigma }{\sqrt{n}}}]}^2\right)$
      • $Z=\frac{\overline{X}-\mu }{\frac{\sigma }{\sqrt{n}}}$
      • Since $Z^2\sim {\chi }_{(1)}^2$
      • $\mathrm{Var}({\overline{X}}^2)=\frac{{\sigma }^4}{n^2}\mathrm{Var}\left({[Z+\frac{\mu }{\frac{\sigma }{\sqrt{n}}}]}^2\right)$
      • Variance of a constant is 0
      • $\mathrm{Var}({\overline{X}}^2)=\frac{{\sigma }^4}{n^2}\mathrm{Var}(Z^2+\frac{2\mu Z}{\frac{\sigma }{\sqrt{n}}}+\frac{{\mu }^2}{\frac{{\sigma }^2}{n}})$
        • No covariance term since independent
      • $\mathrm{Var}({\overline{X}}^2)=\frac{{\sigma }^4}{n^2}[\mathrm{Var}(Z^2)+{\left(\frac{2\mu }{\frac{\sigma }{\sqrt{n}}}\right)}^2\mathrm{Var}(Z)+{\cancel{\mathrm{Var}{\left(\frac{\mu }{\frac{\sigma }{\sqrt{n}}}\right)}^2}}^0]$
      • $\mathrm{Var}({\overline{X}}^2)=\frac{{\sigma }^4}{n^2}[\mathrm{Var}(Z^2)+{\left(\frac{2\mu }{\frac{\sigma }{\sqrt{n}}}\right)}^2\mathrm{Var}(Z)]$
      • $\mathrm{Var}({\chi }_{(1)}^2)=2\nu =2$
      • $\mathrm{Var}({\overline{X}}^2)=\frac{{\sigma }^4}{n^2}[2+{\left(\frac{2\mu }{\frac{\sigma }{\sqrt{n}}}\right)}^2\mathrm{Var}(Z)]$
      • $\mathrm{Var}(Z)=\mathrm{Var}(N(0,1))=1$
      • $\mathrm{Var}({\overline{X}}^2)=\frac{{\sigma }^4}{n^2}[2+{\left(\frac{2\mu }{\frac{\sigma }{\sqrt{n}}}\right)}^2]$
      • $\mathrm{Var}({\overline{X}}^2)=\frac{{\sigma }^4}{n^2}[2+{\left(\frac{2\mu }{\frac{\sigma }{\sqrt{n}}}\right)}^2]=4\frac{{\mu }^2{\sigma }^2}{n}+2\frac{{\sigma }^4}{n^2}$
      • $\mathrm{Var}({\overline{X}}^2)=4\frac{2{\mu }^2}{n}+2\frac{4}{n^2}$
      • $\mathrm{Var}({\overline{X}}^2)=\frac{8{\mu }^2}{n}+\frac{8}{n^2}$
      • $\mathrm{MSE}[{\overline{X}}^2]=\mathrm{Var}({\overline{X}}^2)+b({\overline{X}}^2{)}^2$
      • $\mathrm{MSE}[{\overline{X}}^2]=[\frac{8{\mu }^2}{n}+\frac{8}{n^2}]+\frac{4}{n^2}$
      • $=[\frac{8{\mu }^2}{n}+\frac{8}{n^2}]+\frac{4}{n^2}=\frac{8{\mu }^{2}n+12}{n^2}m$