STA260 Lecture 18

• Common pivotal quantities
  • This is where the distribution of the statistic does not depend on any unknown parameters.
  • So like:
    • $X_1,\dots ,X_n\sim N(\mu ,{\sigma }^2)$ and ${\sigma }^2$ is known.
      • $Z=\frac{\overline{X}-\mu }{\frac{\sigma }{\sqrt{n}}}\sim N(0,1)$
        • $\sigma$ is known
    • $X_1,\dots ,X_n\sim N(\mu ,{\sigma }^2)$ and ${\sigma }^2$ is unknown.
      • $\frac{\overline{X}-\mu }{\frac{S}{\sqrt{n}}}\sim t_{(n-1)}$
    • $X_1,..,X_n\sim \mathrm{Exp}(\beta )$
      • $\frac{\sum _{i=1}^n{x}_i}{\beta }\sim {\chi }_{(2n)}^2$
    • $X_1,\dots ,X_n\sim N(\mu ,{\sigma }^2)$ and $\mu$ is unknown.
      • $\frac{(n-1)S^2}{{\sigma }^2}\sim {\chi }_{(n-1)}^2$
    • $X\sim \mathrm{Bernoulli}(p)$
      • $\sum _{i=1}^n{X}_i\sim \mathrm{Bin}(n,p)$
      • $\hat{p}=\overline{x}$
      • $Z=\frac{\hat{p}-p}{\sqrt{\frac{p(1-p)}{n}}}\sim N(0,1)$
    • $X_1,\dots ,X_n\sim \mathrm{Poisson}(\lambda )$
      • $2\sum _{i=1}^n{X}_i\sim {\chi }_{\left(2\sum _{i=1}^n{X}_i\right)}^2$
• 1
  • For each distribution below.
  • a:
    • Find the Pivotal Quantity for the stated parameter
  • b:
    • Use it to get a $100(1-\alpha )\mathrm{\%}$ CI for it.
  • 1
    • $X_1,\dots ,X_n\sim N(\mu ,{\sigma }^2)$
    • $\mu$ is unknown. ${\sigma }^2$ is known.
    • We want a Pivotal Quantity for $\mu$.
    • $Z=\frac{\overline{X}-\mu }{\frac{\sigma }{\sqrt{n}}}\sim N(0,1)$
    • $P(-Z_{\frac{\alpha }{2}}\le \frac{\overline{X}-\mu }{\frac{\sigma }{\sqrt{n}}}\le Z_{\frac{\alpha }{2}})=1-\alpha$
    • $P(-Z_{\frac{\alpha }{2}}\frac{\sigma }{\sqrt{n}}\le \overline{X}-\mu \le Z_{\frac{\alpha }{2}}\frac{\sigma }{\sqrt{n}})=1-\alpha$
    • $P(-Z_{\frac{\alpha }{2}}\frac{\sigma }{\sqrt{n}}-\overline{X}\le -\mu \le Z_{\frac{\alpha }{2}}\frac{\sigma }{\sqrt{n}}-\overline{X})=1-\alpha$
    • $P(Z_{\frac{\alpha }{2}}\frac{\sigma }{\sqrt{n}}+\overline{X}\ge \mu \ge -Z_{\frac{\alpha }{2}}\frac{\sigma }{\sqrt{n}}+\overline{X})=1-\alpha$
    • $P(-Z_{\frac{\alpha }{2}}\frac{\sigma }{\sqrt{n}}+\overline{X}\le \mu \le Z_{\frac{\alpha }{2}}\frac{\sigma }{\sqrt{n}}+\overline{X})=1-\alpha$
    • So the CI is $[\overline{X}-Z_{\frac{\alpha }{2}}\frac{\sigma }{\sqrt{n}},\overline{X}+Z_{\frac{\alpha }{2}}\frac{\sigma }{\sqrt{n}}]$
  • 2
    • $X_1,\dots ,X_n\sim N(\mu ,{\sigma }^2)$ where $\mu$ is known and ${\sigma }^2$ is unknown.
    • We want a Pivotal Quantity for ${\sigma }^2$.
    • $Z_i=\frac{X_i-\mu }{\sigma }$
    • $Z_i^2=\frac{(X_i-\mu {)}^2}{{\sigma }^2}\sim {\chi }_{(1)}^2$.
    • Since we have a sample.
    • $\sum _{i=1}^n{Z}_i^2=\sum _{i=1}^n{\left(\frac{X_i-\mu }{\sigma }\right)}^2\sim {\chi }_{(n)}^2$
    • $P({\chi }_{(n;\frac{\alpha }{2})}^2\le \sum _{i=1}^n{\left(\frac{X_i-\mu }{\sigma }\right)}^2\le {\chi }_{(n;1-\frac{\alpha }{2})}^2)=1-\alpha$
    • $P(\frac{1}{{\chi }_{(n;\frac{\alpha }{2})}^2}\ge \frac{{\sigma }^2}{\sum _{i=1}^n(X_i-\mu {)}^2}\ge \frac{1}{{\chi }_{(n;1-\frac{\alpha }{2})}^2})=1-\alpha$
    • $P(\frac{\sum _{i=1}^n(X_i-\mu {)}^2}{{\chi }_{(n;1-\frac{\alpha }{2})}^2}\le {\sigma }^2\le \frac{\sum _{i=1}^n(X_i-\mu {)}^2}{{\chi }_{(n;\frac{\alpha }{2})}^2})=1-\alpha$
    • So the CI is $[\frac{\sum _{i=1}^n(X_i-\mu {)}^2}{{\chi }_{(n;1-\frac{\alpha }{2})}^2},\frac{\sum _{i=1}^n(X_i-\mu {)}^2}{{\chi }_{(n;\frac{\alpha }{2})}^2}]$
  • 3
    • $X_1,\dots ,X_n\sim N(\mu ,{\sigma }^2)$
    • $\mu ,{\sigma }^2$ both unknown.
    • We want a Pivotal Quantity for ${\sigma }^2$.
    • $\frac{(n-1)S^2}{{\sigma }^2}\sim {\chi }_{(n-1)}^2$
    • $P({\chi }_{(n;\frac{\alpha }{2})}^2\le \frac{(n-1)S^2}{{\sigma }^2}\le {\chi }_{(n;1-\frac{\alpha }{2})}^2)=1-\alpha$
    • $P(\frac{1}{{\chi }_{(n;\frac{\alpha }{2})}^2}\ge \frac{{\sigma }^2}{(n-1)S^2}\ge \frac{1}{{\chi }_{(n;1-\frac{\alpha }{2})}^2})=1-\alpha$
    • $P(\frac{(n-1)S^2}{{\chi }_{(n;1-\frac{\alpha }{2})}^2}\le {\sigma }^2\le \frac{(n-1)S^2}{{\chi }_{(n;\frac{\alpha }{2})}^2})=1-\alpha$
    • So the CI is $[\frac{(n-1)S^2}{{\chi }_{(n;1-\frac{\alpha }{2})}^2},\frac{(n-1)S^2}{{\chi }_{(n;\frac{\alpha }{2})}^2}]$
  • 4
    • $X_1,\dots ,X_n\sim N(\mu ,{\sigma }^2)$
    • $\mu ,{\sigma }^2$ both unknown.
    • We want a Pivotal Quantity for $\mu$.
    • $\frac{\overline{X}-\mu }{\frac{S}{\sqrt{n}}}\sim t_{(n-1)}$
    • $P(t_{(n-1;\frac{\alpha }{2})}\le \frac{\overline{X}-\mu }{\frac{S}{\sqrt{n}}}\le t_{(n-1;1-\frac{\alpha }{2})})=1-\alpha$
    • $P(t_{(n-1;\frac{\alpha }{2})}\frac{S}{\sqrt{n}}\le \overline{X}-\mu \le t_{(n-1;1-\frac{\alpha }{2})}\frac{S}{\sqrt{n}})=1-\alpha$
    • $P(t_{(n-1;\frac{\alpha }{2})}\frac{S}{\sqrt{n}}-\overline{X}\le -\mu \le t_{(n-1;1-\frac{\alpha }{2})}\frac{S}{\sqrt{n}}-\overline{X})=1-\alpha$
    • $P(t_{(n-1;1-\frac{\alpha }{2})}\frac{S}{\sqrt{n}}+\overline{X}\ge \mu \ge t_{(n-1;\frac{\alpha }{2})}\frac{S}{\sqrt{n}}-\overline{X})=1-\alpha$
    • $P(\overline{X}-t_{(n-1;\frac{\alpha }{2})}\frac{S}{\sqrt{n}}\le \mu \le \overline{X}+t_{(n-1;\frac{\alpha }{2})}\frac{S}{\sqrt{n}})=1-\alpha$
    • #tk
  • 5
    • $\beta$ is unknown
    • $\frac{2n\overline{x}}{\beta }=\frac{\sum _{i=1}^n{x}_i}{\beta }\sim {\chi }_{(2n)}^2$
    • $P({\chi }_{(2n;\frac{\alpha }{2})}^2\le \frac{2n\overline{x}}{\beta }\le {\chi }_{(2n;1-\frac{\alpha }{2})}^2)=1-\alpha$
    • $P(\frac{1}{{\chi }_{(2n;\frac{\alpha }{2})}^2}\ge \frac{\beta }{2n\overline{x}}\ge \frac{1}{{\chi }_{(2n;1-\frac{\alpha }{2})}^2})=1-\alpha$
    • $P(\frac{2n\overline{x}}{{\chi }_{(2n;1-\frac{\alpha }{2})}^2}\le \beta \le \frac{2n\overline{x}}{{\chi }_{(2n;\frac{\alpha }{2})}^2})=1-\alpha$
    • So the CI is $[\frac{2n\overline{x}}{{\chi }_{(2n;1-\frac{\alpha }{2})}^2},\frac{2n\overline{x}}{{\chi }_{(2n;\frac{\alpha }{2})}^2}]$
• 2
  • Let $X$ have $P_X(x|\theta )=\{\begin{array}{ll}\frac{1-\theta }{2}& x=1\\ \frac{1-\theta }{2}& x=2\\ \theta & x=3\\ 0& \text{Otherwise}\end{array}$ where $0<\theta <1$
  • 3 independent observations were sampled.
  • $X_1=3,X_2=3,X_3=3$
  • a: $\theta$ has a uniform prior on $0,1$. Find it's posterior distribution given the data.
    • $f_X(\theta |x)\propto f_{X|\mathrm{\Theta }}(x|\theta )f_{\mathrm{\Theta }}(\theta )$
    • Likelihood:
      • $f_{X|\mathrm{\Theta }}(x|\theta )=\prod _{i=1}^3{P}_X(x_i|\theta )$
      • $f_{X|\mathrm{\Theta }}(x|\theta )=f(X_1|\theta )f(X_2|\theta )f(X_3|\theta )$
      • $f_{X|\mathrm{\Theta }}(x|\theta )={\theta }^3$
    • Prior:
      • $f_{\mathrm{\Theta }}(\theta )=\{\begin{array}{ll}\frac{1}{1-0}=1& 0<\theta <1\\ 0& \text{Otherwise}\end{array}$
    • $f(\theta |x)\propto {\theta }^{3}1$
    • $f(\theta |x)\propto {\theta }^3$
    • $f(\theta |x)\propto {\theta }^{4-1}(1-\theta {)}^{1-1}$
    • $\alpha =4$
    • $\beta =1$
    • $f(\theta |x)\sim \mathrm{Beta}(4,1)$
    • $f(\theta |x)\propto {\theta }^{4-1}(1-\theta {)}^{1-1}$
    • $f(\theta |x)\propto \frac{\mathrm{\Gamma }(5)}{\mathrm{\Gamma }(4)\mathrm{\Gamma }(1)}{\theta }^{4-1}(1-\theta {)}^{1-1}$
    • $f(\theta |x)=\frac{\mathrm{\Gamma }(5)}{\mathrm{\Gamma }(4)\mathrm{\Gamma }(1)}{\theta }^{4-1}(1-\theta {)}^{1-1}$ for $0<\theta <1$ and $0$ otherwise.
    • $f(\theta |x)=\frac{(4)!}{(4)!(1)!}{\theta }^{4-1}(1-\theta {)}^{1-1}$ for $0<\theta <1$ and $0$ otherwise.
    • $f(\theta |x)=4{\theta }^{4-1}(1-\theta {)}^{1-1}$ for $0<\theta <1$ and $0$ otherwise.
    • #tk
    • Or
    • $f(\theta |x)\propto {\theta }^3$
    • $f(\theta |x)=k{\theta }^3$
    • ${\int }_0^{1}k{\theta }^3\text{differential}x=1$
    • $k{\int }_0^1{\theta }^3\text{differential}x=1$
    • $\left[k\frac{{\theta }^4}{4}\right]{}_0^1=1$
    • $\frac{k}{4}=1$
    • $k=4$
  • b: Find the posterior mean
    • We found it's a beta.
    • So $\mathrm{E}[X]=\frac{\alpha }{\alpha +\beta }$
    • So $\mathrm{E}[\theta ]=\frac{4}{4+1}=\frac{4}{5}$
    • If we couldn't see this.
    • We know $f(\theta |x)=4{\theta }^3$
    • Then $\mathrm{E}[\theta ]={\int }_0^1\theta 4{\theta }^3\text{differential}\theta$
    • $4{\int }_0^1{\theta }^4\text{differential}\theta$
    • $4\left[\frac{1}{5}{\theta }^5\right]{}_0^1$
    • $\frac{4}{5}$
• 3
  • Inverse Gamma
  • $f(x)=\frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}{x}^{-(\alpha +1)}{e}^{-\frac{\beta }{x}}$
  • $\mathrm{E}[X]=\frac{\beta }{\alpha -1}$
  • $\mathrm{Var}(X)=\frac{{\beta }^2}{(\alpha -1{)}^2(\alpha -2)}$
  • Let $X_1,\dots ,X_n\sim \mathrm{Exp}(\theta )$
  • The prior is $\mathrm{InvGamma}(1,2)$
  • Find the Posterior Distribution
  • $f(\theta |X)\propto f(X|\theta )f(\theta )$
  • $f(X|\theta )=\prod _{i=1}^n{f}_X(X_i|\theta )$
  • $f(X|\theta )=\frac{1}{{\theta }^n}{e}^{-{\theta }^{-1}\sum _{i=1}^n{x}_i}$
  • $f(\theta |X)\propto f(X|\theta )f(\theta )$
  • $f(\theta |X)\propto {\theta }^{-n}{e}^{-{\theta }^{-1}\sum _{i=1}^n{x}_i}\frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}{\theta }^{-(\alpha +1)}{e}^{-\frac{\beta }{\theta }}$
  • $f(\theta |X)\propto {\theta }^{-n}{e}^{-{\theta }^{-1}\sum _{i=1}^n{x}_i}2{\theta }^{-2}{e}^{-\frac{2}{\theta }}$
  • $f(\theta |X)\propto 2{\theta }^{-n-2}{e}^{-{\theta }^{-1}\sum _{i=1}^n{x}_i-2{\theta }^{-1}}$
  • $f(\theta |X)\propto 2{\theta }^{-n-2}{e}^{-3{\theta }^{-1}\sum _{i=1}^n{x}_i}$
  • $f(\theta |X)\propto {\theta }^{-n-2}{e}^{-3{\theta }^{-1}\sum _{i=1}^n{x}_i}$
    • #tk
  • $f(\theta |X)\propto {\theta }^{-(n+2)}{e}^{\frac{-(\sum _{i=1}^n{x}_i+2)}{\theta }}$
    • ${\theta }^{-(n+2)}\sim x^{-(\alpha +1)}$
    • $e^{\frac{-(\sum _{i=1}^n{x}_i+2)}{\theta }}\sim e^{-\frac{\beta }{x}}$
    • $\alpha +1=n+2$
    • $\alpha =n+1$
    • $\beta =\sum _{i=1}^n{x}_i+2$
  • b:
    • Find the posterior mean
    • $\mathrm{E}[\theta ]=\frac{\beta }{\alpha -1}$
    • $\mathrm{E}[\theta ]=\frac{\sum _{i=1}^n{x}_i+2}{n+1-1}$
    • $\mathrm{E}[\theta ]=\frac{\sum _{i=1}^n{x}_i+2}{n}$