• ↑
  • STA260

STA260 Lecture 10

STA260 Lecture 10 Raw
STA260 Lecture 10 Flashcards

• Completed Notes Status

  • Completed insertions: 8
  • Ambiguities left unresolved: 2

• Lecture Summary

  • Central objective: Connect Fisher Information (curvature/precision of estimation) with core sampling distributions used for inference (Chi-square Distribution, t Distribution, F Distribution).
  • Key concepts:
    • Fisher Information
      • Fisher information for a parameter $\theta$ can be written as $\mathcal{I}(\theta )=\mathrm{E}\left[{(\frac{\partial }{\partial \theta }\log f_X(X;\theta ))}^2\right]$ or equivalently $\mathcal{I}(\theta )=-\mathrm{E}[\frac{{\partial }^2}{\partial {\theta }^2}\log f_X(X;\theta )]$ (under regularity).
      • High Fisher information corresponds to a sharply peaked Likelihood Function around its maximizer, which supports more precise estimation.
    • Exponential Distribution (Scale Parameter)
      • For $X\sim \mathrm{Exp}(\lambda )$ with $f(x;\lambda )=\frac{1}{\lambda }{e}^{-x/\lambda }$, the Fisher information in $\lambda$ is $\mathcal{I}(\lambda )=\frac{1}{{\lambda }^2}$ (both score-squared and negative-Hessian methods).
    • Sample Variance
      • The "biased" sample variance $S_B^2=\frac{1}{n}\sum _{i=1}^n(X_i-\overline{X}{)}^2$ equals $\frac{n-1}{n}{S}_u^2$, where $S_u^2=\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2$ is the unbiased sample variance.
      • Using $W=\frac{(n-1)S_u^2}{{\sigma }^2}\sim {\chi }_{(n-1)}^2$ (normal sample case) and $\mathrm{Var}({\chi }_{\nu }^2)=2\nu$, you can compute $\mathrm{E}[S_B^2]=\frac{n-1}{n}{\sigma }^2$ and $\mathrm{Var}(S_B^2)=\frac{2{\sigma }^4(n-1)}{n^2}$.
  • Connections:
    • The "precision" idea in Fisher Information (steep likelihood) matches the inferential idea that estimators with smaller variance are more concentrated, and the review distributions (Chi-square Distribution, t Distribution, F Distribution) are the standard tools for quantifying that concentration in normal-model inference.

• TK Resolutions

  • #tk: "Do the second Fisher information method (negative expected second derivative) for $X\sim \mathrm{Exp}(\lambda )$."
    • Answer:
      • $\log f(x;\lambda )=-\log \lambda -\frac{x}{\lambda }$
      • $\frac{{\partial }^2}{\partial {\lambda }^2}\log f(x;\lambda )=\frac{1}{{\lambda }^2}-\frac{2x}{{\lambda }^3}$
      • $\mathcal{I}(\lambda )=-\mathrm{E}[\frac{1}{{\lambda }^2}-\frac{2X}{{\lambda }^3}]=\frac{1}{{\lambda }^2}$
  • #tk: "Square $-\frac{1}{\lambda }+\frac{x}{{\lambda }^2}$ (exercise)."
    • Answer:
      • ${(-\frac{1}{\lambda }+\frac{x}{{\lambda }^2})}^2=\frac{1}{{\lambda }^2}-\frac{2x}{{\lambda }^3}+\frac{x^2}{{\lambda }^4}$
  • #tk: "Useful information for test."
    • If not answerable: Need the instructor's test blueprint (topics/chapters, allowed formulas, and which distributional results are in-scope).

• Practice Questions

  • Remember/Understand:
    • State two equivalent formulas for Fisher Information $\mathcal{I}(\theta )$ and explain (in words) what "high information" suggests about the Likelihood Function.
    • For $X\sim \mathrm{Exp}(\lambda )$ (scale), write $\log f(x;\lambda )$ and the Score Function $\frac{\partial }{\partial \lambda }\log f(x;\lambda )$.
    • Define Biased Sample Variance $S_B^2$ and Unbiased Sample Variance $S_u^2$ and state the relationship between them.
  • Apply/Analyze:
    • Compute $\mathcal{I}(\lambda )$ for $X\sim \mathrm{Exp}(\lambda )$ using the negative expected second derivative method.
    • If $W\sim {\chi }_{(\nu )}^2$, compute $\mathrm{E}[W]$ and $\mathrm{Var}(W)$ and use them to derive $\mathrm{Var}(S_u^2)$ when $\frac{(n-1)S_u^2}{{\sigma }^2}\sim {\chi }_{(n-1)}^2$.
    • Let $X_1,\dots ,X_{25}\stackrel{iid}{\sim }N(0,1)$; find the distributions of $\sum _{i=1}^{4}2X_i$, $\sum _{i=5}^{16}{X}_i^2$, and $\frac{X_{20}}{\sqrt{\left(\sum _{i=2}^{17}{X}_i^2\right)/16}}$.
  • Evaluate/Create:
    • Build a "distribution-recognition checklist" that decides when a ratio becomes t Distribution vs F Distribution, including the independence conditions you must verify.

• Challenging Concepts

  • Fisher Information:
    • Why it's challenging: The two formulas for $\mathcal{I}(\theta )$ look unrelated until you remember the regularity conditions and how expectations interact with derivatives.
    • Study strategy: Practise deriving $\mathcal{I}(\theta )$ both ways for 2–3 models (e.g., exponential scale, normal mean with known variance) and explicitly write each derivative step.
  • Distribution engineering:
    • Why it's challenging: Many expressions "almost" look like $t$ or $F$, but small details (df scaling, independence, whether a term appears inside the sum) completely change the result.
    • Study strategy: For every expression, rewrite it into the canonical form: $Z/\sqrt{U/\nu }$ for $t$ and $(U/m)/(V/n)$ for $F$, then check independence and scaling constants.

• Action Plan

  • Immediate review actions:
    • Review all [COMPLETED] insertions and verify with course materials
    • Address all #tk items (if any)
    • Re-read Lecture Summary and confirm key links
  • Practice and application:
    • Answer Remember/Understand questions
    • Answer Apply/Analyze questions
    • Attempt Evaluate/Create questions
  • Deep dive study:
    • Focus on Challenging Concepts (above)
    • Extend atomic notes only where the new lecture adds content
  • Verification and integration:
    • Cross-reference notes with textbook/slides
    • Identify remaining gaps
    • Link this lecture to prior/future lecture notes (lecture notes only; no MOCs)

• Footnotes