STA260 Lecture 13

Completed Notes Status

Central objective: To evaluate and compare the performance of statistical estimators using efficiency, variance, and Mean Squared Error (MSE), with applied examples involving order statistics.
Key concepts:
- Estimator Efficiency: Focuses on how well estimators utilise data, measured by variance. The most efficient unbiased estimator is the one with the minimum variance.
- Mean Squared Error: Resolves the trade-off between bias and variance for biased estimators. The formula is $MSE (\hat{θ}) = Var (\hat{θ}) + (Bias (\hat{θ}))^{2}$ .
- Order Statistics: Represents the ordered values of a sample ( $X_{(1)} \leq \dots \leq X_{(n)}$ ). Their probability distributions are derived using CDF techniques rather than MGFs, specifically applying complement rules for minimums and direct products for maximums.
Connections:
- Estimator Efficiency and Mean Squared Error are directly connected as criteria for estimator selection. Unbiased estimators are compared directly through variance (efficiency), whereas biased estimators require minimizing MSE.

#tk: practice this derivation
- Answer: By linearity of expectation, $E [\tilde{X}] = E [\bar{X} + \frac{1}{n}] = E [\bar{X}] + \frac{1}{n} = μ + \frac{1}{n}$ .
#tk: ask in office hours if this is valid. Why or why not. Why do we use the CDF instead of MGF? Why was I wrong?
- Answer: Assuming $X_{1}$ is the minimum WLOG is conceptually incorrect because it ignores the joint probability distribution of all possible orderings. The CDF method is required because the event $min (X_{i}) > x$ cleanly breaks down into the intersection of independent events ( $X_{1} > x \cap X_{2} > x \cap \dots \cap X_{n} > x$ ). MGFs are suited for sums of random variables, not minimums or maximums.
#tk: is this the right derivation?
- Answer: Yes. The minimum ${\hat{θ}}_{4} \sim Exp (θ / 3)$ . Because the variance of an exponential distribution is the square of its mean parameter, $Var ({\hat{θ}}_{4}) = (θ / 3)^{2} = θ^{2} / 9$ .
#tk: maybe wrong?
- Answer: The derivation is correct. Using the property $Γ (n + 1) = n!$ for integers, the constant for $Beta (n, 1)$ evaluates exactly to $n$ , matching the density function constant derived earlier.

Remember/Understand:
- Define estimator efficiency in the context of unbiased estimators.
- State the mathematical relationship between Mean Squared Error (MSE), variance, and bias.
- What does $X_{(n)}$ denote in a set of order statistics?
Apply/Analyze:
- Given $X_{i} \sim Exp (θ)$ , derive the CDF of the minimum order statistic $X_{(1)}$ .
- Calculate the expected value of $\hat{θ} = \frac{X_{1} + 2 X_{2}}{3}$ assuming $X_{i}$ are iid with mean $θ$ .
- Compare the efficiency of ${\hat{θ}}_{1} = X_{1}$ and ${\hat{θ}}_{2} = \frac{X_{1} + X_{2}}{2}$ given identical variances $σ^{2}$ .
Evaluate/Create:
- Justify why the CDF method is fundamentally necessary over the Moment Generating Function (MGF) method when deriving the distribution of $X_{(1)}$ or $X_{(n)}$ .

Order Statistics:
- Why it's challenging: Transitioning from standard marginal distributions to the joint distributions of ordered sets requires conceptualizing the intersection of multiple inequalities (e.g., all variables being less than $x$ for the maximum).
- Study strategy: Practice setting up the CDF limits manually. Draw number lines to visualize the probability that the maximum or minimum falls above/below a threshold $x$ .
Mean Squared Error:
- Why it's challenging: Balancing the bias-variance trade-off mathematically can be counterintuitive when an estimator is biased but heavily preferred due to exceptionally low variance.
- Study strategy: Run simulations in R/Python plotting the MSE curves of biased vs. unbiased estimators to see visually when the biased estimator performs better.

Immediate review actions:
- Review all [COMPLETED] insertions and verify with course materials
- Address all #tk items (if any)
- Re-read Lecture Summary and confirm key links
Practice and application:
- Answer Remember/Understand questions
- Answer Apply/Analyze questions
- Attempt Evaluate/Create questions
Deep dive study:
- Focus on Challenging Concepts (above)
- Extend atomic notes only where the new lecture adds content
Verification and integration:
- Cross-reference notes with textbook/slides
- Identify remaining gaps
- Link this lecture to prior/future lecture notes (lecture notes only; no MOCs)