Class Notes, Chapter 9, 9.3 - 9.5, 9.14
Estimation (of population parameters)
Reading: 9.3-9.5, 9.14
Parent processes, parameters
Sampling schemes, statistics, sampling distributions
Today: Estimation - from sample to parameter
The difference between an estimator and an estimate: the formula to get the number vs. the number.
- (The "hat" is used to indicate an estimator)
The estimator for a population parameter
- doesn't have to be the
analog of the parameter.
- Example: the sample median is an estimator of the population
estimators are better than others.
- The sample mean is a good estimator of the population mean.
- Twice the sample mode is not a good estimator of the population
Properties of estimators
- is the "formula" used
with the sample data to get a number;
- is the actual population value being
(A) (the formula for the sample mean) is an unbiased
estimator of for
any parent distribution.
- Use the Algebra of
(B) is an unbiased estimator of for any parent
- Here is the
(C) is not an unbiased estimator of sigma
Bias is the difference between the value of a parameter
and the expected value of an estimator.
- Here is
(2) Efficient, i.e. small variance
for the sampling distribution
- We would like (the number derived from the sample using
) not only to on
average equal , but
also to be close to
every time we take a sample.
An unbiased estimator that has less variance than
any other unbiased estimator is called the Minimum Variance Unbiased
Estimator, MVUE. Such an estimator is especially good.
(A) is MVUE for for a Normal parent distribution
estimator may be biased. We can still compare 2 estimators to see which is
better (more efficient, sampling distribution has smaller variance); in
that case we use relative efficiency.
Consistent, i.e. as n, the sample size, gets larger, the
estimator gets better and better.
(A) is a consistent estimator of .
We look for estimators that are unbiased, efficient, and consistent.
Methods for finding estimators
estimator for a population parameter may not be the sample analog of
the population value, and we would like to have general methods for finding
estimators that are unbiased, efficient, consistent, etc. There are
several such methods commonly used, for example:
- the method of
- the method of Maximum Likelihood
- the method of Least
- Bayesian methods
(1) Method of Moments
(2) Maximum Likelihood
Here is the example of lambda of a Poisson:
(3) Least Squares
Strictly empirical approach to estimation
- Parent model includes predictor variables
- Used for curve fitting (regression)
- Find curve with minimal deviation from all data points
- Define best fit as minimizing the sum of squared errors
- Related to the mean, which minimizes SSE about itself
Examples from linear regression
Linear relationship, parameters are slope and intercept
- Empirical model (Ch. 1): y = ax + b
- Statistical model: y = ax + b + epsilon "noise"
- xi predictors, yi outcomes
- Y = aX + b + e random
- yi = axi + b + ei one observation
- ei = yi - (axi + b) error for one observation
I. Mean of a normal distribution with "known" variance, as an
Where do we put (two places) so
that is not too extreme?
If is actually at point x, will
fall within the Confidence Interval? Find the limits of where cab be:
II. Mean of a normal distribution with unknown variance; use t
If n is large, use degrees of freedom "infinity", i.e. approximate by
the normal distribution z.
estimated standard error of the mean