(Un)biased estimator of variance

[Christopher Kolago]

How one can check if:
Theta = 1/(2n) * SUM_{k=1}^{n} {( X_k - X_(k-1) )^2}
is unbiased estimator of variance in normal distribution N(u,s)?

I'm a little confused and I have no idea how to check this fact. (Although I suspect that it will be a biased estimator.) I was trying to find the expected value of Theta but I'm not sure how to calculate it.

Chris

PS. I'm looking for some books in which there are some basic examples connected with checking whether something is biased or unbiased estimator (perhaps with the minimal variance) of something else. I'll be very grateful for some suggestions.

[Jon Slaughter]

Do you know what it means for an estimator to be unbiased? If not then maybe
that is the place to start?

[Christopher Kolago]

I believe that in the context of my problem Theta is unbiased if and only if E(Theta) = s, where E(Theta) is an expected value of Theta and s is the deviation from the normal distribution N(u, s).

Chris

[Jon Slaughter]

http://en.wikipedia.org/wiki/Bias_of_an_estimator

Bias[theta_hat] = E[theta_hat] - theta

E[1/(2n) * SUM_{k=1}^{n} {( X_k - X_(k-1))^2]

= 1/2n*sum(E[(X_k - X_(k-1))^2])
= 1/2n*sum(E[(X_k - u) - (X_(k-1) - u))^2])
= 1/2n*sum(E[(X_k - u)^2 + (X_k(-1) - u)^2 - 2*(X_k-u)*(X_(k-1) - u)])

Now continue along these lines and be careful with the sum. Break it into
two sums because of the difference in indexing so that the indexes are the
same(and you just have "endpoint issues"). Simply and see what you come up
with. It's very similar to the example on that page except here you have
X_(k-1) where X_bar is.

Note that for an unbiased estimater you have Bias = 0. So if you don't have
that then it is biased.

If you read up on http://en.wikipedia.org/wiki/Estimation_theory and folow
the links to bias, error, etc then you'll probably understand whats going
on.

http://en.wikipedia.org/wiki/Estimator is a good place to start too.

Goto definition 5 in that link and read what it says about "error" and bias.
The bias is the expected value of the error.

[Herman Rubin]

In article <1678286274.1555.12579...@gallium.mathforum.org>,

Christopher Kolago <krzyszto...@uj.edu.pl> wrote:
>How one can check if:
>Theta = 1/(2n) * SUM_{k=1}^{n} {( X_k - X_(k-1) )^2}
>is unbiased estimator of variance in normal distribution N(u,s)?

>I'm a little confused and I have no idea how to check this fact. (Although I suspect that it will be a biased estimator.) I was trying to find the expected value of Theta but I'm not sure how to calculate it.

Compute its expected value term by term, assuming that
X_k and X_(k-1) are uncorrelated and identically distributed.
Normality is not needed.
--
This address is for information only. I do not claim that these views
are those of the Statistics Department or of Purdue University.
Herman Rubin, Department of Statistics, Purdue University
hru...@stat.purdue.edu Phone: (765)494-6054 FAX: (765)494-0558

[Christopher Kolago]

Thanks. I was trying do calculate it this way yesterday, but when I reached the line:

E(Theta) = s^2 + E[ (X_k - u) * [1/n * (u - X_(k-1)) - (X_k - u)] ]

I gave up, 'cause I don't know if everything after "s^2 +" is non-zero. I thought that this way of approaching this problem must be wrong, because I didn't use that fact that we are dealing with the normal distribution N(u, s).

I suspect that Theta is asymptotically unbiased (but biased in the normal sense), because of the fact that we deal with N(u, s), so everything after "s^2 +" will be converging to zero.

Chris

[Christopher Kolago]

> Thanks. I was trying do calculate it this way
> yesterday, but when I reached the line:
>
> E(Theta) = s^2 + E[ (X_k - u) * [1/n * (u - X_(k-1))
> - (X_k - u)] ]

Ups... It should be:

E(Theta) = s^2 + E[ 1/n * (sum_{k=1}^{n} {(X_k - u) * (u - X_(k-1))} - (X_k - u)^2 ]

[Ray Vickson]

On Nov 11, 2:41 pm, Christopher Kolago <krzysztof.kol...@uj.edu.pl>
wrote:

E[sum(X_k-X_{k-1})^2 = sum E(X_k-X_{k-1})^2, whether or not
X_0,X_1,...,X_n are iid. However, if they are iid then E(X_k-X_{k-1})
^2 = E(X_1-X_0)^2 for all k, and X_1-X_0 is normal with mean 0 and
variance 2s^2. So, what is E(X_1-X_0)^2? What conclusion can you draw?

R.G. Vickson

[Christopher Kolago]

I recalculated everything using facts that:

(1) Theta = 1/(2n) * SUM_{k=2}^{n} {X_k^2 - 2*X_k*X_(k-1) + X_(k-1)^2,
(2) E(X_i * X_j) = u2 (assuming that X_i and X_j are iid),
(3) E(X_i * X_i) = s^2 + u^2,

and it follows that: E(Theta) = (n-1)/n * s^2, so Theta is a biased estimator. :)

Chris