如何计算估计的OLS的方差


17

我知道

β0^=y¯β1^x¯
,这是我得到多远,当我计算方差:

Var(β0^)=Var(y¯β1^x¯)=Var((x¯)β1^+y¯)=Var((x¯)β1^)+Var(y¯)=(x¯)2Var(β1^)+0=(x¯)2Var(β1^)+0=σ2(x¯)2i=1n(xix¯)2

但这距离我还很远。我想要计算的最终公式是

Var(β0^)=σ2n1i=1nxi2i=1n(xix¯)2

我不确定如何获得假设我的数学到那里是正确的) 。

(x¯)2=1ni=1nxi2

这是正确的道路吗?

(x¯)2=(1ni=1nxi)2=1n2(i=1nxi)2

我敢肯定,这很简单,因此,如果有人暗示将我推向正确的方向,答案可能会稍等。


2
这不是正确的道路。第四个方程式不成立。例如,对于x1=1x2=0,和x3=1,左项为零,而正确的术语是2/3。问题来自拆分方差的步骤(第二个方程的第三行)。明白为什么吗?
QuantIbex

对Quantlbex点的提示:方差不是线性函数。它违反了可加性和标量乘法。
大卫·马克思

@DavidMarx即步骤应该是
=Var((x¯)β1^+y¯)=(x¯)2Var(β1^)+y¯
,我想,然后一旦我在替代对于β1^y¯(不知道该怎么为这个做的,但我会更多的考虑一下),把我在正确的道路,我希望。
MT

这是不正确的。考虑一下求和等于方差之和的条件。
QuantIbex

2
否,y¯是随机的,因为yi=β0+β1xi+ϵ,其中,ϵ表示(随机)的噪声。但是,好的,我之前的评论可能会引起误解。另外,如果ab表示常数,则Var(aX+b)=a2Var(X)ab
QuantIbex

Answers:


19

这是一个自学型问题,因此,我提供了一些提示,希望可以帮助找到解决方案,并且将根据您的反馈/进度来编辑答案。

参数估计值,最大限度地减少平方的总和是 β 0 要获得的方差β0,从它的表达启动和替代的表达β1,并执行代数 V- [R β0=V- [R ˉý-β1ˉX=...

β^0=y¯β^1x¯,β^1=i=1n(xix¯)yii=1n(xix¯)2.
β^0β^1
Var(β^0)=Var(Y¯β^1x¯)=

编辑:
我们有 两个方差术语是 V- [R ˉ Ý=V- [R 1

Var(β^0)=Var(Y¯β^1x¯)=Var(Y¯)+(x¯)2Var(β^1)2x¯Cov(Y¯,β^1).
V - [R β 1
Var(Y¯)=Var(1ni=1nYi)=1n2i=1nVar(Yi)=σ2n,
并且协方差项是 ÇÒv ˉ Ý β 1
Var(β^1)=1[i=1n(xix¯)2]2i=1n(xix¯)2Var(Yi)=σ2i=1n(xix¯)2,
Cov(Y¯,β^1)=Cov{1ni=1nYi,j=1n(xjx¯)Yji=1n(xix¯)2}=1n1i=1n(xix¯)2Cov{i=1nYi,j=1n(xjx¯)Yj}=1ni=1n(xix¯)2i=1n(xjx¯)j=1nCov(Yi,Yj)=1ni=1n(xix¯)2i=1n(xjx¯)σ2=0
since i=1n(xjx¯)=0.
And since
i=1n(xix¯)2=i=1nxi22x¯i=1nxi+i=1nx¯2=i=1nxi2nx¯2,
we have
Var(β^0)=σ2n+σ2x¯2i=1n(xix¯)2=σ2ni=1n(xix¯)2{i=1n(xix¯)2+nx¯2}=σ2i=1nxi2ni=1n(xix¯)2.

Edit 2

Why do we have var(i=1nYi)=i=1nVar(Yi)?

The assumed model is Yi=β0+β1Xi+ϵi, where the ϵi are independant and identically distributed random variables with E(ϵi)=0 and var(ϵi)=σ2.

Once we have a sample, the Xi are known, the only random terms are the ϵi. Recalling that for a random variable Z and a constant a, we have var(a+Z)=var(Z). Thus,

var(i=1nYi)=var(i=1nβ0+β1Xi+ϵi)=var(i=1nϵi)=i=1nj=1ncov(ϵi,ϵj)=i=1ncov(ϵi,ϵi)=i=1nvar(ϵi)=i=1nvar(β0+β1Xi+ϵi)=i=1nvar(Yi).
The 4th equality holds as cov(ϵi,ϵj)=0 for ij by the independence of the ϵi.

I think I got it! The book has suggested steps, and I was able to prove each step separately (I think). It's not as satisfying as just sitting down and grinding it out from this step, since I had to prove intermediate conclusions for it to help, but I think everything looks good.
M T

See edit for the development of the suggested approach.
QuantIbex

The variance of the sum equals the sum of the variances in this step:
Var(Y¯)=Var(1ni=1nYi)=1n2i=1nVar(Yi)
because since the Xi are independent, this implies that the Yi are independent as well, right?
M T

Also, you can factor out a constant from the covariance in this step:
1n1i=1n(xix¯)2Cov{i=1nYi,j=1n(xjx¯)Yj}
even though it's not in both elements because the formula for covariance is multiplicative, right?
M T

1
@oort, in the numerator you have the sum of n terms that are identical (and equal to σ2), so the numerator is nσ2.
QuantIbex

1

I got it! Well, with help. I found the part of the book that gives steps to work through when proving the Var(β^0) formula (thankfully it doesn't actually work them out, otherwise I'd be tempted to not actually do the proof). I proved each separate step, and I think it worked.

I'm using the book's notation, which is:

SSTx=i=1n(xix¯)2,
and ui is the error term.

1) Show that β^1 can be written as β^1=β1+i=1nwiui where wi=diSSTx and di=xix¯.

This was easy because we know that

β^1=β1+i=1n(xix¯)uiSSTx=β1+i=1ndiSSTxui=β1+i=1nwiui

2) Use part 1, along with i=1nwi=0 to show that β1^ and u¯ are uncorrelated, i.e. show that E[(β1^β1)u¯]=0.

E[(β1^β1)u¯]=E[u¯i=1nwiui]=i=1nE[wiu¯ui]=i=1nwiE[u¯ui]=1ni=1nwiE(uij=1nuj)=1ni=1nwi[E(uiu1)++E(uiuj)++E(uiun)]

and because the u are i.i.d., E(uiuj)=E(ui)E(uj) when ji.

When j=i, E(uiuj)=E(ui2), so we have:

=1ni=1nwi[E(ui)E(u1)++E(ui2)++E(ui)E(un)]=1ni=1nwiE(ui2)=1ni=1nwi[Var(ui)+E(ui)E(ui)]=1ni=1nwiσ2=σ2ni=1nwi=σ2nSSTxi=1n(xix¯)=σ2nSSTx(0)=0

3) Show that β0^ can be written as β0^=β0+u¯x¯(β1^β1). This seemed pretty easy too:

β0^=y¯β1^x¯=(β0+β1x¯+u¯)β1^x¯=β0+u¯x¯(β1^β1).

4) Use parts 2 and 3 to show that Var(β0^)=σ2n+σ2(x¯)2SSTx:

Var(β0^)=Var(β0+u¯x¯(β1^β1))=Var(u¯)+(x¯)2Var(β1^β1)=σ2n+(x¯)2Var(β1^)=σ2n+σ2(x¯)2SSTx.

I believe this all works because since we provided that u¯ and β1^β1 are uncorrelated, the covariance between them is zero, so the variance of the sum is the sum of the variance. β0 is just a constant, so it drops out, as does β1 later in the calculations.

5) Use algebra and the fact that SSTxn=1ni=1nxi2(x¯)2:

Var(β0^)=σ2n+σ2(x¯)2SSTx=σ2SSTxSSTxn+σ2(x¯)2SSTx=σ2SSTx(1ni=1nxi2(x¯)2)+σ2(x¯)2SSTx=σ2n1i=1nxi2SSTx

There might be a typo in point 1; I think var(β^) should read β^.
QuantIbex

You might want to clarify notations, and specify what ui and SSTx are.
QuantIbex

ui is the error term and SSTx is the total sum of squares for x (defined in the edit).
M T

1
In point 1, the term β1 is missing in the last two lines.
QuantIbex

1
In point 2, you can't take u¯ out of the expectation, it's not a constant.
QuantIbex
By using our site, you acknowledge that you have read and understand our Cookie Policy and Privacy Policy.
Licensed under cc by-sa 3.0 with attribution required.