为什么RSS分布卡方数np？

我想了解为什么在OLS模型下RSS（残差平方和）分布为（是模型中参数的数量，是观测值的数量）。

χ 2 \cdot (n - p)

$\chi^2\cdot (n-p)$

p $p$

n $n$

对于提出这样的基本问题，我深表歉意，但似乎无法在线（或在我的面向应用程序的教科书中）找到答案。

regression distributions least-squares

— 塔尔·加利利
source

请注意，答案展示说法并不完全正确：RSS的分布是

σ2 $\sigma^2$ （不

n−p $n-p$ ）乘以

χ2(n−p) $\chi^2(n-p)$ 分布，其中

σ2 $\sigma^2$ 是错误的真正变化。

— ub

Answers:

我考虑以下线性模型： ${y} = X \beta + \epsilon$ 。

残差的向量由下式估算

ϵ^= y - X β^= (I - X (X' X) - 1 X') y = Q y = Q (X β + ϵ) = Q ϵ

$\hat{\epsilon} = y - X \hat{\beta} = (I - X (X'X)^{-1} X') y = Q y = Q (X \beta + \epsilon) = Q \epsilon$

其中。 $Q = I - X (X'X)^{-1} X'$

观察到（在循环置换下轨迹是不变的），并且。因此，的特征值是和（下面有一些详细信息）。因此，存在一个unit矩阵使得（当且仅当它们是正态时，矩阵才可以由unit矩阵对角化。） $\textrm{tr}(Q) = n - p$ $Q'=Q=Q^2$ $Q$ $0$ $1$ $V$

V' Q V = Δ = diag (1, \dots, 1        n - p times, 0, \dots, 0        p times)

$V'QV = \Delta = \textrm{diag}(\underbrace{1, \ldots, 1}_{n-p \textrm{ times}}, \underbrace{0, \ldots, 0}_{p \textrm{ times}})$

现在，让我们。 $K = V' \hat{\epsilon}$

由于，我们有，并且因此。从而 $\hat{\epsilon} \sim N(0, \sigma^2 Q)$ $K \sim N(0, \sigma^2 \Delta)$ $K_{n-p+1}=\ldots=K_n=0$

∥ K ∥ 2 σ 2 = ∥ K ⋆ ∥ 2 σ 2 \sim χ 2 n - p

$\frac{\|K\|^2}{\sigma^2} = \frac{\|K^{\star}\|^2}{\sigma^2} \sim \chi^2_{n-p}$

与。 $K^{\star} = (K_1, \ldots, K_{n-p})'$

此外，由于是a矩阵，所以我们也有 $V$

∥ ϵ^∥ 2 = ∥ K ∥ 2 = ∥ K ⋆ ∥ 2

$\|\hat{\epsilon}\|^2 = \|K\|^2=\|K^{\star}\|^2$

从而

RSS σ 2 \sim χ 2 n - p

$\frac{\textrm{RSS}}{\sigma^2} \sim \chi^2_{n-p}$

最后，观察到该结果意味着

E (RSS n - p) = σ 2

$E\left(\frac{\textrm{RSS}}{n-p}\right) = \sigma^2$

Since $Q^2 - Q =0$ , the minimal polynomial of $Q$ divides the polynomial $z^2 - z$ . So, the eigenvalues of $Q$ are among $0$ and $1$ . Since $\textrm{tr}(Q) = n-p$ is also the sum of the eigenvalues multiplied by their multiplicity, we necessarily have that $1$ is an eigenvalue with multiplicity $n-p$ and zero is an eigenvalue with multiplicity $p$ .

— ocram
source

(+1) Good answer. One can restrict attention to orthogonal, instead of unitary,

V $V$ since

Q $Q$ is real and symmetric. Also, what is

SCR $\mathrm{SCR}$ ? I do not see it defined. By slightly rejiggering the argument, one can also avoid the use of a degenerate normal, in case that causes some consternation to those not familiar with it.

— cardinal

@Cardinal. Good point. SCR ('Somme des Carrés Résiduels' in french) should have been RSS.

— ocram

Thank you for the detailed answer Ocram! Some steps will require me to look more, but I have an outline to think about now - thanks!

— Tal Galili

@Glen_b: Oh, I made an edit a couple of days ago to change SCR to SRR. I didn't remember that SCR is mentionned in my comment. Sorry for the confusion.

— ocram

@Glen_b: It was supposed to mean RSS :-S Edited again. Thx

— ocram

IMHO, the matricial notation $Y=X\beta+\epsilon$ complicates things. Pure vector space language is cleaner. The model can be written $\boxed{Y=\mu + \sigma G}$ where $G$ has the standard normal distributon on $\mathbb{R}^n$ and $\mu$ is assumed to belong to a vector subspace $W \subset \mathbb{R}^n$ .

Now the language of elementary geometry comes into play. The least-squares estimator $\hat\mu$ of $\mu$ is nothing but $P_WY$ : the orthogonal projection of the observable $Y$ on the space $W$ to which $\mu$ is assumed to belong. The vector of residuals is $P^\perp_WY$ : projection on the orthogonal complement $W^\perp$ of $W$ in $\mathbb{R^n}$ . The dimension of $W^\perp$ is $\dim(W^\perp)=n-\dim(W)$ .

Finally,

P ⊥ W Y = P ⊥ W (μ + σ G) = 0 + σ P ⊥ W G,

$P^\perp_WY = P^\perp_W(\mu + \sigma G) = 0 + \sigma P^\perp_WG,$ and

P⊥WG $P^\perp_WG$ has the standard normal distribution on

$W^\perp$ , hence its squared norm has the

$\chi^2$ distribution with

$\dim(W^\perp)$ degrees of freedom.

This demonstration uses only one theorem, actually a definition-theorem:

Definition and theorem. A random vector in $\mathbb{R}^n$ has the standard normal distribution on a vector space $U \subset \mathbb{R}^n$ if it takes its values in $U$ and its coordinates in one ( $\iff$ in all) orthonormal basis of $U$ are independent one-dimensional standard normal distributions

(from this definition-theorem, Cochran's theorem is so obvious that it is not worth to state it)

— Stéphane Laurent
source