协方差等于零是否意味着二进制随机变量具有独立性？

如果 $X$ 和 $Y$ 是两个只能具有两个可能状态的随机变量，我如何证明 $Cov(X,Y) = 0$ 表示独立性？这种违背了我回想起 $Cov(X,Y) = 0$ 并不意味着独立...

提示说从 $1$ 和开始 $0$ 作为可能的状态，并从那里开始进行概括。我可以这样做并显示 $E(XY) = E(X)E(Y)$ ，但这并不意味着独立？

我猜这有点困惑如何数学上做到这一点。

covariance independence

— 用户名
source

你的问题的标题表明它不是一般的真实..

— 迈克尔·Chernick

您要证明的陈述确实是正确的。如果

X

$X$ 和

Y

$Y$ 分别是参数参数为

p_{1}

$p_1$ 和

伯努利随机变量

p_{2}

$p_2$ ，则

E [X] = p_{1}

$E[X]=p_1$ 和

E [Y] = p_{2}

$E[Y]=p_2$ 。所以，

cov (X, Y) = E [X Y] - E [X] E [Y]

$\operatorname{cov}(X,Y)=E[XY]-E[X]E[Y]$ 等于

0

$0$ 仅当

E [X Y] = P {X = 1, Y = 1}

$E[XY]=P\{X=1,Y=1\}$ 等于

p_{1} p_{2} = P {X = 1} P {Y = 1}

$p_1p_2=P\{X=1\}P\{Y=1\}$ 表示

{X = 1}

$\{X=1\}$ 和

{Y = 1}

$\{Y=1\}$ 是独立事件。标准结果是，如果

A

$A$ 和

B

$B$ 是一对独立事件，那么

A, B^{c}

$A,B^c$ 和

A^{c}, B

$A^c,B$ 以及

A^{c}, B^{c}

$A^c,B^c$ 独立事件也是如此，即

X

$X$ 和

Y

$Y$ 是独立随机变量。现在概括。

— Dilip Sarwate

Answers:

对于二进制变量，其期望值等于它们等于1的概率。因此，

E (X Y) = P (X Y = 1) = P (X = 1 \cap Y = 1) E (X) = P (X = 1) E (Y) = P (Y = 1)

$E(XY) = P(XY = 1) = P(X=1 \cap Y=1) \\ E(X) = P(X=1) \\ E(Y) = P(Y=1) \\$

如果两个协方差为零，则意味着 $E(XY) = E(X)E(Y)$ ，这意味着

P (X = 1 \cap Y = 1) = P (X = 1) \cdot P (Y = 1)

$P(X=1 \cap Y=1) = P(X=1) \cdot P(Y=1)$

使用关于独立事件的基本规则（即，如果 $A$ 和 $B$ 是独立的，则它们的补码是独立的，等等），看到所有其他联合概率也相乘是很简单的，这意味着联合质量函数是因式分解的两个随机变量是独立的。

— 游戏玩家
source

简洁优雅。优！+1 = D

— Marcelo Ventura '02

相关性和协方差都可以测量两个给定变量之间的线性关联，并且没有义务检测其他任何形式的关联。

因此，这两个变量可能以其他几种非线性方式关联，并且协方差（因此，相关性）无法与独立情况区分开。

作为一个非常说教，人工和非现实的例子，可以考虑，使得为，并且还考虑。请注意，它们不仅是关联的，而且是另一个的功能。但是，它们的协方差为0，因为它们的关联与协方差可以检测到的关联正交。 $X$ $P(X=x)=1/3$ $x=−1,0,1$ $Y=X^2$

编辑

确实，正如@whuber所指出的那样，以上原始答案实际上是关于如果两个变量都不一定是二分法的话该断言不是普遍正确的评论。我的错！

因此，让我们数学起来。（与Barney Stinson的“ Suit up！”相当）

特殊案例

如果两个和是二分，则可以假定，不失一般性，这两个假设只的值和具有任意的概率，和由下式给出 $X$ $Y$ $0$ $1$ $p$ $q$ $r$ 表征完全的联合分布和。接受@DilipSarwate的提示，请注意，这三个值足以确定的联合分布，因为

\begin{aligned} P (X = 1) = p \in [0, 1] \\ P (Y = 1) = q \in [0, 1] \\ P (X = 1, Y = 1) = r \in [0, 1], \end{aligned}

$\begin{align*} P(X=1) = p \in [0,1] \\ P(Y=1) = q \in [0,1] \\ P(X=1,Y=1) = r \in [0,1], \end{align*}$

X

$X$

Y

$Y$

(X, Y)

$(X,Y)$

（在一个侧面说明，当然

势必尊重两个

，

和

\begin{aligned} P (X = 0, Y = 1) & = P (Y = 1) - P (X = 1, Y = 1) = q - r \\ P (X = 1, Y = 0) & = P (X = 1) - P (X = 1, Y = 1) = p - r \\ P (X = 0, Y = 0) & = 1 - P (X = 0, Y = 1) - P (X = 1, Y = 0) - P (X = 1, Y = 1) \\ = 1 - (q - r) - (p - r) - r = 1 - p - q - r . \end{aligned}

$\begin{align*} P(X=0,Y=1) &= P(Y=1) - P(X=1,Y=1) = q - r\\ P(X=1,Y=0) &= P(X=1) - P(X=1,Y=1) = p - r\\ P(X=0,Y=0) &= 1 - P(X=0,Y=1) - P(X=1,Y=0) - P(X=1,Y=1) \\ &= 1 - (q - r) - (p - r) - r = 1 - p - q - r. \end{align*}$

r

$r$

p - r \in [0, 1]

$p-r\in[0,1]$

q - r \in [0, 1]

$q-r\in[0,1]$

超出

，这是说

。）

1 - p - q - r \in [0, 1]

$1-p-q-r\in[0,1]$

r \in [0, 1]

$r\in[0,1]$

r \in [0, min (p, q, 1 - p - q)]

$r\in[0,\min(p,q,1-p-q)]$

Notice that $r = P(X=1,Y=1)$ might be equal to the product $p\cdot q = P(X=1) P(Y=1)$ , which would render $X$ and $Y$ independent, since

\begin{aligned} P (X = 0, Y = 0) & = 1 - p - q - p q = (1 - p) (1 - q) = P (X = 0) P (Y = 0) \\ P (X = 1, Y = 0) & = p - p q = p (1 - q) = P (X = 1) P (Y = 0) \\ P (X = 0, Y = 1) & = q - p q = (1 - p) q = P (X = 0) P (Y = 1) . \end{aligned}

$\begin{align*} P(X=0,Y=0) &= 1 - p - q - pq = (1-p)(1-q) = P(X=0)P(Y=0)\\ P(X=1,Y=0) &= p - pq = p(1-q) = P(X=1)P(Y=0)\\ P(X=0,Y=1) &= q - pq = (1-p)q = P(X=0)P(Y=1). \end{align*}$

Yes, $r$ might be equal to $pq$ , BUT it can be different, as long as it respects the boundaries above.

Well, from the above joint distribution, we would have

\begin{aligned} E (X) & = 0 \cdot P (X = 0) + 1 \cdot P (X = 1) = P (X = 1) = p \\ E (Y) & = 0 \cdot P (Y = 0) + 1 \cdot P (Y = 1) = P (Y = 1) = q \\ E (X Y) & = 0 \cdot P (X Y = 0) + 1 \cdot P (X Y = 1) \\ = P (X Y = 1) = P (X = 1, Y = 1) = r \\ C o v (X, Y) & = E (X Y) - E (X) E (Y) = r - p q \end{aligned}

$\begin{align*} E(X) &= 0\cdot P(X=0) + 1\cdot P(X=1) = P(X=1) = p \\ E(Y) &= 0\cdot P(Y=0) + 1\cdot P(Y=1) = P(Y=1) = q \\ E(XY) &= 0\cdot P(XY=0) + 1\cdot P(XY=1) \\ &= P(XY=1) = P(X=1,Y=1) = r\\ Cov(X,Y) &= E(XY) - E(X)E(Y) = r - pq \end{align*}$

Now, notice then that $X$ and $Y$ are independent if and only if $Cov(X,Y)=0$ . Indeed, if $X$ and $Y$ are independent, then $P(X=1,Y=1)=P(X=1)P(Y=1)$ , which is to say $r=pq$ . Therefore, $Cov(X,Y)=r-pq=0$ ; and, on the other hand, if $Cov(X,Y)=0$ , then $r-pq=0$ , which is to say $r=pq$ . Therefore, $X$ and $Y$ are independent.

General Case

About the without loss of generality clause above, if $X$ and $Y$ were distributed otherwise, let's say, for $a<b$ and $c<d$ ,

\begin{aligned} P (X = b) = p \\ P (Y = d) = q \\ P (X = b, Y = d) = r \end{aligned}

$\begin{align*} P(X=b)=p \\ P(Y=d)=q \\ P(X=b, Y=d)=r \end{align*}$ then

X^{'}

$X'$ and

Y^{'}

$Y'$ given by

X^{'} = \frac{X - a}{b - a} and Y^{'} = \frac{Y - c}{d - c}

$X'=\frac{X-a}{b-a} \qquad \text{and} \qquad Y'=\frac{Y-c}{d-c}$ would be distributed just as characterized above, since

X = a \Leftrightarrow X^{'} = 0, X = b \Leftrightarrow X^{'} = 1, Y = c \Leftrightarrow Y^{'} = 0 and Y = d \Leftrightarrow Y^{'} = 1.

$X=a \Leftrightarrow X'=0, \quad X=b \Leftrightarrow X'=1, \quad Y=c \Leftrightarrow Y'=0 \quad \text{and} \quad Y=d \Leftrightarrow Y'=1.$ So

X

$X$ and

Y

$Y$ are independent if and only if

X^{'}

$X'$ and

Y^{'}

$Y'$ are independent.

Also, we would have

\begin{aligned} E (X^{'}) & = E (\frac{X - a}{b - a}) = \frac{E (X) - a}{b - a} \\ E (Y^{'}) & = E (\frac{Y - c}{d - c}) = \frac{E (Y) - c}{d - c} \\ E (X^{'} Y^{'}) & = E (\frac{X - a}{b - a} \frac{Y - c}{d - c}) = \frac{E [(X - a) (Y - c)]}{(b - a) (d - c)} \\ = \frac{E (X Y - X c - a Y + a c)}{(b - a) (d - c)} = \frac{E (X Y) - c E (X) - a E (Y) + a c}{(b - a) (d - c)} \\ C o v (X^{'}, Y^{'}) & = E (X^{'} Y^{'}) - E (X^{'}) E (Y^{'}) \\ = \frac{E (X Y) - c E (X) - a E (Y) + a c}{(b - a) (d - c)} - \frac{E (X) - a}{b - a} \frac{E (Y) - c}{d - c} \\ = \frac{[E (X Y) - c E (X) - a E (Y) + a c] - [E (X) - a] [E (Y) - c]}{(b - a) (d - c)} \\ = \frac{[E (X Y) - c E (X) - a E (Y) + a c] - [E (X) E (Y) - c E (X) - a E (Y) + a c]}{(b - a) (d - c)} \\ = \frac{E (X Y) - E (X) E (Y)}{(b - a) (d - c)} = \frac{1}{(b - a) (d - c)} C o v (X, Y) . \end{aligned}

$\begin{align*} E(X') &= E\left(\frac{X-a}{b-a}\right) = \frac{E(X)-a}{b-a} \\ E(Y') &= E\left(\frac{Y-c}{d-c}\right) = \frac{E(Y)-c}{d-c} \\ E(X'Y') &= E\left(\frac{X-a}{b-a} \frac{Y-c}{d-c}\right) = \frac{E[(X-a)(Y-c)]}{(b-a)(d-c)} \\ &= \frac{E(XY-Xc-aY+ac)}{(b-a)(d-c)} = \frac{E(XY)-cE(X)-aE(Y)+ac}{(b-a)(d-c)} \\ Cov(X',Y') &= E(X'Y')-E(X')E(Y') \\ &= \frac{E(XY)-cE(X)-aE(Y)+ac}{(b-a)(d-c)} - \frac{E(X)-a}{b-a} \frac{E(Y)-c}{d-c} \\ &= \frac{[E(XY)-cE(X)-aE(Y)+ac] - [E(X)-a] [E(Y)-c]}{(b-a)(d-c)}\\ &= \frac{[E(XY)-cE(X)-aE(Y)+ac] - [E(X)E(Y)-cE(X)-aE(Y)+ac]}{(b-a)(d-c)}\\ &= \frac{E(XY)-E(X)E(Y)}{(b-a)(d-c)} = \frac{1}{(b-a)(d-c)} Cov(X,Y). \end{align*}$ So

C o v (X, Y) = 0

$Cov(X,Y)=0$ if and only

C o v (X^{'}, Y^{'}) = 0

$Cov(X',Y')=0$ .

— Marcelo Ventura
source

I recycled that answer from this post.

— Marcelo Ventura

Verbatim cut and paste from your other post. Love it. +1

— gammer

The problem with copy-and-paste is that your answer no longer seems to address the question: it is merely a comment on the question. It would be better, then, to post a comment with a link to your other answer.

— whuber

How is thus an answer to the question asked?

— Dilip Sarwate

Your edits still don't answer the question, at least not at the level the question is asked. You write "Notice that

r \dots

$r~\ldots$ not necessarily equal to the product

p q

$pq$ . That exceptional situation corresponds to the case of independence between

X

$X$ and

Y

$Y$ ." which is a perfectly true statement but only for the cognoscenti because for the hoi polloi, independence requires not just that

\begin{matrix} (1) & P (X = 1, Y = 1) = P (X = 1) P (Y = 1) \end{matrix}

$P(X=1,Y=1)=P(X=1)P(Y=1)\tag 1$ but also

\begin{matrix} (2) & P (X = u, Y = v) = P (X = u) P (Y = v), u . v \in {0, 1} . \end{matrix}

$P(X=u,Y=v)=P(X=u)P(Y=v),~u.v\in\{0,1\}.\tag 2$ Yes,

(1) ⟹ (2)

$(1) \implies(2)$ as the cognoscenti know; for lesser mortals, a proof that

(1) ⟹ (2)

$(1) \implies (2)$ is helpful.

— Dilip Sarwate

IN GENERAL:

The criterion for independence is $F(x,y) = F_X(x)F_Y(y)$ . Or

\begin{matrix} (1) & f_{X, Y} (x, y) = f_{X} (x) f_{Y} (y) \end{matrix}

$f_{X,Y}(x,y)=f_X(x)\,f_Y(y)\tag 1$

"If two variables are independent, their covariance is $0.$ But, having a covariance of $0$ does not imply the variables are independent."

This is nicely explained by Macro here, and in the Wikipedia entry for independence.

$\text {independence} \Rightarrow \text{zero cov}$ , yet

$\text{zero cov}\nRightarrow \text{independence}.$

Great example: $X \sim N(0,1)$ , and $Y= X^2.$ Covariance is zero (and $\mathbb E(XY)=0$ , which is the criterion for orthogonality), yet they are dependent. Credit goes to this post.

IN PARTICULAR (OP problem):

These are Bernoulli rv's, $X$ and $Y$ with probability of success $\Pr(X=1)$ , and $\Pr(Y=1)$ .

$\begin{align}\mathrm{cov}(X,Y)&=\mathrm E[XY] - \mathrm E[X]\,\mathrm E[Y]\\[2ex] &\underset{*}{=} \Pr(X=1 \cap Y=1) - \Pr(X=1)\, \Pr(Y=1)\\[2ex] &\implies \Pr(X=1 , Y=1) = \Pr (X=1)\,\Pr(Y=1). \end{align}$

This is equivalent to the condition for independence in Eq. $(1).$

$(*)$ :

E [X Y] \underset{* *}{=} \sum_{domain X, Y} Pr (X = x \cap Y = y) x y \underset{\neq 0 iff x \times y \neq 0}{=} Pr (X = 1 \cap Y = 1) .

$\mathrm E[XY]\quad \underset{**}{=} \quad \displaystyle \sum_{\text{domain X, Y}} \Pr(X=x\cap Y=y)\, x\,y \underset{\neq\,0\text{ iff } x \times y\neq 0}= \Pr(X=1 \cap Y=1).$

$(**)$ : by LOTUS.

As pointed out below, the argument is incomplete without what Dilip Sarwate had pointed out in his comments shortly after the OP appeared. After searching around, I found this proof of the missing part here:

If events $A$ and $B$ are independent, then events $A^c$ and $B$ are independent, and events $A^c$ and $B^c$ are also independent.

Proof By definition,

$A$ and $B$ are independent $\iff P(A\cap B) = P(A)P(B).$

But $B=(A\cap B) + ( A^c \cup B)$ , so $P(B)= P(A\cap B) + P(A^c \cup B)$ , which yields:

$\small P(A^c \cap B) = P(B) - P(A\cap B) = P(B) - P(A)\,P(B) = P(B) \left[1 - P(A)\right] = P(B)\,P( A^c).$

Repeat the argument for the events $A^c$ and $B^c,$ this time starting from the statement that $A^c$ and $B$ are independent and taking the complement of $B.$

Similarly. $A$ and $B^c$ are independent events.

So, we have shown already that

Pr (X = 1, Y = 1) = Pr (X = 1) Pr (Y = 1)

$\Pr(X=1 , Y=1) = \Pr (X=1)\,\Pr(Y=1)$ and the above shows that this implies that

Pr (X = i, Y = j) = Pr (X = i) Pr (Y = j), i, j \in {0, 1}

$\Pr(X=i , Y=j) = \Pr (X=i)\,\Pr(Y=j), ~~i, j \in \{0,1\}$ that is, the joint pmf factors into the product of marginal pmfs everywhere, not just at

(1, 1)

$(1,1)$ . Hence, uncorrelated Bernoulli random variables

X

$X$ and

Y

$Y$ are also independent random variables.

— Antoni Parellada
source

Actually that's not an equivalent condition to Eq (1). All you showed was that

f_{X, Y} (1, 1) = f_{X} (1) f_{Y} (1)

$f_{X,Y}(1,1) = f_{X}(1) f_{Y}(1)$

— gammer

Please consider replacing that image with your own equations, preferably ones that don't use overbars to denote complements. The overbars in the image are very hard to see.

— Dilip Sarwate

@DilipSarwate No problem. Is it better, now?

— Antoni Parellada

Thanks. Also, note that strictly speaking, you also need to show that

A

$A$ and

B^{c}

$B^c$ are independent events since the factorization of the joint pdf into the product of the marginal pmts must hold at all four points. Perhaps adding the sentence "Similarly.

A

$A$ and

B^{c}

$B^c$ are independent events" right after the proof that

A^{c}

$A^c$ and

B

$B$ are independent events will work.

— Dilip Sarwate

@DilipSarwate Thank you very much for your help getting it right. The proof as it was before all the editing seemed self-explanatory, because of all the inherent symmetry, but it clearly couldn't be taken for granted. I am very appreciative of your assistance.

— Antoni Parellada