表明对5个主题进行100次测量比对100个主题进行5次测量提供的信息少得多

在一次会议上，我无意中听到以下声明：

5个主题的100次测量所提供的信息比100个主题的5次测量要少得多。

显然这是对的，但是我想知道如何用数学方式证明这一点……我认为可以使用线性混合模型。但是，我对用于估算它们的数学知识不甚了解（我只lmer4为LMM和bmrsGLMM 运行：）您能给我展示一个真实的例子吗？与R中的某些代码相比，我更希望提供一些公式的答案。请随意假设一个简单的设置，例如具有正态分布的随机截距和斜率的线性混合模型。

PS不涉及LMM的基于数学的答案也是可以的。我之所以想到LMM，是因为它们在我看来是一种自然的工具，可以解释为什么来自更多学科的较少量度要比来自少数学科的更多量度更好，但是我很可能错了。

简短的答案是，当且仅当数据中的类内相关性为正时，您的猜想才为真。从经验上讲，大多数时候大多数聚类数据集都显示出正的类内相关性，这意味着实际上您的猜想通常是正确的。但是，如果类内相关为0，则您提到的两种情况同样有用。而且，如果类内相关性为负，那么对更多主题进行更少的测量实际上就没有什么意义。实际上，我们宁愿（就减少参数估计的方差而言）对单个主题进行所有测量。

从统计学上讲，我们可以从两个角度考虑这一问题：您在问题中提到的随机效应（或混合）模型，或边际模型，最终在这里提供了更多信息。

随机效应（混合）模型

假设我们有一组主题，每个主题我们都进行了m次测量。那么第i个对象的第j次测量的简单随机效应模型可能是 y i j = β + u i + e i j， 其中β是固定截距，u i是随机对象效应（方差σ 2 ü），ê 我Ĵ是观察级别误差项（具有方差σ 2 $n$$m$$j$$i$

$$y_{ij} = \beta + u_i + e_{ij},$$ $\beta$$u_i$$\sigma^2_u$$e_{ij}$$\sigma^2_e$），而后两个随机项是独立的。

在此模型中，表示总体平均值，并且在平衡的数据集（即，每个受试者的测量数量相等）的情况下，我们的最佳估计值就是样本平均值。因此，如果我们采用“更多信息”来表示此估计的较小方差，则基本上我们想知道样本均值的方差如何取决于n和m。有了一点代数，我们就可以算出 var （$\beta$$n$$m$

$$\begin{aligned} \text{var}(\frac{1}{nm}\sum_i\sum_jy_{ij}) &= \text{var}(\frac{1}{nm}\sum_i\sum_j\beta + u_i + e_{ij}) \\ &= \frac{1}{n^2m^2}\text{var}(\sum_i\sum_ju_i + \sum_i\sum_je_{ij}) \\ &= \frac{1}{n^2m^2}\Big(m^2\sum_i\text{var}(u_i) + \sum_i\sum_j\text{var}(e_{ij})\Big) \\ &= \frac{1}{n^2m^2}(nm^2\sigma^2_u + nm\sigma^2_e) \\ &= \frac{\sigma^2_u}{n} + \frac{\sigma^2_e}{nm}. \end{aligned}$$ Examining this expression, we can see that whenever there is any subject variance (i.e., $\sigma^2_u>0$), increasing the number of subjects ($n$) will make both of these terms smaller, while increasing the number of measurements per subject ($m$) will only make the second term smaller. (For a practical implication of this for designing multi-site replication projects, see this blog post I wrote a while ago.)

Now you wanted to know what happens when we increase or decrease $m$ or $n$ while holding constant the total number of observations. So for that we consider $nm$ to be a constant, so that the whole variance expression just looks like

$$\frac{\sigma^2_u}{n} + \text{constant},$$ which is as small as possible when $n$ is as large as possible (up to a maximum of $n=nm$, in which case $m=1$, meaning we take a single measurement from each subject).

My short answer referred to the intra-class correlation, so where does that fit in? In this simple random-effects model the intra-class correlation is

$$\rho = \frac{\sigma^2_u}{\sigma^2_u + \sigma^2_e}$$ (sketch of a derivation here). So we can write the variance equation above as $$\text{var}(\frac{1}{nm}\sum_i\sum_jy_{ij}) = \frac{\sigma^2_u}{n} + \frac{\sigma^2_e}{nm} = \Big(\frac{\rho}{n} + \frac{1-\rho}{nm}\Big)(\sigma^2_u+\sigma^2_e)$$ This doesn't really add any insight to what we already saw above, but it does make us wonder: since the intra-class correlation is a bona fide correlation coefficient, and correlation coefficients can be negative, what would happen (and what would it mean) if the intra-class correlation were negative?

In the context of the random-effects model, a negative intra-class correlation doesn't really make sense, because it implies that the subject variance $\sigma^2_u$ is somehow negative (as we can see from the $\rho$ equation above, and as explained here and here)... but variances can't be negative! But this doesn't mean that the concept of a negative intra-class correlation doesn't make sense; it just means that the random-effects model doesn't have any way to express this concept, which is a failure of the model, not of the concept. To express this concept adequately we need to consider the marginal model.

Marginal model

For this same dataset we could consider a so-called marginal model of $y_{ij}$,

$$y_{ij} = \beta + e^*_{ij},$$ where basically we've pushed the random subject effect $u_i$ from before into the error term $e_{ij}$ so that we have $e^*_{ij} = u_i + e_{ij}$. In the random-effects model we considered the two random terms $u_i$ and $e_{ij}$ to be i.i.d., but in the marginal model we instead consider $e^*_{ij}$ to follow a block-diagonal covariance matrix $\textbf{C}$ like $$\textbf{C}= \sigma^2\begin{bmatrix} \textbf{R} & 0& \cdots & 0\\ 0& \textbf{R} & \cdots & 0\\ \vdots & \vdots & \ddots & \vdots \\ 0& 0& \cdots &\textbf{R}\\ \end{bmatrix}, \textbf{R}= \begin{bmatrix} 1 & \rho & \cdots & \rho \\ \rho & 1 & \cdots & \rho \\ \vdots & \vdots & \ddots & \vdots \\ \rho & \rho & \cdots &1\\ \end{bmatrix}$$ In words, this means that under the marginal model we simply consider $\rho$ to be the expected correlation between two $e^*$s from the same subject (we assume the correlation across subjects is 0). When $\rho$ is positive, two observations drawn from the same subject tend to be more similar (closer together), on average, than two observations drawn randomly from the dataset while ignoring the clustering due to subjects. When $\rho$ is negative, two observations drawn from the same subject tend to be less similar (further apart), on average, than two observations drawn completely at random. (More information about this interpretation in the question/answers here.)

So now when we look at the equation for the variance of the sample mean under the marginal model, we have

$$\begin{aligned} \text{var}(\frac{1}{nm}\sum_i\sum_jy_{ij}) &= \text{var}(\frac{1}{nm}\sum_i\sum_j\beta + e^*_{ij}) \\ &= \frac{1}{n^2m^2}\text{var}(\sum_i\sum_je^*_{ij}) \\ &= \frac{1}{n^2m^2}\Big(n\big(m\sigma^2 + (m^2-m)\rho\sigma^2\big)\Big) \\ &= \frac{\sigma^2\big(1+(m-1)\rho\big)}{nm} \\ &= \Big(\frac{\rho}{n}+\frac{1-\rho}{nm}\Big)\sigma^2, \end{aligned}$$ which is the same variance expression we derived above for the random-effects model, just with $\sigma^2_e+\sigma^2_u=\sigma^2$, which is consistent with our note above that $e^*_{ij} = u_i + e_{ij}$. The advantage of this (statistically equivalent) perspective is that here we can think about a negative intra-class correlation without needing to invoke any weird concepts like a negative subject variance. Negative intra-class correlations just fit naturally in this framework.

(BTW, just a quick aside to point out that the second-to-last line of the derivation above implies that we must have $\rho \ge -1/(m-1)$, or else the whole equation is negative, but variances can't be negative! So there is a lower bound on the intra-class correlation that depends on how many measurements we have per cluster. For $m=2$ (i.e., we measure each subject twice), the intra-class correlation can go all the way down to $\rho=-1$; for $m=3$ it can only go down to $\rho=-1/2$; and so on. Fun fact!)

So finally, once again considering the total number of observations $nm$ to be a constant, we see that the second-to-last line of the derivation above just looks like

$$\big(1+(m-1)\rho\big) \times \text{positive constant}.$$ So when $\rho>0$, having $m$ as small as possible (so that we take fewer measurements of more subjects--in the limit, 1 measurement of each subject) makes the variance of the estimate as small as possible. But when $\rho<0$, we actually want $m$ to be as large as possible (so that, in the limit, we take all $nm$ measurements from a single subject) in order to make the variance as small as possible. And when $\rho=0$, the variance of the estimate is just a constant, so our allocation of $m$ and $n$ doesn't matter.