2
超参数化模型的Fisher信息矩阵行列式
考虑一个带有参数(成功概率)的伯努利随机变量。似然函数和Fisher信息(矩阵)为:X∈{0,1}X∈{0,1}X\in\{0,1\}θθ\theta1×11×11 \times 1 L1(θ;X)I1(θ)=p(X|θ)=θX(1−θ)1−X=detI1(θ)=1θ(1−θ)L1(θ;X)=p(X|θ)=θX(1−θ)1−XI1(θ)=detI1(θ)=1θ(1−θ) \begin{align} \mathcal{L}_1(\theta;X) &= p(\left.X\right|\theta) = \theta^{X}(1-\theta)^{1-X} \\ \mathcal{I}_1(\theta) &= \det \mathcal{I}_1(\theta) = \frac{1}{\theta(1-\theta)} \end{align} 现在考虑带有两个参数的“过度参数化”版本:成功概率θ1θ1\theta_1和失败概率θ0θ0\theta_0。(请注意θ1+θ0=1θ1+θ0=1\theta_1+\theta_0=1,并且此约束表示参数之一是多余的。)在这种情况下,似然函数和Fisher信息矩阵(FIM)为: L2(θ1,θ0;X)I2(θ1,θ0)detI2(θ)=p(X|θ1,θ0)=θX1θ1−X0=(1θ1001θ0)=1θ1θ0=1θ1(1−θ1)L2(θ1,θ0;X)=p(X|θ1,θ0)=θ1Xθ01−XI2(θ1,θ0)=(1θ1001θ0)detI2(θ)=1θ1θ0=1θ1(1−θ1) \begin{align} \mathcal{L}_2(\theta_1,\theta_0;X) &= p(\left.X\right|\theta_1,\theta_0) = \theta_1^{X}\theta_0^{1-X} \\ \mathcal{I}_2(\theta_1,\theta_0) &= \left( \begin{matrix} \frac{1}{\theta_1} & 0 \\ 0 & \frac{1}{\theta_0} \end{matrix} \right) \\ \det \mathcal{I}_2(\theta) &= \frac{1}{\theta_1 \theta_0} = \frac{1}{\theta_1 (1-\theta_1)} \end{align} …