截断分布的最大似然估计

考虑从随机变量获得的独立样本，假定该随机变量遵循已知（有限）最小值和最大值和的截断分布（例如，截断的正态分布），但是参数和未知。如果遵循非截短的分布中，最大似然估计和为和从将样本均值$N$$S$$X$$a$$b$$\mu$$\sigma^2$$X$$\widehat\mu$$\widehat\sigma^2$$\mu$$\sigma^2$$S$$\widehat\mu = \frac{1}{N} \sum_i S_i$和样本方差。但是，对于截断的分布，以这种方式定义的样本方差以为界，因此它并不总是一致的估计量：对于，它不可能收敛到当达到无穷大时，。因此，对于，和似乎不是和的最大似然估计。当然，这是可以预期的，因为和$\widehat\sigma^2 = \frac{1}{N} \sum_i (S_i - \widehat\mu)^2$$(b-a)^2$$\sigma^2 > (b-a)^2$$\sigma^2$$N$$\widehat\mu$$\widehat\sigma^2$$\mu$$\sigma^2$$\mu$$\sigma^2$ 截断正态分布的参数不是其均值和方差。

那么，已知最小值和最大值的截断分布的和参数的最大似然估计是多少？$\mu$$\sigma$

考虑由“标准”分布确定的任何位置范围的族。$F$

$$\Omega_F = \left\{F_{(\mu, \sigma)}: x \to F\left(\frac{x-\mu}{\sigma}\right) \mid \sigma \gt 0\right\}.$$

假设微的，我们很容易发现PDF为。$F$$\frac{1}{\sigma}f\left((x-\mu)/\sigma\right)dx$

截断这些分布以将它们的支持限制在和之间，，意味着将PDF替换为$a$$b$$a \lt b$

$$f_{(\mu, \sigma; a,b)}(x) = \frac{f\left(\frac{x-\mu}{\sigma}\right)dx}{\sigma C(\mu, \sigma, a, b)}, a \le x \le b$$

(and are zero for all other values of $x$) where $C(\mu, \sigma, a, b) = F_{(\mu,\sigma)}(b) - F_{(\mu,\sigma)}(a)$ is the normalizing factor needed to ensure that $f_{(\mu, \sigma; a, b)}$ integrates to unity. (Note that $C$ is identically $1$ in the absence of truncation.) The log likelihood for iid data $x_i$ therefore is

$$\Lambda(\mu, \sigma) = \sum_i \left[\log{f\left(\frac{x_i-\mu}{\sigma}\right)} - \log{\sigma}-\log{C(\mu, \sigma, a, b)}\right].$$

Critical points (including any global minima) are found where either $\sigma=0$ (a special case I will ignore here) or the gradient vanishes. Using subscripts to denote derivatives, we may formally compute the gradient and write the likelihood equations as

$$\eqalign{ 0 &= \frac{\partial\Lambda}{\partial\mu} &= \sum_i \left[\frac{-f_\mu\left(\frac{x_i-\mu}{\sigma}\right)}{f\left(\frac{x_i-\mu}{\sigma}\right)} -\frac{C_\mu(\mu,\sigma,a,b)}{C(\mu,\sigma,a,b)}\right] \\ 0 &= \frac{\partial\Lambda}{\partial\sigma} &= \sum_i \left[\frac{-f_\sigma\left(\frac{x_i-\mu}{\sigma}\right)}{\sigma^2f\left(\frac{x_i-\mu}{\sigma}\right)} -\frac{1}{\sigma}-\frac{C_\sigma(\mu,\sigma,a,b)}{C(\mu,\sigma,a,b)}\right] }$$

Because $a$ and $b$ are fixed, drop them from the notation and write $nC_\mu(\mu, \sigma, a, b)/C(\mu, \sigma,a,b)$ as $A(\mu,\sigma)$ and $nC_\sigma(\mu, \sigma, a, b)/C(\mu, \sigma,a,b)$ as $B(\mu, \sigma)$. (With no truncation, both functions would be identically zero.) Separating the terms involving the data from the rest gives

$$\eqalign{ -A(\mu,\sigma) &= \sum_i \frac{f_\mu\left(\frac{x_i-\mu}{\sigma}\right)}{f\left(\frac{x_i-\mu}{\sigma}\right)} \\ -\sigma^2 B(\mu,\sigma) - n\sigma &= \sum_i \frac{f_\sigma\left(\frac{x_i-\mu}{\sigma}\right)}{f\left(\frac{x_i-\mu}{\sigma}\right)} }$$

By comparing these to the no-truncation situation it is evident that

• Any sufficient statistics for the original problem are sufficient for the truncated problem (because the right hand sides have not changed).

• Our ability to find closed-form solutions relies on the tractability of $A$ and $B$. If these do not involve $\mu$ and $\sigma$ in simple ways, we cannot hope to obtain closed-form solutions in general.

For the case of a normal family, $C(\mu,\sigma,a,b)$ of course is given by the cumulative normal PDF, which is a difference of error functions: there is no chance that a closed-form solution can be obtained in general. However, there are only two sufficient statistics (the sample mean and variance will do) and the CDF is as smooth as can be, so numerical solutions will be relatively easy to obtain.