是否存在两个分布之间的Hellinger距离的无偏估计量？

在一个观察密度为的分布的分布的环境中，我想知道是否存在一个对密度为另一分布即的Hellinger距离的无偏估计量（基于）。 $X_1,\ldots,X_n$ $f$ $X_i$ $f_0$

H (f, f_{0}) = {1 - \int_{X} \sqrt{f (x) f_{0} (x)} d x}^{1 / 2} .

$\mathfrak{H}(f,f_0) = \left\{ 1 - \int_\mathcal{X} \sqrt{f(x)f_0(x)} \text{d}x \right\}^{1/2}\,.$

— 西安
source

因此f0是已知的且固定的。但是f是已知的还是参数家族的，或者您是在非参数框架中使用您从样本中获得的关于f的全部知识来进行此操作的？我认为尝试答案时会有所不同。

— Michael R. Chernick 2012年

@MichaelChernick：假设您对

所有知识

f

$f$ 就是样本

X_{1}, \dots, X_{n}

$X_1,\ldots,X_n$ 。

— 西安

我不认为它已经过计算（如果存在）。如果存在，那么AIC会有一个失散的兄弟。

如果假设

f

$f$ 和

f_{0}

$f_0$ 是离散的，则对这个问题进行攻击似乎是可行的。这导致一个明显的估计量（计算EDF和

之间的Hellinger距离

f_{0}

$f_0$ ）。自举（理论上，不是通过模拟！）将为我们提供可能的偏差的处理方法，以及减少（甚至消除）偏差的方法。我希望通过平方距离而不是距离本身能够成功，因为它在数学上更容易处理。离散

的假设

f

$f$ 在应用中没有问题。无论如何，离散

的空间

f

$f$ 是一个密集子集。

— ub

想到罗森布拉特的证明，没有

“善意”无偏估计量。我们可以克服这个问题并得到

无偏估计量吗？我不知道。

f

$f$

H (f, f_{0})

$H(f,f_0)$

— 2012年

Answers:

对于任何合理的广义非参数分布类别中的，均不存在或无偏估计。 $\mathfrak{H}$ $\mathfrak{H}^2$ $f$

我们可以用以下简单的论证来证明这一点

比克尔和莱曼（1969）。凸族的无偏估计。《数学统计年鉴》，40（5）1523–1535。（项目欧几里得）

固定一些分布，和，其密度分别为，和。让分别表示，并让来的一些估计基于 IID样本。 $F_0$ $F$ $G$ $f_0$ $f$ $g$ $H(F)$ $\mathfrak{H}(f, f_0)$ $\hat H(\mathbf X)$ $H(F)$ $n$ $X_i \sim F$

假设是无偏从以下形式的任何分布样品但是然后 $\hat H$

M_{α} := α F + (1 - α) G .

$M_\alpha := \alpha F + (1 - \alpha) G .$

从而使

必须是在多项式

的度最多

。

\begin{aligned} Q (α) & = H (M_{α}) \\ = \int_{x_{1}} \dots \int_{x_{n}} \hat{H} (X) d M_{α} (x_{1}) \dots d M_{α} (x_{n}) \\ = \int_{x_{1}} \dots \int_{x_{n}} \hat{H} (X) [α d F (x_{1}) + (1 - α) d G (x_{1})] \dots [α d F (x_{n}) + (1 - α) d G (x_{n})] \\ = α^{n} E_{X \sim F^{n}} [\hat{H} (X)] + \dots + (1 - α)^{n} E_{X \sim G^{n}} [\hat{H} (X)], \end{aligned}

$\begin{align} Q(\alpha) &= H(M_\alpha) \\&= \int_{x_1} \cdots \int_{x_n} \hat H(\mathbf X) \,\mathrm{d}M_\alpha(x_1) \cdots\mathrm{d}M_\alpha(x_n) \\&= \int_{x_1} \cdots \int_{x_n} \hat H(\mathbf X) \left[ \alpha \mathrm{d}F(x_1) + (1-\alpha) \mathrm{d}G(x_1) \right] \cdots \left[ \alpha \mathrm{d}F(x_n) + (1-\alpha) \mathrm{d}G(x_n) \right] \\&= \alpha^n \operatorname{\mathbb{E}}_{\mathbf X \sim F^n}[ \hat H(\mathbf X)] + \dots + (1 - \alpha)^n \operatorname{\mathbb{E}}_{\mathbf X \sim G^n}[ \hat H(\mathbf X)] ,\end{align}$

Q (α)

$Q(\alpha)$

α

$\alpha$

n

$n$

现在，让我们专门研究一个合理的情况，并证明相应的不是多项式。 $Q$

让是一些分布这对恒定密度：对所有。（其行为之外的范围并不重要）设是只在支持的一些分布，和一些分布只支持上。 $F_0$ $[-1, 1]$ $f_0(x) = c$ $\lvert x \rvert \le 1$ $F$ $[-1, 0]$ $G$ $[0, 1]$

现在其中

\begin{aligned} Q (α) & = H (m_{α}, f_{0}) \\ = \sqrt{1 - \int_{R} \sqrt{m_{α} (x) f_{0} (x)} d x} \\ = \sqrt{1 - \int_{- 1}^{0} \sqrt{c α f (x)} d x - \int_{0}^{1} \sqrt{c (1 - α) g (x)} d x} \\ = \sqrt{1 - \sqrt{α} B_{F} - \sqrt{1 - α} B_{G}}, \end{aligned}

$\begin{align} Q(\alpha) &= \mathfrak{H}(m_\alpha, f_0) \\&= \sqrt{1 - \int_{\mathbb R} \sqrt{m_\alpha(x) f_0(x)} \mathrm{d}x} \\&= \sqrt{1 - \int_{-1}^0 \sqrt{c \, \alpha f(x)} \mathrm{d}x - \int_{0}^1 \sqrt{c \, (1 - \alpha) g(x)} \mathrm{d}x} \\&= \sqrt{1 - \sqrt{\alpha} B_F - \sqrt{1 - \alpha} B_G} ,\end{align}$

，对于

同样。注意，对于任何具有密度的分布

，

。

B_{F} := \int_{R} \sqrt{f (x) f_{0} (x)} d x

$B_F := \int_{\mathbb R} \sqrt{f(x) f_0(x)} \mathrm{d}x$

B_{G}

$B_G$

B_{F} > 0

$B_F > 0$

B_{G} > 0

$B_G > 0$

F

$F$

G

$G$

$\sqrt{1 - \sqrt{\alpha} B_F - \sqrt{1 - \alpha} B_G}$ $\hat H$ $\mathfrak{H}$ $M_\alpha$

$1 - \sqrt{\alpha} B_F - \sqrt{1 - \alpha} B_G$ $\mathfrak{H}^2$ $M_\alpha$

This excludes pretty much all reasonable nonparametric classes of distributions, except for those with densities bounded below (an assumption nonparametric analyses sometimes make). You could probably kill those classes too with a similar argument by just making the densities constant or something.

— Dougal
source

I don't know how to construct (if it exists) an unbiased estimator of the Hellinger distance. It seems possible to construct a consistent estimator. We have some fixed known density $f_0$ , and a random sample $X_1,\dots,X_n$ from a density $f>0$ . We want to estimate

H (f, f_{0}) = \sqrt{1 - \int_{X} \sqrt{f (x) f_{0} (x)} d x} = \sqrt{1 - \int_{X} \sqrt{\frac{f_{0} (x)}{f (x)}} f (x) d x}

$H(f,f_0) = \sqrt{1 - \int_\mathscr{X} \sqrt{f(x)f_0(x)}\,dx} = \sqrt{1 - \int_\mathscr{X} \sqrt{\frac{f_0(x)}{f(x)}}\;\;f(x)\,dx}$

= \sqrt{1 - E [\sqrt{\frac{f_{0} (X)}{f (X)}}]},

$= \sqrt{1 - \mathbb{E}\left[\sqrt{\frac{f_0(X)}{f(X)}}\;\;\right] }\, ,$ where

X \sim f

$X\sim f$ . By the SLLN, we know that

\sqrt{1 - \frac{1}{n} \sum_{i = 1}^{n} \sqrt{\frac{f_{0} (X_{i})}{f (X_{i})}}} \to H (f, f_{0}),

$\sqrt{1 - \frac{1}{n} \sum_{i=1}^n \sqrt{\frac{f_0(X_i)}{f(X_i)}}} \quad \rightarrow H(f,f_0) \, ,$ almost surely, as

n \to \infty

$n\to\infty$ . Hence, a resonable way to estimate

H (f, f_{0})

$H(f,f_0)$ would be to take some density estimator

\hat{f_{n}}

$\hat{f_n}$ (such as a traditional kernel density estimator) of

f

$f$ , and compute

\hat{H} = \sqrt{1 - \frac{1}{n} \sum_{i = 1}^{n} \sqrt{\frac{f_{0} (X_{i})}{\hat{f_{n}} (X_{i})}}} .

$\hat{H}=\sqrt{1 - \frac{1}{n} \sum_{i=1}^n \sqrt{\frac{f_0(X_i)}{\hat{f_n}(X_i)}}} \, .$

— Zen
source

@Zen: Good point! I consider this answer as the answer because it made me realise

H

$H$ sounds very much like a standard deviation, for which there exists no unbiased estimator. As for the variance of

{\hat{H}}_{n}^{2}

$\hat H^2_n$ , no worries:

E [(\sqrt{f_{0} (X) / f (X)})^{2}] = 1

$\mathbb{E}[(\sqrt{f_0(X)/f(X)})^2]=1$ implies that this estimator has a finite variance.

— Xi'an

Thanks for the clarification about the variance of the estimator, Xi'an!

— Zen

Some work on other consistent estimators: (a) arxiv.org/abs/1707.03083 and related work based on

k

$k$ -NN density estimators; (b) arxiv.org/abs/1402.2966 based on correcting kernel density estimates; (c) ieeexplore.ieee.org/document/5605355 based on a connection to classification. (Many of these are based on samples from both

f

$f$ and

f_{0}

$f_0$ , because that's the work I knew about offhand, but I think there are variants for known

f_{0}

$f_0$ .)

— Dougal