应用于近似奇异系统的迭代线性求解器的停止准则

考虑与几乎奇异这意味着有一个本征值的，这是非常小的。迭代方法的通常停止准则基于残差并认为当且迭代次数为时，迭代可以停止。但是在我们正在考虑的情况下，可能会有较大的误差 $Ax=b$ $A$ $\lambda_0$ $A$ $r_n:=b-Ax_n$ $\|r_n\|/\|r_0\|<tol$ $n$ $v$ 住在与该小特征值有关的本征空间赋予小的残余。假设初始残差大，则可能发生在但误差仍然很大。在这种情况下，更好的错误指示符是什么？是 $\lambda_0$ $Av=\lambda_0v$ $r_0$ $\|r_n\|/\|r_0\|<tol$ $x_n-x$ $\|x_{n}-x_{n-1}\|$ 好候选人？

linear-algebra

— 张慧
source

您可能需要考虑一下“几乎单数”的定义。矩阵

（具有

和

单位矩阵）具有非常小的特征值，但是从单数据的任何矩阵就越高。

I \cdot ϵ

$I \cdot \epsilon$

ϵ ≪ 1

$\epsilon\ll 1$

I

$I$

— David Ketcheson，2012年

另外，

似乎是错误的表示法。

更典型，不是吗？

| | r_{n} / r_{0} | |

$||r_n/r_0||$

| | r_{n} | | / | | r_{0} | |

$||r_n||/||r_0||$

— 比尔·巴特

是的，你是对的，比尔！我会纠正这个错误。

— 张晖

什么

？究竟您的算法是什么？

‖ b - A x ‖ / ‖ b ‖

$\| b - Ax \| / \| b \|$

— shuhalo 2012年

附录：我认为以下论文可以很好地解决您所担心的病态系统，至少在使用CG的情况下：Axelson，Kaporin：错误范数估计和预处理共轭梯度迭代中的终止标准。DOI：10.1002 / nla.244

— shuhalo 2012年

Answers:

请不要使用连续迭代之间的差异来定义停止条件。这会误诊停滞而导致收敛。大多数非对称矩阵迭代不是单调的，甚至在没有重新启动的精确算术中的GMRES可能在突然收敛之前停滞任意数量的迭代（直到矩阵的维数）。参见Nachtigal，Reddy和Trefethen（1993）中的示例。

定义收敛的更好方法

通常，我们对解决方案的准确性比对残差的大小更感兴趣。具体来说，我们可能想保证近似解与精确解之间的差满足对于某些用户指定的。事实证明，通过找到一个实现这个使得其中是的最小奇异值，原因是 $x_n$ $x$

| x_{n} - x | < c

$|x_n - x| < c$

c

$c$

x_{n}

$x_n$

| A x_{n} - b | < c ϵ

$|A x_n - b| < c\epsilon$

ϵ

$\epsilon$

A

$A$

\begin{aligned} | x_{n} - x | & = | A^{- 1} A (x_{n} - x) | \\ \leq \frac{1}{ϵ} | A x_{n} - A x | \\ = \frac{1}{ϵ} | A x_{n} - b | \\ < \frac{1}{ϵ} \cdot c ϵ = c \end{aligned}

$\begin{align} |x_n - x| &= |A^{-1} A (x_n - x)| \\ & \le \frac 1 \epsilon |A x_n - A x| \\ & = \frac 1 \epsilon |A x_n - b| \\ & < \frac 1 \epsilon \cdot c \epsilon = c \end{align}$

其中我们使用是（第二行）的最大奇异值，并且精确地求解 $1/\epsilon$ $A^{-1}$ $x$ （第三行）。 $A x = b$

估计最小奇异值 $\epsilon$

通常无法直接从问题中获得最小奇异值的准确估计，但可以将其估计为共轭梯度或GMRES迭代的副产品。注意，虽然最大的特征值和奇异值的估计通常只有几次迭代后相当不错，最小本征/奇异值的准确估计通常只有一次达到收敛得到。在收敛之前，估计值通常会大大大于真实值。这表明您必须先实际求解方程式，然后才能定义正确的公差 $\epsilon$ 。具有用户提供的精度的自动收敛容差 $c\epsilon$ $c$ 为解决方案和估计最小奇异值因为估计与克雷洛夫方法的当前状态可能会收敛得太早比实际值要大得多。 $\epsilon$ $\epsilon$

笔记

上面的讨论还与替换为左预条件算子和预调节残留或与右预条件算和误差。如果 $A$ $P^{-1}A$ $P^{-1} (A x^n - b)$ $A P^{-1}$ $P (x_n - x)$ $P^{-1}$ 是一个很好的前提条件，前提条件是操作员条件良好。对于左预处理，这意味着可以使预处理残差变小，但实际残差可能不会变小。对于正确的预处理，很容易变小，但真正的错误未必。这就解释了为什么左预处理有利于减小误差，而右预处理有利于减小残差（以及调试不稳定的预处理器）。 $|P(x_n - x)|$ $|x_n-x|$
有关GMRES和CG最小化的规范的更多讨论，请参见此答案。
极值奇异值的估计可以使用 -ksp_monitor_singular_value任何PETSc程序。请参阅KSPComputeExtremeSingularValues（）以从代码计算奇异值。
使用GMRES估计奇异值时，至关重要的是不要使用重启（例如-ksp_gmres_restart 1000在PETSc中）。

— 杰德·布朗
source

“也可以用A替换为预处理算子”-但是，如果使用

，则它仅适用于预处理残差

。到预调节误差

如果

被使用。

P^{- 1} r

$P^{-1}r$

P^{- 1} A

$P^{-1}A$

P^{- 1} δ x

$P^{-1}\delta x$

A P^{- 1}

$AP^{-1}$

— 阿诺德·诺伊迈耶

好点，我编辑了答案。注意，右预处理例给出你控制的

，退绕预条件（施加

）典型地放大在误差低能量模式。

P δ x

$P\delta x$

P^{- 1}

$P^{-1}$

— 杰德·布朗

解决此问题的另一种方法是考虑离散逆问题的工具，即涉及解决或其中非常病态（即，第一和最后一个奇异值之间的比率 $Ax=b$ $\min ||Ax-b||_2$ $A$ $\sigma_1/\sigma_n$ 大）。

在这里，我们有几种选择停止准则的方法，对于迭代方法，我建议使用L曲线准则，因为它仅涉及已经可用的量（免责声明：我的顾问率先采用了这种方法，因此我肯定偏向于它）。我已经在迭代方法中成功使用了它。

$\rho_k=||Ax_k-b||_2$ and the solution norm $\eta_k=||x_k||_2$ , where $x_k$ is the $k$ 'th iterate. As you iterate, this begins to draw the shape of an L in a loglog(rho,eta) plot, and the point at the corner of that L is the optimal choice.

This allows you to implement a criterion where you keep an eye on when you have passed the corner (i.e. looking at the gradient of $(\rho_k,\eta_k)$ ), and then choose the iterate that was located at the corner.

The way I did it involved storing the last 20 iterates, and if the gradient $abs(\frac{\log(\eta_k)-\log(\eta_{k-1})}{\log(\rho_k)-\log(\rho_{k-1})})$ was larger than some threshold for 20 successive iterations, I knew that I was on the vertical part of the curve and that I had passed the corner. I then took the first iterate in my array (i.e. the one 20 iterations ago) as my solution.

There are also more detailed methods for finding the corner, and these work better but require storing a significant number of iterates. Play around with it a bit. If you are in matlab, you can use the toolbox Regularization Tools, which implements some of this (specifically the "corner" function is applicable).

Note that this approach is particularly suitable for large-scale problems, since the extra computing time involved is minuscule.

— OscarB
source

Thanks a lot! So in loglog(rho,eta) plot we begin from the right of the L curve and end at the top of L, is it? I just do not know the principle behind this criterion. Can you explain why it always behave like an L curve and why we choose the corner?

— Hui Zhang

You're welcome :-D. For an iterative method, we begin from right and end at top always. It behaves as an L due to the noise in the problem - the vertical part happens at

| | A x - b | |_{2} = | | e | |_{2}

$||Ax-b||_2=||e||_2$ , where

e

$e$ is the noise vector

b_{e x a c t} = b + e

$b_{exact}=b+e$ . For more analysis see Hansen, P. C., & O'Leary, D. P. (1993). The use of the L-curve in the regularization of discrete ill-posed problems. SIAM Journal on Scientific Computing, 14. Note that I just made a slight update to the post.

— OscarB

@HuiZhang: it isn't always an L. If the regularization is ambiguous it may be a double L, leading to two candidates for the solution, one with gross featurse better resolved, the other with certain details better resolved. (And of course, mor ecomplex shapes may appear.)

— Arnold Neumaier

Does the L-curve apply to ill-conditioned problems where there should be a unique solution? That is, I'm interested in problems Ax = b where b is known "exactly" and A is nearly singular but still technically invertible. It would seem to me that if you use something like GMRES the norm of your current guess of x doesn't change too much over time, especially after the first however many iterations. It seems to me that the vertical part of the L-curve occurs because there is no unique/valid solution in an ill-posed problem; would this vertical feature be present in all ill-conditioned problems?

— nukeguy

At one point, you will reach such a vertical line, typically because the numerical errors in your solution method result in ||Ax-b|| not decreasing. However, you are right that in such noise-free problems the curve does not always look like an L, meaning that you typically have a few corners to choose from and choosing one over the other can be hard. I believe that the paper I referenced in my comment above discusses noise-free scenarios briefly.

— OscarB

应用于近似奇异系统的迭代线性求解器的停止准则

定义收敛的更好方法

估计最小奇异值ϵϵ\epsilon

笔记

估计最小奇异值 $\epsilon$