Stopping Criteria

Common rules for terminating a reconstruction:

• Discrepancy principle: stop when $\Vert \mathcal{F}({\sigma }_k)-y^{\delta }\Vert \le \tau \delta$ ($\tau \gtrsim 1$, $\delta$ the noise level). Iterating further fits noise, so for ill-posed problems this is the principled rule (see Choosing the Regularization Parameter and Implicit Regularization).
• Stationarity: $\Vert \nabla \Phi ({\sigma }_k)\Vert \le {\epsilon }_{\text{abs}}+{\epsilon }_{\text{rel}}\Vert \nabla \Phi ({\sigma }_0)\Vert$. For bound constraints use the projected gradient $\Vert P_{\Sigma }(\sigma -\nabla \Phi )-\sigma \Vert$ (see KKT Conditions).
• Stagnation: relative change of the objective $|{\Phi }_k-{\Phi }_{k+1}|/|{\Phi }_k|$ or of the iterate $\Vert {\sigma }_{k+1}-{\sigma }_k\Vert /\Vert {\sigma }_k\Vert$ below a tolerance.
• Stagnation at the noise level: a step that decreases the whitened misfit by less than a few standard deviations $\sqrt{m/2}$ of its noise-only value is rejected, and the previous iterate returned. It needs no target value, so it also works when the noise model is only estimated (see Noise-Level Stagnation Test).
• Splitting methods: primal and dual residuals of ADMM below tolerance.
• Budget: a maximal number of iterations or forward solves.

Tolerances of the inner linear solves should be tied to the outer progress. Solving to $10^{-12}$ when the gradient is still large wastes effort, while solving too loosely produces gradient noise that stalls line searches.

In ModularEIT.jl: minimize, OptimizationState, discrepancy_target.

References

1. J. Nocedal, S. J. Wright (2006). Numerical Optimization, 2nd ed. Springer. doi:10.1007/978-0-387-40065-5
2. H. W. Engl, M. Hanke, A. Neubauer (1996). Regularization of Inverse Problems. Kluwer. doi:10.1007/978-94-009-1740-8
3. M. Hanke (1997). A regularizing Levenberg–Marquardt scheme, with applications to inverse groundwater filtration problems. Inverse Problems 13(1), 79–95. doi:10.1088/0266-5611/13/1/007