Choosing the Regularization Parameter

The parameter $\beta$ in a variational objective balances data fit against prior. If it is too small, the reconstruction fits the noise; if it is too large, detail is lost. Standard selection rules:

Morozov’s discrepancy principle. Choose the largest $\beta$ with

$$\Vert \mathcal{F}({\sigma }_{\beta })-y^{\delta }\Vert \le \tau \delta ,\tau >1\text{ fixed},$$

where $\delta$ is the known noise level: do not fit the data better than the noise allows. It is a regularisation strategy with convergence guarantees. The same rule stops iterative methods (regularising Levenberg–Marquardt, Landweber) early.

L-curve. Plot $\log \Vert \mathcal{F}({\sigma }_{\beta })-y^{\delta }\Vert$ against $\log \mathcal{R}({\sigma }_{\beta })$ for many $\beta$. The curve typically looks like an “L”, and its corner (maximum curvature) is a heuristic choice that needs no noise level.

Generalised cross-validation (GCV) minimises a leave-one-out prediction error estimate. It needs no noise level.

Continuation. In practice $\beta$ is often decreased gradually during the iterations. This also helps nonlinear solvers avoid poor local minima.

In ModularEIT.jl: discrepancy_target.

References

1. H. W. Engl, M. Hanke, A. Neubauer (1996). Regularization of Inverse Problems. Kluwer. doi:10.1007/978-94-009-1740-8
2. P. C. Hansen (1992). Analysis of Discrete Ill-Posed Problems by Means of the L-Curve. SIAM Review 34(4), 561–580. doi:10.1137/1034115
3. G. H. Golub, M. Heath, G. Wahba (1979). Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter. Technometrics 21(2), 215–223. doi:10.1080/00401706.1979.10489751