Smoothed Total Variation

To make Total Variation differentiable, the Euclidean norm $|\nabla z|$ is replaced by a smooth approximation ${\varphi }_{\epsilon }(|\nabla z|)$ with a small $\epsilon >0$:

Charbonnier (pseudo-Huber) smoothing

$${\mathrm{TV}}_{\epsilon }(z)={\int }_{\Omega }\sqrt{|\nabla z{|}^2+{\epsilon }^2}\mathrm{d}x.$$

Huber smoothing (quadratic near zero, linear far away)

$${\varphi }_{\epsilon }(t)=\{\begin{array}{ll}\frac{t^2}{2\epsilon },& t\le \epsilon ,\\ t-\frac{\epsilon }{2},& t>\epsilon ,\end{array}{\mathrm{TV}}_{\epsilon }^{\text{Hub}}(z)={\int }_{\Omega }{\varphi }_{\epsilon }(|\nabla z|)\mathrm{d}x.$$

Both converge to TV as $\epsilon \to 0$. Huber is exactly linear above $\epsilon$ and preserves edges slightly better. Charbonnier is $C^{\infty }$.

Gradient. The first variation of ${\mathrm{TV}}_{\epsilon }$ is, with natural boundary conditions,

$$\nabla {\mathrm{TV}}_{\epsilon }(z)=-\nabla \cdot \left(\frac{\nabla z}{\sqrt{|\nabla z{|}^2+{\epsilon }^2}}\right).$$

In a finite element space this is assembled as the vector $({\int }_{\Omega }\frac{\nabla z\cdot \nabla {\phi }_i}{\sqrt{|\nabla z{|}^2+{\epsilon }^2}}\mathrm{d}x{)}_i$. No second derivatives of $z$ are needed.

Hessian. The Hessian has the structure of a weighted stiffness matrix with a rank-one correction. It is sparse but becomes very ill-conditioned as $\epsilon \to 0$. A common alternative is the lagged diffusivity fixed-point iteration (Vogel & Oman), which freezes the weight $1/\sqrt{|\nabla z{|}^2+{\epsilon }^2}$ at the previous iterate. Quasi-Newton methods such as L-BFGS avoid forming the Hessian altogether.

Choosing $\epsilon$. If $\epsilon$ is too large, the result looks like $H^1$-Tikhonov Regularization and edges blur. If it is too small, the problem becomes stiff and optimisers slow down. Continuation, decreasing $\epsilon$ during the iterations, is common.

In ModularEIT.jl: TotalVariationRegularizer, total_variation.

References

1. P. Charbonnier, L. Blanc-Féraud, G. Aubert, M. Barlaud (1997). Deterministic edge-preserving regularization in computed imaging. IEEE Trans. Image Process. 6(2), 298–311. doi:10.1109/83.551699
2. P. J. Huber (1964). Robust Estimation of a Location Parameter. Ann. Math. Statist. 35(1), 73–101. doi:10.1214/aoms/1177703732
3. C. R. Vogel, M. E. Oman (1996). Iterative Methods for Total Variation Denoising. SIAM J. Sci. Comput. 17(1), 227–238. doi:10.1137/0917016
4. A. Borsic, B. M. Graham, A. Adler, W. R. B. Lionheart (2010). In Vivo Impedance Imaging With Total Variation Regularization. IEEE Trans. Med. Imaging 29(1), 44–54. doi:10.1109/TMI.2009.2022540