Reynolds Operator

The Reynolds operator turns an arbitrary function into an invariant one by averaging over a group $G$ with Haar measure $\mu$:

$$({\mathcal{P}}_{G}f)(x)=\frac{1}{\mu (G)}{\int }_{G}f({\rho }_X(g)x)\mathrm{d}g.$$

The equivariant version also transforms the output back:

$$({\mathcal{P}}_G^{\text{eq}}f)(x)=\frac{1}{\mu (G)}{\int }_G{\rho }_Y(g{)}^{-1}f({\rho }_X(g)x)\mathrm{d}g.$$

For a finite group, the integral becomes a sum. For the Dihedral Group D4:

$$({\mathcal{P}}_{D_4}f)(x)=\frac{1}{8}\sum _{g\in D_4}f({\rho }_X(g)x).$$

Properties.

• ${\mathcal{P}}_{G}f$ is invariant (respectively equivariant), and ${\mathcal{P}}_{G}f=f$ if $f$ already is. So ${\mathcal{P}}_G$ is a projection.
• For unitary representations, ${\mathcal{P}}_G$ is the orthogonal projection in $L^2$ onto the subspace of invariant (equivariant) functions. It gives the best invariant approximation of $f$ in the $L^2$ sense.

Uses. Symmetrising convolution filters to obtain invariant filters (see Invariant Filter Banks). Test-time augmentation, which averages a network’s predictions over transformed inputs. Symmetrising learned priors or denoisers so that they respect the Symmetries of the EIT Problem. The cost grows with $|G|$, while constrained architectures (see Equivariant Convolutions) have equivariance built in at no extra runtime cost.

References

1. M. Reisert, H. Burkhardt (2007). Learning Equivariant Functions with Matrix Valued Kernels. J. Mach. Learn. Res. 8, 385–408. jmlr.org/papers/v8/reisert07a.html
2. B. Sturmfels (2008). Algorithms in Invariant Theory, 2nd ed. Springer. doi:10.1007/978-3-211-77417-5
3. M. Finzi, M. Welling, A. G. Wilson (2021). A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups. ICML 2021. arXiv:2104.09459