DDPM Forward Process

The denoising diffusion probabilistic model (DDPM) uses a Markov chain that gradually adds Gaussian noise to a data point $x_0$:

$$x_t=\sqrt{1-{\beta }_t}{x}_{t-1}+\sqrt{{\beta }_t}{\epsilon }_t,{\epsilon }_t\sim \mathcal{N}(0,I),t=1,\dots ,T,$$

with a Noise Schedule $0<{\beta }_1<\cdots <{\beta }_T<1$.

Closed-form marginals. With ${\alpha }_t=1-{\beta }_t$ and ${\bar{\alpha }}_t={\prod }_{s=1}^t{\alpha }_s$, compounding the Gaussian steps gives

$$q(x_t\mid x_0)=\mathcal{N}(\sqrt{{\bar{\alpha }}_t}{x}_0,(1-{\bar{\alpha }}_t)I),x_t=\sqrt{{\bar{\alpha }}_t}{x}_0+\sqrt{1-{\bar{\alpha }}_t}\epsilon .$$

So any noise level can be sampled in one step, without simulating the chain:

function ForwardSample(x0, t)
    ε  ← sample N(0, I)
    xt ← sqrt(ᾱ_t) · x0 + sqrt(1 − ᾱ_t) · ε
    return xt, ε

Variance preserving. If $\mathrm{Var}[x_0]=I$, then $\mathrm{Var}[x_t]={\bar{\alpha }}_{t}I+(1-{\bar{\alpha }}_t)I=I$ for all $t$. The signal is replaced by noise without changing the overall scale. For ${\bar{\alpha }}_T\approx 0$, $x_T$ is essentially standard Gaussian.

Posterior of one step. Given both $x_0$ and $x_t$, the previous state is Gaussian as well (see DDPM Ancestral Sampling). The continuous-time version of the chain is the Variance-Preserving SDE (see Continuous Limit of the DDPM Chain).

References

1. J. Ho, A. Jain, P. Abbeel (2020). Denoising Diffusion Probabilistic Models. NeurIPS 33. arXiv:2006.11239
2. J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, S. Ganguli (2015). Deep Unsupervised Learning using Nonequilibrium Thermodynamics. ICML 2015. arXiv:1503.03585