Tweedie’s Formula as the Rosetta Stone of Score Matching, Diffusion, DDIM, and CFM
A mathematical walk from empirical Bayes denoising to modern generative modeling: score matching, diffusion \(x_0\)-, \(\epsilon\)-, \(v\)-prediction, DDIM, reverse SDEs, probability-flow ODEs, and conditional flow matching.
Core thesis. Tweedie’s formula says that denoising, score estimation, noise prediction, velocity prediction, DDIM transport, and CFM vector fields are different coordinate systems for the same posterior geometry induced by Gaussian corruption.
Posterior denoising
The central object is \(\mathbb E[x_0 \mid x_t]\), the clean posterior mean under Gaussian corruption.
Score geometry
The score is the correction field that moves noisy points toward that posterior mean.
Sampler coordinates
Diffusion heads, DDIM, reverse SDEs, ODEs, and CFM velocities reuse the same geometry.
Contents
The Central Question
Suppose the world gives us clean data \(x_0 \sim p_0(x_0)\), but we only observe a noised version
\[ x_t = \alpha_t x_0 + \sigma_t \epsilon, \qquad \epsilon\sim\mathcal N(0,I). \]
Diffusion models, score models, DDIM samplers, and flow matching models differ in how they parameterize the neural network and how they sample. But at the heart of all of them is one posterior object:
\[ \mathbb E[x_0\mid x_t]. \]
Tweedie’s formula says that this posterior denoised estimate can be written using the marginal score \(s_t(x_t)=\nabla_{x_t}\log p_t(x_t)\):
\[ \boxed{ \mathbb E[x_0\mid x_t] = \frac{x_t+\sigma_t^2\nabla_{x_t}\log p_t(x_t)}{\alpha_t}. } \]
The rest of this note is just unpacking this one identity until many seemingly different objectives become the same object in different coordinates.
Deriving Tweedie’s Formula
Let
\[ p_t(x_t\mid x_0)=\mathcal N(x_t;\alpha_t x_0,\sigma_t^2 I). \]
The marginal noisy density is
\[ p_t(x_t)=\int p_t(x_t\mid x_0)p_0(x_0)\,dx_0. \]
Take the score of the marginal:
\[ \nabla_{x_t}\log p_t(x_t) = \frac{\nabla_{x_t}p_t(x_t)}{p_t(x_t)}. \]
Move the derivative inside the integral:
\[ \nabla_{x_t}p_t(x_t) = \int \nabla_{x_t}p_t(x_t\mid x_0)p_0(x_0)\,dx_0. \]
For the Gaussian conditional,
\[ \nabla_{x_t}\log p_t(x_t\mid x_0) = -\frac{x_t-\alpha_t x_0}{\sigma_t^2}. \]
Therefore
\[ \nabla_{x_t}\log p_t(x_t) = \mathbb E\left[ -\frac{x_t-\alpha_t x_0}{\sigma_t^2} \,\middle|\,x_t \right]. \]
Since \(x_t\) is fixed inside the conditional expectation,
\[ \nabla_{x_t}\log p_t(x_t) = -\frac{x_t-\alpha_t\mathbb E[x_0\mid x_t]}{\sigma_t^2}. \]
Rearrange:
\[ \boxed{ \mathbb E[x_0\mid x_t] = \frac{x_t+\sigma_t^2s_t(x_t)}{\alpha_t}. } \]
Interpretation. The score is not merely a gradient field for sampling. It is also the correction term that moves a noisy point toward its posterior clean mean.
\[ \mathbb E[x_0\mid x_t]-\frac{x_t}{\alpha_t} = \frac{\sigma_t^2}{\alpha_t}s_t(x_t). \]
Interactive Plot 1: Tweedie Denoising in 1D
We use a one-dimensional mixture prior. Move the observation \(y=x_t\) and noise level \(\sigma\) to see the posterior mean \(\mathbb E[x_0\mid y]\), the marginal score, and Tweedie’s identity.
The Same Estimator in Different Diffusion Parameterizations
Modern diffusion papers often choose different neural-network outputs. But if the forward noising process is Gaussian, these outputs are algebraically convertible.
\[ x_t=\alpha_t x_0+\sigma_t\epsilon, \qquad \epsilon\sim\mathcal N(0,I). \]
| Parameterization | What the network learns | Tweedie interpretation | Implied score |
|---|---|---|---|
| \(x_0\)-prediction | \(x_\theta(x_t,t)\approx x_0\) | \(x_\theta\approx\mathbb E[x_0\mid x_t]\) | \(s_\theta=\dfrac{\alpha_t x_\theta-x_t}{\sigma_t^2}\) |
| \(\epsilon\)-prediction | \(\epsilon_\theta(x_t,t)\approx\epsilon\) | \(\epsilon_\theta\approx\mathbb E[\epsilon\mid x_t] = -\sigma_t s_t(x_t)\) | \(s_\theta=-\dfrac{\epsilon_\theta}{\sigma_t}\) |
| score prediction | \(s_\theta(x_t,t)\approx\nabla_{x_t}\log p_t(x_t)\) | directly learns Tweedie correction field | \(s_\theta\) |
| \(v\)-prediction | \(v_\theta\) in an orthogonal coordinate system | learns a rotation of clean and noise posteriors | see below |
Noise Prediction Is Posterior Residual Prediction
From the forward equation,
\[ \mathbb E[\epsilon\mid x_t] = \frac{x_t-\alpha_t\mathbb E[x_0\mid x_t]}{\sigma_t}. \]
Plug in Tweedie:
\[ \boxed{ \mathbb E[\epsilon\mid x_t]= -\sigma_t s_t(x_t). } \]
So \(\epsilon\)-prediction is not arbitrary. It is a scaled score prediction.
Diffusion v-Prediction: A Rotation of Clean and Noise
A common continuous-time convention writes
\[ x_t=\alpha_t x_0+\sigma_t\epsilon, \qquad \alpha_t^2+\sigma_t^2=1. \]
The velocity-style diffusion target is often
\[ v_t=\alpha_t\epsilon-\sigma_t x_0. \]
This is a rotation of \((x_0,\epsilon)\). The inverse rotation is
\[ \boxed{x_0=\alpha_t x_t-\sigma_t v_t}, \qquad \boxed{\epsilon=\sigma_t x_t+\alpha_t v_t}. \]
Taking conditional expectations gives the Tweedie interpretation:
\[ v_\theta(x_t,t) \approx \mathbb E[v_t\mid x_t] = \alpha_t\mathbb E[\epsilon\mid x_t] -\sigma_t\mathbb E[x_0\mid x_t]. \]
Using \(\mathbb E[\epsilon\mid x_t]= -\sigma_t s_t\) and Tweedie for \(\mathbb E[x_0\mid x_t]\),
\[ \boxed{ v_t^{\star}(x_t) = -\frac{\sigma_t}{\alpha_t}x_t -\frac{\sigma_t}{\alpha_t}s_t(x_t) } \quad \text{when }\alpha_t^2+\sigma_t^2=1. \]
More usefully, the conversions are:
\[ \boxed{\hat x_0=\alpha_t x_t-\sigma_t v_\theta}, \qquad \boxed{\hat\epsilon=\sigma_t x_t+\alpha_t v_\theta}, \qquad \boxed{s_\theta=-\frac{\sigma_t x_t+\alpha_t v_\theta}{\sigma_t}}. \]
Notation warning. Diffusion \(v\)-prediction and CFM velocity prediction are related in spirit but not the same object. Diffusion \(v\) is usually a rotation of clean/noise variables; CFM velocity is a path derivative \(\dot x_t\).
Interactive Plot 2: Convert One Prediction Head into All Others
Choose \(t\) and a scalar noisy point. We use a simple VP schedule \(\alpha=\cos(\frac{\pi t}{2})\), \(\sigma=\sin(\frac{\pi t}{2})\) and an analytic mixture score. The table updates all equivalent outputs.
Conditional Flow Matching through the Same Lens
For the simple Gaussian-to-data linear interpolation used in many CFM explanations, write
\[ x_t=(1-t)x_0+t x_1, \]
where \(x_0\sim\mathcal N(0,I)\) is noise and \(x_1\sim p_{\text{data}}\) is clean data. This is the same Gaussian corruption form, just with the clean endpoint called \(x_1\):
\[ x_t=t x_1+(1-t)\epsilon, \qquad \alpha_t=t, \quad \sigma_t=1-t. \]
Tweedie gives the clean endpoint posterior:
\[ \boxed{ \mathbb E[x_1\mid x_t] = \frac{x_t+(1-t)^2s_t(x_t)}{t}. } \]
The CFM target velocity is the conditional mean path derivative:
\[ u_t(x_t) = \mathbb E[\dot x_t\mid x_t] = \mathbb E[x_1-x_0\mid x_t]. \]
Because \(x_0=(x_t-tx_1)/(1-t)\),
\[ x_1-x_0=\frac{x_1-x_t}{1-t}. \]
Therefore
\[ \boxed{ u_t(x_t) = \frac{\mathbb E[x_1\mid x_t]-x_t}{1-t}. } \]
Plug in Tweedie:
\[ \boxed{ u_t(x_t)=\frac{x_t}{t}+\frac{1-t}{t}s_t(x_t) }, \qquad \boxed{ s_t(x_t)=\frac{t u_t(x_t)-x_t}{1-t}. } \]
And the endpoints implied by a CFM velocity are
\[ \boxed{\hat x_1=x_t+(1-t)u_t(x_t)}, \qquad \boxed{\hat x_0=x_t-t u_t(x_t)}. \]
CFM interpretation. CFM velocity prediction learns the posterior mean displacement from the noise endpoint to the clean endpoint. Score matching learns the same thing after a time-dependent affine change of coordinates.
DDIM as Deterministic Posterior-Mean Transport
In DDPM, the reverse process is usually stochastic. DDIM showed that one can construct a non-Markovian reverse process with the same training objective but deterministic or partially stochastic sampling. The key practical formula starts from a predicted clean estimate \(\hat x_0\) and predicted noise \(\hat\epsilon\):
\[ \hat x_0= \frac{x_t-\sqrt{1-\bar\alpha_t}\,\epsilon_\theta(x_t,t)} {\sqrt{\bar\alpha_t}}, \qquad \hat\epsilon=\epsilon_\theta(x_t,t). \]
The deterministic DDIM step from time \(t\) to \(s<t\) is
\[ \boxed{ x_s= \sqrt{\bar\alpha_s}\,\hat x_0 + \sqrt{1-\bar\alpha_s}\,\hat\epsilon. } \]
Through Tweedie, \(\hat x_0\) is a posterior clean mean, while \(\hat\epsilon=-\sigma_t s_t\) is a posterior residual/score estimate. DDIM therefore reuses the same estimated clean/noise coordinates and moves them to a lower noise level.
Interactive Plot 3: Deterministic DDIM vs Stochastic Reverse Steps
This toy 1D plot uses the analytic score of a mixture distribution. Deterministic paths are smooth; stochastic paths add fresh noise during reverse sampling.
Reverse SDE vs Probability-Flow ODE
The score-SDE framework starts from a forward SDE
\[ dx=f(x,t)dt+g(t)dW_t. \]
The reverse-time SDE is
\[ \boxed{ dx= \left[ f(x,t)-g(t)^2\nabla_x\log p_t(x) \right]dt + g(t)d\bar W_t, } \]
where time is run backward. The probability-flow ODE has the same marginal densities \(p_t\) but no Brownian noise:
\[ \boxed{ dx= \left[ f(x,t)-\frac{1}{2}g(t)^2\nabla_x\log p_t(x) \right]dt. } \]
Both require the same score. Tweedie tells us that this score is equivalent to a posterior denoiser. Thus:
| Sampler | Uses | Same marginal path? | Intuition |
|---|---|---|---|
| Reverse SDE | score + stochastic noise | Yes | remove noise while injecting calibrated randomness |
| Probability-flow ODE | score only | Yes | deterministic transport through same marginals |
| DDIM with \(\eta=0\) | predicted \(\hat x_0\), \(\hat\epsilon\) | discrete deterministic analogue | moves in clean/noise coordinates |
| CFM ODE | velocity field \(u_t\) | depends on chosen path | learns deterministic vector field directly |
Fisher Information: The Energy of the Score Field
The Fisher information of the noisy marginal is
\[ \mathcal I(t) = \mathbb E_{x_t\sim p_t} \left[ \|\nabla_{x_t}\log p_t(x_t)\|^2 \right]. \]
In Gaussian denoising, Tweedie gives
\[ s_t(x_t) = \frac{\alpha_t\mathbb E[x_0\mid x_t]-x_t}{\sigma_t^2} = -\frac{\mathbb E[\epsilon\mid x_t]}{\sigma_t}. \]
So Fisher information measures how much posterior residual information remains in \(x_t\). Large score norm means the noisy marginal has sharp structure; small score norm means the distribution is closer to a broad Gaussian.
Useful identity. For Gaussian noise corruption \(Y=X+\sigma Z\), de Bruijn’s identity links entropy growth to Fisher information:
\[ \frac{d}{d\tau}h(X+\sqrt{\tau}Z) = \frac{1}{2}\mathcal I(X+\sqrt{\tau}Z). \]
As you inject Gaussian noise, entropy increases at a rate controlled by Fisher information. This is one reason Fisher information naturally appears in score-based generative modeling.
Interactive Plot 4: Fisher Information vs Noise
For the same 1D mixture prior, we numerically integrate \(\mathcal I(\sigma)=\mathbb E[s_\sigma(Y)^2]\). Larger noise smooths the distribution and usually reduces sharp score structure.
The Whole Unification in One Table
| View | Network target | Tweedie meaning | Conversion to score |
|---|---|---|---|
| Score matching | \(s_\theta\) | learns correction toward posterior mean | \(s_\theta\) |
| Diffusion \(x_0\)-prediction | \(x_\theta\) | \(\mathbb E[x_0\mid x_t]\) | \(\dfrac{\alpha_t x_\theta-x_t}{\sigma_t^2}\) |
| Diffusion \(\epsilon\)-prediction | \(\epsilon_\theta\) | \(\mathbb E[\epsilon\mid x_t]\) | \(-\dfrac{\epsilon_\theta}{\sigma_t}\) |
| Diffusion \(v\)-prediction | \(v_\theta=\alpha_t\epsilon-\sigma_t x_0\) | rotated posterior clean/noise coordinate | \(-\dfrac{\sigma_t x_t+\alpha_t v_\theta}{\sigma_t}\), for VP convention |
| DDIM | \(\hat x_0,\hat\epsilon\) reused across times | deterministic movement in Tweedie clean/noise coordinates | same as \(x_0\) or \(\epsilon\) prediction |
| Reverse SDE | \(s_\theta\) | stochastic denoising dynamics | direct |
| Probability-flow ODE | \(s_\theta\) | deterministic transport with same marginals | direct |
| CFM velocity | \(u_\theta\approx\mathbb E[\dot x_t\mid x_t]\) | posterior mean endpoint displacement | for \(x_t=(1-t)x_0+t x_1\): \(\dfrac{t u_\theta-x_t}{1-t}\) |
One-sentence takeaway. Tweedie’s formula turns many generative-modeling objects into translations of a single posterior-denoising identity.
Overall Map
References and Further Reading
- Jonathan Ho, Ajay Jain, Pieter Abbeel. Denoising Diffusion Probabilistic Models, NeurIPS 2020. arXiv:2006.11239.
- Jiaming Song, Chenlin Meng, Stefano Ermon. Denoising Diffusion Implicit Models, ICLR 2021. arXiv:2010.02502.
- Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, Ben Poole. Score-Based Generative Modeling through Stochastic Differential Equations, ICLR 2021. arXiv:2011.13456.
- Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, Matthew Le. Flow Matching for Generative Modeling, ICLR 2023. arXiv:2210.02747.
- Alexandre Thiery. Reverse Diffusions, Score and Tweedie. Blog note.
- For recent generalizations: Tweedie’s Formulae and Diffusion Generative Models Beyond Gaussian. arXiv:2605.19391.