Diffusion Conventions and Samplers

Read three diffusion papers and you will meet three clocks, three ways to write the noise and three things the network is asked to predict. They describe the same machine, but each paper uses its own words. This lesson gives you a translation table, shows that a training schedule, a loss weight and a sampler are three separate dials, and then walks through how the common samplers take their steps. Every number is checked by code you can run on this page.

Recap of earlier lessons. Diffusion Models trains a network to predict the noise and samples with DDPM, DDIM and guidance. Score-Based Models shows the same idea as a field that points toward likelier data. Flow Matching trains a velocity along straight paths. Read those first if any of the three is new. This lesson adds no new model. It connects the ones you already know.

Learning Objectives

After this lesson, you will be able to:

Translate one noise level between the DDPM, EDM, flow matching, score-based and v-prediction conventions, and check the translation with code
Say which way time runs in each convention, and what the network is asked to predict in each
Write classifier-free guidance in the track's one convention and map it to a paper's w and to a library's guidance scale
Explain why the noise schedule, the loss weight and the sampler are three separate dials, and say what each one changes
Describe DDPM ancestral sampling, DDIM with its eta dial, and Euler and Heun steps on the probability-flow ODE
Read a run of errors against step count and tell a first-order method from a second-order one

Prerequisites

Before You Start

Review

#Why Every Paper Looks Different

Start with a small confusion that costs people days. You read that a model predicts "the noise". You open another repository and the network predicts a "velocity". A third one says "sigma" runs from 80 down to 0.002, while your own notes say time runs from 0 up to 1000. Are these the same model? Often yes. The picture being made is the same, and only the labels on the clock and the thing the network outputs have changed.

Three questions settle almost every case.

Which way does time run? In the diffusion lessons, t = 0 is clean data and t = T is pure noise. Some papers run it the other way, with t = 0 as pure noise. Some skip time and name the noise level directly.

How is the noisy picture scaled? In the DDPM form the signal is shrunk while the noise is added, so the picture never gets brighter. In the EDM form the picture is left alone and noise is simply piled on top.

What does the network output? The noise, the clean picture, a velocity, or a score. These are the same information in different clothes, and you can convert between them when you know the noisy input and the noise level.

Words you need8 terms

How much of the noisy picture is noise and how much is signal. Papers name it with t, sigma, alpha_bar or a flow time.

The share of signal power left at a step in the DDPM form. Near 1 means almost clean, near 0 means almost pure noise.

Signal power divided by noise power. It is the one number that every convention can be turned into.

In the EDM and score-based forms, the size of the noise added to an unscaled clean picture.

The direction and speed a point moves along its path from noise to data. In v-prediction and in flow matching it is also what the network predicts.

An ordinary differential equation: a rule that gives the slope at each point, so you can walk along a path by following slopes.

The recipe that turns the network's guesses into steps from noise to a picture.

Number of function evaluations: how many times the network is called. It is the real cost of sampling.

#The Rosetta Table

Here is one noise level written five ways. Rows are the questions above. Columns are the conventions you will meet. The symbols are the ones the sources use, with the data written x0 and the noise written eps so that the rows line up.

Question	DDPM (Ho et al. 2020)	Score-based (Song et al. 2021)	EDM (Karras et al. 2022)	v-prediction (Salimans and Ho 2022)	Flow matching, SD3 (Esser et al. 2024)
Clock	integer t, 0 is data, T is noise	continuous t, 0 is data, T is noise	the noise level `sigma` itself	angle `phi`, or the same t as DDPM	continuous t, 0 is data, 1 is noise
Noisy picture	`sqrt(ab) x0 + sqrt(1-ab) eps`	`x0 + sigma_t eps` (variance-exploding) or a scaled form	`x0 + sigma eps`	`cos(phi) x0 + sin(phi) eps`	`(1-t) x0 + t eps`
Sampling goes	t from T down to 0	t from T down to 0	`sigma` from large down to small	`phi` from the noise end to the data end	t from 1 down to 0
Network predicts	`eps`	the score, about `-eps / sigma_t`	a scaled clean guess through `c_skip` and `c_out`	`v = cos(phi) eps - sin(phi) x0`	velocity `eps - x0`
Noise-level input	integer t	t	`c_noise = ln(sigma) / 4`	t or `phi`	t

Three honest notes on that table. First, the "Score-based" column describes the variance-exploding kind. The variance-preserving kind in the same paper is the DDPM form in continuous time. Second, EDM wraps the network in fixed scale factors, and the next section shows why. Third, the flow column follows the SD3 paper. The flow matching lesson mostly uses the opposite clock, with t = 0 as noise, and says so where it happens.

Every column above can be turned into the signal-to-noise ratio, the squared signal weight over the squared noise weight. In the DDPM form that is ab / (1 - ab). In EDM it is 1 / sigma^2. In the flow form it is ((1-t) / t)^2. Two conventions at the same signal-to-noise ratio show the same amount of structure, so the ratio is the safe way to line them up.

#Hand trace: one noise level, five names

Pick the DDPM step where alpha_bar = 0.25. Work with plain numbers and no code yet.

The signal weight is a = sqrt(0.25) = 0.5. The noise weight is b = sqrt(0.75) = 0.866. The noisy picture is 0.5 x0 + 0.866 eps. The two weights squared add to 1, which is what "variance-preserving" means.
EDM leaves the picture unscaled. Divide the noisy picture by 0.5 and you get x0 + 1.732 eps. So sigma = 0.866 / 0.5 = 1.732, the square root of 3.
Flow matching wants weights that add to 1 without squaring. Divide by a + b = 1.366. The weights become 0.366 for the data and 0.634 for the noise. So t = 0.634, and the signal-to-noise ratio is still 1/3, since (0.366 / 0.634)^2 = 0.333.
For v-prediction, 0.5 = cos(phi) gives phi = 60 degrees. With a clean value x0 = 1 and eps = 0, the target is v = 0.5 x 0 - 0.866 x 1 = -0.866.
For the score, the direction toward likelier data is -eps / b. Its typical size is 1 / 0.866 = 1.155.

The picture is the same in every row. Only the brightness scale, the clock label and the output changed. That is the whole lesson in one paragraph, and the rest of it fills in the details.

What Do You Think?

A flow matching model sits at t = 0.5 in the SD3 convention. Which DDPM alpha_bar sits at the same signal-to-noise ratio?

#Check the Translations in numpy

The hand trace used one number. Below, the code draws 100,000 values of clean data and noise, builds the noisy picture, and tests each translation as a true or false line. It also tests two more claims from the sections to come: that EDM's scale factors make the training target have spread 1 at every noise level, and that guidance computed on the noise guess and on the v guess agree.

Read it in blocks. The first block builds the DDPM picture and the v target. The second and third convert to the EDM and flow forms. The score line shows the target is large when the noise weight is small. The EDM loop uses a data spread of 0.5, which is my recollection of the paper's default, so verify it in the EDM paper's table. The last block checks guidance.

pythonplayground.py · Pyodide

import numpy as np

rng = np.random.default_rng(0)
x0 = rng.normal(0.5, 0.5, size=100_000)       # toy clean data, spread 0.5
eps = rng.normal(size=x0.shape)               # noise
ab = 0.25                                     # alpha_bar at one step
a, b = np.sqrt(ab), np.sqrt(1 - ab)           # signal weight, noise weight
x_t = a * x0 + b * eps                        # DDPM form

# v-prediction (Salimans and Ho): v = a * eps - b * x0
v = a * eps - b * x0
print("a, b:", round(a, 3), round(b, 3), "| a^2 + b^2 =", round(a**2 + b**2, 3))
print("x0 from v:", np.allclose(a * x_t - b * v, x0), "| eps from v:", np.allclose(b * x_t + a * v, eps))

# EDM: sigma = noise over signal, and the data is not scaled
sigma = b / a
print("EDM sigma =", round(sigma, 4), "| x_t / a equals x0 + sigma * eps:", np.allclose(x_t / a, x0 + sigma * eps))

# Flow matching (SD3 style): x = (1 - t) * x0 + t * eps with the same signal-to-noise ratio
t = b / (a + b)
print("flow t =", round(t, 4), "| x_t / (a + b) equals (1 - t) x0 + t eps:", np.allclose(x_t / (a + b), (1 - t) * x0 + t * eps))
print("flow velocity eps - x0 from an eps guess:", np.allclose(eps - (x_t - b * eps) / a, eps - x0))

# Score: the direction toward more likely data is minus eps over b (exact for the noising kernel)
print("score target -eps / b, size:", round(float(np.std(-eps / b)), 3), "(= 1 / b =", round(1 / b, 3), ")")

# EDM preconditioning, sigma_data = 0.5: the training target F has spread 1 at every noise level
sd = 0.5
for s in (0.1, 1.0, 10.0):
  x = x0 - 0.5 + s * eps                     # centred toy data so its spread is sd
  c_skip, c_out = sd**2 / (s**2 + sd**2), s * sd / np.sqrt(s**2 + sd**2)
  target = (x0 - 0.5 - c_skip * x) / c_out
  print(f"sigma {s:5.1f}: c_skip {c_skip:.3f}, c_out {c_out:.3f}, spread of target F {np.std(target):.3f}")

# Guidance is the same affine rule in any of these forms
eps_u, eps_c, s_cfg = rng.normal(size=5), rng.normal(size=5), 3.0
guided_eps = eps_u + s_cfg * (eps_c - eps_u)
to_v = lambda e, xt: a * e - b * (xt - b * e) / a
xt5 = rng.normal(size=5)
guided_v = to_v(eps_u, xt5) + s_cfg * (to_v(eps_c, xt5) - to_v(eps_u, xt5))
print("guide eps then convert to v equals guide v:", np.allclose(to_v(guided_eps, xt5), guided_v))

# Loss weights: the same mistake costs different amounts in each language
x0_hat = x0 + 0.1 * rng.normal(size=x0.shape)          # a guess that is off by e = x0_hat - x0
e2 = np.mean((x0_hat - x0) ** 2)
eps_hat = (x_t - a * x0_hat) / b                       # the eps guess that goes with that x0 guess
v_hat = a * eps_hat - b * x0_hat
flow_hat = eps_hat - x0_hat                            # SD3 style velocity guess
snr = ab / (1 - ab)
print("weight on x0 error, eps loss :", round(float(np.mean((eps_hat - eps) ** 2) / e2), 3), "| SNR =", round(snr, 3))
print("weight on x0 error, v loss   :", round(float(np.mean((v_hat - v) ** 2) / e2), 3), "| SNR + 1 =", round(snr + 1, 3))
print("weight on x0 error, flow loss:", round(float(np.mean((flow_hat - (eps - x0)) ** 2) / e2), 3), "| 1 / t^2 =", round(1 / t**2, 3))

Tests · Check that every True line prints True, that the EDM target spread is about 1 at sigma 0.1, 1 and 10, and that the three loss weights match SNR, SNR + 1 and 1 over t squared.

The printed lines match the comments in the solution box. The last three lines are a preview. They show that the same small mistake in the clean picture costs different amounts depending on which language the loss is written in. The next section turns that into a dial.

#Schedules, Loss Weights and Samplers Are Three Separate Dials

When a new model looks better, people say "a better schedule" or "a better sampler" as if there were one thing to improve. There are three, and they act at different times.

Dial 1: the schedule. It decides how noisy the picture is at each clock value. It is a curve, alpha_bar(t), or a list of sigma values, or a choice of how to draw t. It is used twice. In training it sets which noise levels the network sees and how often. In sampling it sets the grid of noise levels the sampler visits. Linear and cosine schedules (Ho et al. 2020, Nichol and Dhariwal 2021) differ in how fast the picture is destroyed.

Dial 2: the loss weight. It decides how much each noise level counts in the training loss. It acts only in training and is a number multiplying each level's error. Predicting the noise with a plain squared error is one choice. Predicting v is another. The cell above showed the result: the noise loss counts a clean-picture mistake SNR times, the v loss counts it SNR + 1 times, and the flow velocity loss counts it 1 / t^2 times. Different words, different weights.

Dial 3: the sampler. It decides how the finished network is used to walk from noise to a picture. It acts only at sampling time. It chooses the step grid, the formula for one step, and whether to add fresh noise. You can change it without retraining.

Why does the separation matter? Because the dials mix in practice, and one name often hides two of them. The EDM paper is a clear example. It splits the design into the noise levels used in training, the weights on them, the scale factors that wrap the network, and the sampler, and it argues that each can be chosen on its own (Karras et al. 2022). The words "DDIM" and "DPM-Solver" are samplers only. The words "cosine" and "linear" are schedules. The words "eps-prediction" and "v-prediction" change the loss weight and the output form together. A paper that changes all three at once cannot say which change helped, the same trap as in the diffusion transformers lesson.

You change	It acts in	Needs retraining	It mostly affects
Training schedule (which t, how often)	training	yes	what the network learns well
Loss weight or prediction type	training	yes	which noise levels get accuracy
Step grid and solver	sampling	no	speed and fidelity of the walk
Fresh noise per step (eta)	sampling	no	diversity and smoothness
Guidance scale s	sampling	no	prompt match against variety

One caution on the schedule dial. The sampling grid is not forced to match the training grid. A network trained on 1000 steps can be sampled on 20 of them, because the network was shown every noise level. A related pitfall is the end of the schedule. With the original linear schedule (1000 steps, beta from 1e-4 to 0.02), alpha_bar at the last step is 4.0e-5, not 0, so the "pure noise" end is slightly less than pure. Lin et al. (2023) argued this mismatch leaves a trace in the pictures and proposed a fix, which I list here as a pointer to check, not a settled recommendation. Verify it for your model before you rely on it.

#Seeing the schedule dial

The first toy shows the schedule in terms of the number that matters, the log of the signal-to-noise ratio. Move the timestep and the schedule settings, and watch how much of the steps are spent where almost nothing is left of the picture.

Try it: where each schedule spends its stepsInteractive

Loading visualization...

Do: Move the timestep, the linear beta range and the cosine offset.
Notice: The linear schedule leaves under 1 percent of the signal for about a third of its steps (32.7 percent for 1000 steps with beta from 1e-4 to 0.02). The cosine schedule does so for about 6.5 percent.
Why: The schedule decides where the steps go, and steps spent on almost pure noise teach the network little. Drawn stand-in, not a trained model: the picture strip is an illustration, and the curves are exact formulas, not measurements of any real network.

#Guidance in Every Convention

The track uses one guidance convention, set in the diffusion lesson, and it is the same in every language of the table above.

\hat{\varepsilon}_{\text{guided}}=\varepsilon_u+s\,(\varepsilon_c-\varepsilon_u)

Three translations cover almost every case.

The paper's w. A form that reads (1 + w) * eps_c - w * eps_u is the same rule with w = s - 1. A form that reads w * eps_c + (1 - w) * eps_u is also the same rule, with w = s. Always expand the formula and compare it with the one above.
The library's number. In the diffusers library the argument is called guidance_scale, and for the standard two-pass pipelines it is s. A value of 1 means no extra push.
The output form. The rule is a straight-line mix of two outputs, and the outputs of every convention are straight-line functions of each other when x_t and the noise level are fixed. So guiding the noise guess and then converting to v gives the same result as guiding the v guess. The last lines of the cell confirm that. The same rule therefore works for velocities in flow models.

Two cautions. Some models fold guidance into the network through a training step (guidance distillation), so the knob works differently and costs one call instead of two. Whether a given model does this is something to read in its model card, so verify it for the model you use. And a larger s does not simply mean a better picture: it trades variety and sometimes realism for prompt match, as the diffusion models lesson shows.

#Samplers: Taking Steps Along the Path

A sampler answers one question. Given the picture now, at noise level a, and the network's guess, where should the picture be at the next, lower level b? Everything below is a different answer.

Picture first. Imagine a ball on a hillside, and the network is a sign that points downhill at wherever you stand. A sampler is a walker who follows the signs. A walker who takes tiny steps follows the path well but needs many looks at the sign. A walker who takes huge steps looks rarely but walks off the path where it bends. The skill is in taking long steps without leaving the path.

#Two ways to walk: with dice or without

Diffusion has two matching descriptions of the walk. One adds fresh random noise at each step, a stochastic differential equation. The other adds none: it is an ordinary differential equation, called the probability-flow ODE (Song et al. 2021). Both give the same distribution of final pictures if solved exactly. The ODE is a smooth path from each starting noise to one picture, so the same start always gives the same picture. The noisy version wanders and can correct its own mistakes.

DDPM ancestral sampling. Take one step down the noise ladder, then add a small amount of fresh noise. It is the noisy walk. It needs about as many steps as the training used (1000 in the original), because each step is small.

DDIM (Song et al. 2021). Use the network's guess of the clean picture, x0_hat, and of the noise, eps_hat, to jump to an earlier level. The jump is written as

x_{\text{prev}}=\sqrt{\bar\alpha_{\text{prev}}}\,\hat x_0+\sqrt{1-\bar\alpha_{\text{prev}}-\sigma^{2}}\;\hat\varepsilon+\sigma\,z

The dial eta scales sigma. At eta = 0 there is no fresh noise, and the sampler is deterministic. At eta = 1 the sigma equals the DDPM value, so DDIM with eta = 1 is ancestral sampling, which the DDIM paper shows. In between you get a mix. Because the jump is written for any pair of levels, DDIM can skip: it can go from step 1000 to 980 to 960 and so on.

Euler on the probability-flow ODE. Write the ODE in the EDM form. With the unscaled picture and noise level sigma, the slope is

\frac{dx}{d\sigma}=\frac{x-D(x;\sigma)}{\sigma}

An Euler step with this slope is the same walk as DDIM with eta = 0 in the right variables. The DPM-Solver paper states that its first-order method equals DDIM, which is why a deterministic DDIM run can be called a first-order solver (Lu et al. 2022).

Heun, the second-order step. Take the Euler step to a trial point, read the slope there, and step with the average of the two slopes. It costs two network calls per step, except the last, and it follows a curved path much better. EDM uses this (Karras et al. 2022).

DPM-Solver and relatives, at a conceptual level. The ODE has a part that is exactly solvable (the linear scaling of the picture) and a part that depends on the network. DPM-Solver (Lu et al. 2022) solves the first part exactly in the variable log signal-to-noise ratio and approximates only the network part, in first, second or third order. DPM-Solver++ (Lu et al. 2022) rearranges the same idea around the clean-picture guess and a multistep form, which was reported to behave better with strong guidance. Newer solvers extend the pattern. The common idea: treat sampling as numerical integration and spend your network calls where the path bends most. The papers report good samples in roughly 10 to 20 calls, but how few calls suffice depends on the model, the guidance scale and the quality you accept, so measure on your own model.

What Do You Think?

A sampler uses Heun steps and takes 10 steps. About how many network calls is that?

#Euler and Heun by hand

Before the code, do one small ODE by hand. Take dx/dt = -x, with x = 1 at t = 0, and walk to t = 1 in two steps of size 0.5. The exact answer is e^-1 = 0.3679.

Euler. At x = 1 the slope is -1, so the first step lands at 1 + 0.5 x (-1) = 0.5. At x = 0.5 the slope is -0.5, so the second lands at 0.5 + 0.5 x (-0.5) = 0.25. The error is 0.3679 - 0.25 = 0.118.

Heun. First step: the slope at 1 is -1, the trial point is 0.5, the slope there is -0.5, and the average is -0.75, so the step lands at 1 + 0.5 x (-0.75) = 0.625. Second step: the slope at 0.625 is -0.625, the trial point is 0.3125, the slope there is -0.3125, the average is -0.46875, and the step lands at 0.625 + 0.5 x (-0.46875) = 0.3906. The error is about 0.023.

Same two steps, but Heun was five times closer, at the price of four slope calls against two. That is the trade, in miniature: a better step costs more calls, and it is worth it when the path curves.

#Measuring the error as steps grow

Now the real test, on an ODE with a known answer. The data is a bell curve with mean 2 and spread 0.5. For a bell-curve source, the best clean guess has a closed form, so there is no network and no training: every error comes from the stepping alone. The ODE then has an exact solution, x(sigma) = mu + (x_T - mu) sqrt(s0^2 + sigma^2) / sqrt(s0^2 + sigma_T^2), which the code uses as the truth. The code walks from sigma = 10 down to 0 with Euler and with Heun on a grid that is even in sigma^(1/7) (the grid shape EDM suggests, to be checked in the paper) and prints the error against the exact answer.

pythonplayground.py · Pyodide

import numpy as np

mu, s0 = 2.0, 0.5            # data: a bell curve with mean 2 and spread 0.5
sig_max = 10.0               # start noise level (sigma)
x_T = 7.0                    # one starting point on the noisy end

def slope(x, s):             # dx/dsigma on the probability-flow ODE (EDM form)
  denoised = mu + s0**2 / (s0**2 + s**2) * (x - mu)    # the exact best guess of clean x
  return (x - denoised) / s if s > 0 else 0.0

def exact(s):                # closed-form solution of the same ODE
  return mu + (x_T - mu) * np.sqrt(s0**2 + s**2) / np.sqrt(s0**2 + sig_max**2)

def grid(n):                 # n steps, spaced evenly in sigma ** (1/7), ending at 0
  r = np.linspace(sig_max ** (1 / 7), 0.0, n + 1)
  return r ** 7

def euler(n):
  s, x, nfe = grid(n), x_T, 0
  for a, b in zip(s[:-1], s[1:]):
      x += (b - a) * slope(x, a); nfe += 1
  return x, nfe

def heun(n):
  s, x, nfe = grid(n), x_T, 0
  for a, b in zip(s[:-1], s[1:]):
      d1 = slope(x, a); xe = x + (b - a) * d1; nfe += 1
      if b > 0:
          x += (b - a) * 0.5 * (d1 + slope(xe, b)); nfe += 1
      else:
          x = xe
  return x, nfe

print(f"exact answer at sigma = 0: {exact(0.0):.4f}")
print("steps | Euler error (NFE) | Heun error (NFE)")
for n in (2, 4, 8, 16, 32):
  e, ne = euler(n); h, nh = heun(n)
  print(f"{n:5d} | {abs(e-exact(0)):.5f} ({ne:2d})   | {abs(h-exact(0)):.5f} ({nh:2d})")

e32, _ = euler(32); h16, nh16 = heun(16)
print(f"about 32 slope calls: Euler(32) error {abs(e32-exact(0)):.5f}, Heun(16) error {abs(h16-exact(0)):.5f} using {nh16} calls")
print("error shrink per doubling, Euler:", [round(float(abs(euler(n)[0]-exact(0))/abs(euler(2*n)[0]-exact(0))),1) for n in (8,16)])
print("error shrink per doubling, Heun: ", [round(float(abs(heun(n)[0]-exact(0))/abs(heun(2*n)[0]-exact(0))),1) for n in (8,16)])

Tests · Check that the exact value prints 2.2497, that the Euler error roughly halves per doubling of steps, that the Heun error falls about four-fold per doubling from 8 steps on, and that Heun at 16 steps beats Euler at 32 steps for a similar number of calls.

The printed lines match the comments. Four facts are worth saying in words. First, both methods improve as steps rise, since smaller steps follow the path better. Second, Euler's error roughly halves each time the steps double, and Heun's falls by about four, which is what first order and second order mean. Third, at two steps Heun is far worse than Euler: the steps are so long that the second-order guess overshoots. Fewer steps does not always favour the fancier method. Fourth, at about equal cost, 31 calls for Heun against 32 for Euler, Heun is closer to the truth, though only by a modest margin here. The picture changes with the problem, so test on yours.

This toy has a known, smooth path. A real network's slope is noisier and less smooth, so real error curves are less tidy than these. What carries over is the idea: step count and step order trade against each other, and the best use of a fixed budget of network calls is a design question.

#Seeing the step count live

The second toy uses a three-bump distribution whose noise guess is known exactly, so the only error is the skipping. Set the steps low and watch the histogram miss the bumps. Slide them up and watch the gap close. Set eta to 1 and watch fresh noise return.

Try it: steps and eta on a known densityInteractive

Loading visualization...

Do: Slide the step count from 2 up to 200, then set eta to 1 and switch the schedule between linear and cosine.
Notice: The histogram closes in on the true three bumps as the steps grow. At eta = 1 fresh noise returns at every step, and the same number of steps then lands differently.
Why: A long jump trusts one noise guess over a long distance, so on a curved path it lands off the path. Drawn stand-in, not a trained model: the noise guess is exact for a toy density, so every error shown is a stepping error and none is a training error.

Notice that the viz uses the DDIM form from this lesson, while the numpy cell used the EDM-form ODE. They describe the same walk in two clocks, which is what the Rosetta table is for.

#Reading a Config File

All of this shows up as a few lines in a library. In diffusers, a scheduler object holds the schedule and the sampler, and the model holds the prediction type. Three settings are worth knowing by name, and the names are as I recall them from the library, so check them against the version you install.

prediction_type takes values such as epsilon, sample and v_prediction. It must match what the network was trained to output. Using the wrong one gives noise or washed-out pictures with no error message, which is the silent bug from the start of this lesson.
beta_schedule, and for some models the timestep spacing, set the schedule dial.
The scheduler class sets the sampler: DDIMScheduler, EulerDiscreteScheduler, HeunDiscreteScheduler, DPMSolverMultistepScheduler, and a flow matching one for flow models such as SD3. You can swap it without retraining.

The code below swaps the sampler of a published pipeline. It needs torch and a GPU, so it does not run on this page. Copy it to Google Colab.

pythonrunnable cell

# Copy to Colab: needs torch and diffusers, and a GPU, so it cannot run on this page.
# Same network, three different samplers: only the sampler dial changes.
import torch
from diffusers import (StableDiffusionPipeline, DDIMScheduler,
                       EulerDiscreteScheduler, DPMSolverMultistepScheduler)

pipe = StableDiffusionPipeline.from_pretrained(
    "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16
).to("cuda")
print("prediction type:", pipe.scheduler.config.prediction_type)   # what the network outputs

prompt = "a lighthouse on a cliff at sunset"
for name, cls in [("ddim", DDIMScheduler), ("euler", EulerDiscreteScheduler),
                  ("dpm", DPMSolverMultistepScheduler)]:
    pipe.scheduler = cls.from_config(pipe.scheduler.config)         # swap the sampler only
    for steps in (10, 25):
        g = torch.Generator("cuda").manual_seed(7)                   # same starting noise
        image = pipe(prompt, num_inference_steps=steps, guidance_scale=7.5, generator=g).images[0]
        image.save(f"{name}_{steps}.png")                            # compare the six pictures

The model identifier above is a published one as I remember it, so check that it still loads before you rely on it. With the same prompt, the same starting noise and the same guidance scale, differences between the saved pictures come from the sampler alone, and the difference between 10 and 25 steps shows the trade from the toy above on a real network.

#Putting It Together

Here is the reading routine for any new diffusion paper or repository, as five questions.

What is the noisy picture? Find the line that builds it. Is the data scaled down, or is noise piled on?
Which way does time run, and what is its unit? Integer steps, a continuous t, a flow fraction or a sigma.
What does the network output? Noise, clean picture, v, velocity or score. Convert it to the one you know.
What are the three dials set to? The schedule, the loss weight and the sampler, separately.
How is guidance written? Expand it and compare it with eps_u + s (eps_c - eps_u).

The same routine works for a model whose name you have never heard. Most of the confusion in this field comes from skipping step 1.

Elucidating the Design Space of Diffusion-Based Generative Models

Tero Karras, Miika Aittala, Timo Aila, Samuli Laine (2022)

The EDM paper. Separates the noise levels, the loss weights, the network scale factors and the sampler, and uses a second-order Heun sampler on the probability-flow ODE.

Denoising Diffusion Implicit Models

Jiaming Song, Chenlin Meng, Stefano Ermon (2021)

The DDIM paper. A family of samplers with a fresh-noise dial that includes DDPM as one end and a deterministic walk as the other.

Score-Based Generative Modeling through Stochastic Differential Equations

Yang Song and others (2021)

Describes diffusion as a stochastic differential equation and gives the probability-flow ODE with the same distribution at every time.

Progressive Distillation for Fast Sampling of Diffusion Models

Tim Salimans, Jonathan Ho (2022)

Introduces v-prediction, the angle form of the noisy picture, and the loss weighting it implies.

DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps

Cheng Lu and others (2022)

Solves the linear part of the ODE exactly in log signal-to-noise ratio and approximates the rest, with first, second and third order versions.

Common Diffusion Noise Schedules and Sample Steps Are Flawed

Shanchuan Lin and others (2023)

Points out that common schedules do not reach zero signal at the last step and proposes fixes. Treat the proposed fixes as a pointer to check for your model.

#Key Takeaways

Different papers describe the same picture-making machine with different clocks, scales and outputs. Find the line that builds the noisy picture and read everything against it
Signal-to-noise ratio lines up every convention. At alpha_bar = 0.25 the EDM sigma is 1.732 and the SD3 flow time is 0.634
Noise, clean picture, v and velocity are the same information in different clothes, convertible when you know x_t and the noise level. Guidance eps_u + s (eps_c - eps_u) works on any of them
The schedule, the loss weight and the sampler are three separate dials. The loss weight acts only in training, the sampler only in sampling, and the schedule in both. Only the sampler can be changed without retraining
DDIM with eta = 1 is ancestral sampling, and with eta = 0 it is a deterministic walk. Euler and Heun are first and second order steps on the probability-flow ODE
Compare samplers at the same number of network calls. On the known ODE, Euler's error halves per doubling of steps and Heun's falls about four-fold, but Heun is worse at two steps

#Quick Check

Quick Check1 / 4

A DDPM step has alpha_bar = 0.25. What is the EDM sigma at the same signal-to-noise ratio?

Where to next. Continue to Generative Evaluation Deep Dive to see how to measure whether a sampler or a schedule really helped. For the training side of the dials, return to Diffusion Models. For the route that cuts the step count to a handful, read Consistency Models.

Diffusion Conventions and Samplers

Learning Objectives

After this lesson, you will be able to:

Translate one noise level between the DDPM, EDM, flow matching, score-based and v-prediction conventions, and check the translation with code
Say which way time runs in each convention, and what the network is asked to predict in each
Write classifier-free guidance in the track's one convention and map it to a paper's w and to a library's guidance scale
Explain why the noise schedule, the loss weight and the sampler are three separate dials, and say what each one changes
Describe DDPM ancestral sampling, DDIM with its eta dial, and Euler and Heun steps on the probability-flow ODE
Read a run of errors against step count and tell a first-order method from a second-order one