From Noise to Picture

#Adding Static, Step by Step

Picture a photo of a dog taped to a window. Now sprinkle a fine layer of sand over it. The dog is still easy to see. Sprinkle again, and again. After many layers, the glass is a gray mess of sand. No dog is left.

Adding static to a picture works the same way. In each step, a little random speckle is mixed into every dot of the picture. Early on, the picture is still clear. Later, it looks grainy. At the end, only static remains, and the original is gone.

Here is a tiny example with one dot of a photo. Its brightness is 100. One step adds a speckle of 30, so the dot reads 130. Another step adds a speckle of minus 10, so it reads 120. Each dot gets its own random speckle. Do this to every dot, many times, and the whole picture dissolves.

This direction is easy. A fixed rule does it. Nothing has to be learned. The tool below uses a stored picture of the digit 8 and mixes in static by that fixed rule.

Do: Press Forward (Noising). Then drag the Timestep slider from 0 up to 1000.

Notice: The digit gets harder to see as the number rises. At 1000 only static is left.

Why: Each step adds only a little static, so a clear picture fades slowly into pure speckles.

The number in the slider is the step count. Zero means the clean picture. One thousand means all static. The number 1000 belongs to this tool. Other tools use other counts.

The next tool draws a small picture of a sky, a sun and a house, and lets you slide to any step. A bar shows how much of the mix is still picture. It also checks a shortcut. You might expect that to reach step 300 a computer must add static 300 times in a row. It does not have to. One bigger dose of static gives the same picture.

Do: Drag the Step slider to 300, then to 800. Press Jump straight here. Then press the Cosine button.

Notice: At step 300 about 40% of the mix is still picture on the Linear setting, and about 79% on Cosine. The picture from one jump matches the picture from 300 small steps, with a largest difference too small to see (a rounding error).

Why: Mixing in a little static many times adds up to the same as mixing in one bigger dose once, so the computer can skip straight to any step. Cosine adds static more slowly at the start. Drawn stand-in, not a trained model: the picture is drawn by rules and the static is seeded.

#The One Small Job: Guess the Static

Picture a friend who loves spot-the-difference puzzles. You show them a photo with a smudge on it and ask, "Which part is the smudge?" That is a small, clear question. A good guess lets you wipe the smudge away.

The model in diffusion learns exactly this kind of job. It is shown a picture with some static in it. It is asked to guess the static, not to paint a picture. It is also told how much static to expect, since a nearly clean picture has a little and a nearly buried one has a lot.

Here is a tiny example. A dot in the photo has brightness 100. Static of 30 makes it read 130. The model looks at the dot and the dots around it, and guesses the static is 28. Remove 28 and the dot reads 102. That is close to 100, and the picture is a little cleaner.

Now think about how the model gets graded. Before reading on, make a prediction.

Because the static is added by a known rule, the teacher can make endless practice rounds. Take any photo. Pick any amount of static. Mix it in. Then ask the model to guess the static and check the guess. The model gets better with each round.

#Cleaning Up from Pure Static

Picture a restorer cleaning a very dirty painting. They do not scrub it clean in one go. They wipe a little, look, and wipe a little more. The picture shows through slowly.

The model does the same. It starts with a screen of pure static. It guesses the static in it. It removes a small part of that guess. The screen is now a little less staticky. Then it does the same again on the new screen, and again, many times. Early cleanups settle the big shapes and the light and dark areas. Later cleanups settle the small details. At the end, a picture is there that nobody drew.

This is the loop in the tool below. It uses a small grid of squares. The target picture is a cross.

Do: Press Next step until the end. Then press Reset, press Show target, and compare the 20-step and 4-step buttons.

Notice: The middle grid is the static guess. As it settles, the left grid turns into a cross.

Why: Each step removes only part of the guessed static. The oracle cheats, so real models are less exact.

Read the caption again. The guesser in this tool is an oracle. It was told the target picture, so its static guesses are almost perfect. A real model has no such help. It never sees a target. It has learned, from a huge number of pictures, what staticky pictures usually hide underneath. Its guesses are rougher, and the result is a new picture, not a copy of a target.

Here is the full loop as a diagram. The top row is the easy direction, done by a fixed rule. The bottom row is the learned direction. The prompt, if there is one, joins at the guess.

Adding static needs no learning. Cleaning up is where the model's skill matters. Teaching uses the top row to make practice rounds. Making a new picture uses only the bottom row.

#Why Guessing Static Is Easier Than Painting

Picture two jobs. In the first, you are handed a blank canvas and told, "Paint a dog." In the second, you are handed a dog photo with some sand on it and told, "Point to the sand." The second job is much smaller. You do not decide what a dog is. You only notice what does not belong.

That is the gap. Painting from nothing means choosing every dot of the picture at once, and every choice must fit with all the others. Guessing static means looking at what is already there and saying which part is out of place. The model makes one small, local judgment, then the next step makes another.

There are three more reasons it works well.

• It always has a right answer. We know the static we added, so every guess can be graded.
• One skill covers every stage. A buried picture and a nearly clean picture need the same skill, with a different amount of static.
• Small slips can be fixed. If one step guesses badly, the picture changes only a little, and later steps can still work around it.

Compare this with a model that had to paint the whole picture in a single move. It would need to get the big shapes, the small details and the colors all right at once, with no second chance.

#Why Different Static Gives Different Pictures

Picture a potter who follows one plan to make a pot. Before the first touch, the potter picks up a lump of clay. In diffusion, your prompt and the model are the plan. The starting pile of static is the lump of clay.

Before reading on, make a prediction.

Two piles of static look alike to us, but they differ in every dot. The first cleanups settle the big shapes, and the big shapes depend on the pile. So two starts lead to two layouts, and then two pictures. The prompt keeps both on topic.

If you keep the same starting pile, the same prompt and the same settings, and the clean-up adds no fresh randomness along the way, you get the same picture back. Tools often let you fix the starting static, and they may call it a seed. Fix it to repeat a result. Change it for a fresh idea.

This is also why one prompt can give you many different pictures. You are not looking up a stored picture. Each run starts at a new place and travels to a new result.

#Steering the Clean-up with a Prompt

Picture a friend guiding you through a dark room. With no help, you shuffle toward any open space. With a friend saying "to the left, to the left", you head to the door. The firmer the push, the harder you are steered.

A prompt works as that friend. At each step, the model makes two guesses about the static. One guess ignores your words. The other reads them. The gap between the two guesses points toward "more like the prompt". The guidance dial, called s, sets how hard to push along that gap.

Before you try it, make a prediction about the trade.

Do: Drag the slider for s from 1 to 15. Press the reseed button to draw four new samples.

Notice: At low s the samples differ a lot. Near the top they look alike and overdone.

Why: A higher s pushes harder toward the prompt, which removes variety. A low s lets the pictures wander.

The meters under the slider show the same trade. As s rises, "Prompt adherence" rises and "Diversity" falls. "Over-saturation" climbs near the top. These meters are drawn curves, not measurements.

So which setting should you pick? It depends on what you want. If you need a picture that follows the words closely, lean strict. If you want to explore ideas, lean loose. The middle often gives a fair mix of both. Different tools name and scale this dial in different ways, so check what its number means before you compare settings.

#Key Takeaways

• A picture can be turned into pure static by adding a little static many times. A fixed rule does this, and nothing is learned.
• The model learns one small job: guess the static in a picture. It can be graded, because we put the static there ourselves.
• To make a new picture, start from pure static. Guess the static, remove a little, and repeat until a picture appears.
• Guessing static is easier than painting, because it is a small local judgment that has a right answer and is repeated many times.
• A different starting pile of static gives a different picture. The same start and settings give the same picture.
• A prompt steers the clean-up. A low guidance number is loose and varied. A high number is strict and alike.

#Quick Check