1Destroying an image on purpose
A diffusion model starts with a process that is easy to describe: take a training image and add a small amount of random noise to every pixel. Repeat this many times, 1,000 steps in the paper, and the image becomes indistinguishable from pure static. This forward process is fixed; nothing about it is learned. Destroying an image is easy. The interesting part is learning to run the process in reverse.
2Learning to undo one step
A neural network is given a noisy image and the step number, and is trained to make the image slightly less noisy. To generate a new image, you start from pure random noise and apply the network once for each step, from the noisiest step back to the first. After all the steps, what remains is an image. Each application depends on the result of the one before, so generation needs about 1,000 network evaluations in sequence.
3Predict the noise
A useful property makes training simple: the noisy image at any step can be produced in one jump, as a mix of the original image and a single fresh sample of noise, in proportions set by the step. Training picks an image, a random step, and a random noise sample, mixes them, and asks the network to predict which noise was added. The error is the squared difference between the predicted and actual noise. As an illustration, suppose a noisy pixel equals 0.8 times the original plus 0.6 times the noise. With an original of 0.5 and noise of 1.0, the noisy pixel is 0.4 + 0.6 = 1.0. If the network predicts noise of 1.0, the original can be estimated as (1.0 − 0.6) ÷ 0.8 = 0.5. The paper found that this simplified noise-prediction objective gave better image quality than a more complicated weighting.
4Results
On CIFAR-10, a standard dataset of small 32-by-32 pixel images, the model reached an FID of 3.17, the best reported at the time for models generating images without class labels. FID compares statistics of generated images with those of real images; lower is better. The model also produced high-quality 256-by-256 images of faces, bedrooms, and churches.
5Costs and caveats
Generating an image takes about 1,000 network evaluations in sequence, far slower than a generator that produces an image in one pass. The paper also reports that, despite their sample quality, its models did not achieve competitive likelihood scores, a standard measure of how well a model assigns probability to real data, compared with other likelihood-based models. Good-looking samples and high likelihood are different goals, and FID captures only some aspects of image quality.
Jonathan Ho, Ajay Jain, and Pieter Abbeel · 2020. Adapted by DailyPaper; illustrative examples are labeled in the reading.