Diffusion model

Diffusion models are the technology behind many text-to-image tools and many AI video generators. They work quite differently from the large language models behind chat assistants: instead of writing one token after another, they refine a whole picture at once, over and over.

The name comes from physics, where diffusion describes particles spreading out, like a drop of ink in water. The model learns to run that spreading process in reverse, turning scattered noise back into structure. The same idea also drives video and audio generators as well as image tools.

How it works

Training starts with real images. The system takes each image and adds a little random noise, then a little more, over many steps, until the image is pure static. The model's job during training is to look at a noisy version and predict the noise that was added, so it can be removed. By practising this on enormous numbers of images at every noise level, the model learns what real images look like at every stage.

To generate something new, the process runs backwards. The model starts from fresh random noise and, step by step, predicts and removes noise. A text prompt, converted into numbers by a separate text encoder, steers each step so the emerging image matches the description. After a few dozen steps, often fewer in modern systems, a clean image appears.

Many systems do this work in a compressed version of the image, called latent space, rather than on every pixel, which makes generation much faster. The same approach can be extended across time for video or applied to sound. Because each run starts from different noise, the same prompt gives different results unless the starting noise, called the seed, is fixed. Fewer steps make generation faster but can reduce detail, so tools often trade speed against quality behind a single setting.

Example

An architecture practice uses an image tool built on a diffusion model to explore moods for a client presentation. A designer uploads a simple sketch of a building and asks for "evening light, wet pavement, warm windows". The model keeps the sketch's outline as a guide and generates several finished-looking variations in under a minute, giving the client something to react to early.

When the client picks one, the designer fixes the seed and changes only the lighting words, so the composition stays the same while the mood shifts. The images are clearly labelled as concept visuals in the presentation, because details such as window spacing and materials are the model's invention, not the actual design.

Why it matters for your business

You do not need to understand the mathematics to use these tools, but knowing how diffusion works explains their behaviour: why results vary each time, why fine details like text and hands can go wrong, and why a reference image or sketch gives more control than words alone. It also explains why image generation can be slower and more costly per item than generating text. For concept work and decorative images, diffusion tools save real time; for anything that must show reality accurately, they are the wrong tool. Image tools also differ in their rules on commercial use, so read the terms of the one you pick.

Common mistakes

• Expecting identical results from the same prompt without fixing the seed.
• Presenting diffusion-generated concept images as accurate depictions of a real design or product.
• Writing longer and longer prompts when a simple reference image or sketch would give more control.
• Assuming a diffusion model knows your brand's exact colours, logo or products without being shown them.

Questions people ask

Is a diffusion model the same as a large language model?

No. A large language model writes text one token at a time. A diffusion model generates images, video or audio by gradually removing noise from a whole output at once. Some products combine both, using a language model to interpret the request and a diffusion model to draw.

Why do diffusion models give different images each time?

Each generation starts from fresh random noise, and that starting point shapes the final image. Fixing the seed, the number that sets the starting noise, makes results repeatable with the same prompt and settings. Many tools show or let you set the seed for this reason.

Are diffusion models only for images?

No. The same idea is used for video, audio and music, and researchers are applying it to other kinds of data. Images were the first widely popular use, which is why the term is most associated with picture generators.