What is generative AI? How one idea writes text, draws images and codes

Generative AI is software that creates new content, such as text, images, code, audio or video, instead of only sorting or labelling what already exists. It does this by learning the statistical patterns in a very large amount of training data and then producing new output that follows those patterns.

The US standards body NIST uses a definition from a US executive order: models that copy the structure and characteristics of the data they learned from in order to generate new, synthetic content. In plain terms, a model sees enough examples of something and learns to make more of it.

The core idea: predict the next piece

Most generative models work by guessing one piece at a time.

A language model splits text into tokens, small chunks of words. Given the tokens so far, it predicts which token is most likely to come next, adds it, and repeats. NIST's generative AI profile describes large language models the same way, as systems that predict the next token or word. A long answer is thousands of these small predictions in a row.

Many of these models are built on the Transformer, a design introduced in the 2017 paper "Attention Is All You Need". Its key part, called self-attention, lets the model relate every position in a sequence to every other, so each new token is chosen with the whole context in view. The paper's model generated output step by step, using the symbols it had already produced as input for the next one. That loop is still how a chatbot writes a reply.

Images and video: start from noise

Pictures are usually made a different way, by a method called diffusion. The 2020 paper on denoising diffusion models set out the approach:

1. Training. Take a real image and add random noise to it in many small steps until nothing of the original is left. The paper used 1,000 steps.
2. Learning. Train a neural network to undo one step at a time, predicting the noise that was added.
3. Generating. Start from pure random noise and run the learned undo steps in order. A new image appears that was never in the training data.

The two ideas now overlap. A 2022 paper on diffusion transformers replaced the usual image network with a Transformer that works on patches, small squares of a compressed image, much as a language model works on tokens.

What it can make

• Text and code. Anthropic's Claude models take text and images as input and produce text. Its documentation lists summarising, answering questions, translating, and explaining and generating code.
• Images. Google DeepMind's Nano Banana creates and edits images.
• Video. Google DeepMind's Veo generates video with sound.
• Music and speech. Google DeepMind lists Lyria for music and Gemini Audio for speech and audio.

Many products now combine these. A chat assistant may call an image model, or run code it has just written. When a model is given tools and allowed to take several steps on its own, it becomes an AI agent.

Where it fails

Making things up. NIST calls this confabulation: content that is "confidently stated but erroneous or false". The term most people use is hallucination. NIST says it follows from the design itself: a model that produces statistically likely output can produce likely-sounding output that is wrong. A 2025 paper by Adam Tauman Kalai and colleagues adds a second cause: training and benchmark scoring tend to reward a confident guess over admitting uncertainty, so models learn to guess.

Bias. NIST lists harmful bias and homogenisation as a core risk. Models can amplify biases in their training data, perform worse for some groups or languages, and produce samey output that skews decisions.

Copyright. Models learn from huge collections of text and images, and who owns that material is being fought over in court. On 20 July 2026 a US court gave final approval to a $1.5 billion settlement between Anthropic and authors, Bloomberg Law reported. The class action was over the company downloading millions of pirated books, and the deal covers more than 480,000 works. NIST separately lists the easy copying of copyrighted or licensed content as a risk of generative AI.

Our take: the settlement turned on how the books were obtained. It does not settle the wider question of when training on someone's work is allowed, and other cases on that are still open.

How to judge a generative AI claim

1. Can you check the output? Generated text reads well whether it is right or wrong. Use it where you, or a test, can verify the result, such as code that has to run.
2. Does it cite sources you can open? A confident answer with no source is a guess until proven otherwise.
3. What was it trained on? If a company will not say, you cannot judge bias or copyright risk.
4. Was the demo picked? Generators can produce many attempts. Ask how often the first try is good.
5. Who is responsible when it is wrong? That answer tells you how much the seller trusts it.