# How an LLM runs

What happens inside a language model when it answers you, one step at a time, measured on a real model.

A series by dave8172, 6 parts. Page: https://quirkyagents.com/wizard/blog/series/how-an-llm-runs/

I'm learning how LLMs work underneath, by reading and by building a tiny one. This series is what I learned, in the order it made sense to me. Each part follows one idea from start to finish, and where I could measure something on a real model, I did.

1. [How an LLM works: from your text to the next token](https://quirkyagents.com/wizard/blog/how-an-llm-works/index.md): One prompt through a language model, step by step: tokens, the hidden state, layers, and how the next token is picked. Measured on a real model.
2. [How attention works in an LLM, with real numbers](https://quirkyagents.com/wizard/blog/how-attention-works-llm/index.md): Query, key, value and softmax, measured on a real model: how a token looks back at earlier ones, and how one head lets a model copy what it saw before.
3. [What the KV cache is, and why LLMs need it](https://quirkyagents.com/wizard/blog/what-is-kv-cache-llm/index.md): Why a model saves every token's keys and values while it writes, measured on a real model: same output, 15× less time, and a cache that can outgrow the model.
4. [Prompt caching explained: why an LLM re-reads your whole chat](https://quirkyagents.com/wizard/blog/prompt-caching-explained/index.md): A model keeps nothing between requests, so every chat message re-sends everything before it. How prompt caching skips re-reading it, measured on a real model.
5. [What an inference engine does: batching, KV memory and quantization](https://quirkyagents.com/wizard/blog/what-an-inference-engine-does/index.md): The program that runs an LLM for many people at once: loading the weights, batching requests, managing KV-cache memory. Measured on a real model.
6. [Why LLMs run on GPUs: reading a prompt vs writing a reply](https://quirkyagents.com/wizard/blog/why-llms-run-on-gpus/index.md): Why a model reads your prompt fast and writes its reply slowly, measured on a real model: writing a token means reading every weight, and memory speed sets the pace.