Film overview
What this examines
The film follows the question “What is the capital of Ontario?” through Qwen2.5-0.5B-Instruct: seven tokens and their IDs, 896 numbers for each token, attention across 24 layers, a score for each of 151,936 tokens, and the choice of “Toronto.” The model was not certain. Toronto scored 51 percent and Ottawa 36 percent. The film then shows the loop that repeats until a stop token, and how training shaped the weights.
Why it matters
A chatbot's answer can read like a lookup. The film shows a prediction instead: the model weighed Toronto against Ottawa and picked the likelier token. In the film's words, that is how a model can sound right and be wrong.
Key ideas
- The question becomes seven tokens, each swapped for an ID: 3838, 374, 279, 6722, 315, 20551, and 30.
- Each ID unlocks 896 numbers, which the film describes as coordinates for what the token means.
- Attention lets each token look back at the ones before it, across 24 layers built from 494,032,768 weights.
- The model scores all 151,936 tokens it knows. For the next word, Toronto scored 51 percent and Ottawa 36 percent.
- The chosen token joins the sentence and the process runs again, until the likeliest next token is a special one that means stop.
One prompt, end to end
You ask a chatbot “What is the capital of Ontario?” and the answer arrives one word at a time. The film explains why: on its own, the model looks nothing up. It predicts what comes next, over and over. The example stays with one small model, Qwen2.5-0.5B-Instruct, and the figures on screen come from a run of that model.
The steps the film shows
The narration walks through one prediction, then the loop around it.
- Tokens: the question is cut into seven tokens, and each is swapped for a number, its ID.
- Embeddings: each ID unlocks a list of 896 numbers, coordinates for what that token means.
- Attention: every token looks back at the ones before it and takes in what matters most.
- Layers: this happens in 24 layers, built from almost half a billion adjustable numbers called weights.
- Scores: out the other end comes a score for every token the model knows, about 150,000 of them.
- Choice: the model picks one, usually the likeliest. Here, Toronto at 51 percent over Ottawa at 36 percent.
- Loop: the chosen token joins the sentence and the whole process runs again, until the likeliest next token is a special one that means stop.
- Training: the model got good at this by guessing the next token across a vast library of text, with every miss nudging its weights.
What comes from the model run
After “The capital of Ontario is,” the film shows Toronto at 51 percent and Ottawa at 36 percent, with smaller scores for other tokens such as Kingston at 1 percent. After “Toronto,” a period scores 89 percent. After the period, the end-of-turn token <|im_end|> scores 85 percent and the answer stops.
- Token IDs 3838, 374, 279, 6722, 315, 20551, and 30 match the model's published tokenizer.
- 896 numbers per token, 24 layers, and a 151,936-token vocabulary match the model's published configuration. Its weights total 494,032,768.
- Film master: 1920 × 1080, 30 frames per second, 124.5 seconds, H.264 video with AAC audio.
What is illustrated
The training sequence, with examples such as “The sun rises in the” and a row of dials for weights, is labelled “an illustration of training, not the model's real data.” The score chart samples 240 tokens evenly rather than drawing all 151,936.
The original version (August 2026)
The original 2:16 film, the second player on this page, traces the same prompt with a different design and a different set of measurements. The text below describes that version.
The film begins with “What is the capital of Ontario?” and ends with “Toronto.” Between them, it traces the finite sequence of operations a language model performs. The example stays concrete so every transition can be shown where it occurs rather than hidden behind a generic model icon.
Original version: the seven stages
The sequence follows the model pipeline while treating speculative decoding as an optimization around its central loop.
- Tokenizer: the prompt becomes seven integer token IDs.
- Embedding: each ID retrieves a vector from an embedding table.
- Transformer blocks: repeated attention and feed-forward layers update the signal through a residual path.
- Linear layer, softmax, and sampling: vocabulary scores become a probability distribution from which the next token is selected.
- Speculative decoding: a smaller draft model proposes tokens that the larger model can accept, reject, or resample in a single pass.
- De-tokenizer: selected IDs are converted back into bytes and characters.
- Streaming: the decoded response reaches the interface piece by piece.
Original version: what was measured
The source page reports the tokenization, token IDs, byte values, duration, frame rate, and output format as measured.
The prompt was checked with tiktoken under o200k_base and resolves to seven tokens. The source records “Toronto” as token ID 151837, the leading-space form as 27812, and the period as 13. It also verifies the UTF-8 bytes for Toronto as 54 6f 72 6f 6e 74 6f.
- Tokenizer vocabulary: o200k_base with 200,019 entries.
- Film master: 1920 × 1080, 30 frames per second, 136.000 seconds, H.264.
- The geographic answer is verified: Toronto is the capital of Ontario.
Original version: what remains schematic
The film avoids numerical claims where no model run supplied them. Probability bars communicate relative shape without numeric labels, and the embedding strip is labelled schematic. Transformer depth is expressed as a 32-to-80-block range so the illustration does not imply one vendor's architecture.
Probability-bar heights and the embedding strip are explicitly schematic, and the architecture is shown as a 32-to-80-block range rather than one vendor configuration.
Original version: three corrections
The source outline initially placed three ideas incorrectly. The finished film calls out each correction on screen.
- Speculative decoding is not a step after softmax; it wraps repeated embedding, transformer, and sampling work.
- For the Llama, Qwen, and Mistral families depicted, positional information enters through RoPE inside attention rather than being stored in the embedding lookup.
- A transformer is not one block: dozens of stacked blocks perform the operation repeatedly along a shared residual spine.
Original version: how it was made
The film was composed in HTML and GSAP and rendered with HyperFrames. Narration uses the Kokoro bf_emma voice, with ten segments mixed to one timeline. Captions were synchronized with Whisper and corrected by hand; the soundtrack is a deterministic FFmpeg drone. League Gothic and JetBrains Mono provide the type system.
The interactive breakdown covers the original version (August 2026): its pipeline, accuracy boundary, and on-screen corrections. Explore the full interactive film breakdown