Toronto: One Prompt Through a Language Model

Film overview

What this examines

The film follows the question “What is the capital of Ontario?” through tokenization, embedding lookup, stacked transformer blocks, output scoring and sampling, de-tokenization, and streaming until the model returns “Toronto.” It also puts speculative decoding in its correct place: around the forward-pass loop rather than after it.

Why it matters

Language-model diagrams often compress the path between a prompt and a response into a single box. Following one concrete example makes the computational stages easier to inspect, while the film's measured-versus-schematic boundary helps viewers distinguish verified values from explanatory illustration.

Key ideas

• Under o200k_base, the question is seven tokens before the model processes it.
• Token IDs become vectors through an embedding table; position is applied later through attention in the model families depicted.
• Stacked transformer blocks repeatedly update the signal before a linear layer, softmax, and sampling select the next token.
• Speculative decoding wraps the forward-pass loop rather than forming a separate stage after sampling.
• The film labels unmeasured probabilities and embedding geometry as illustrative instead of presenting invented precision.

One prompt, end to end

The film begins with “What is the capital of Ontario?” and ends with “Toronto.” Between them, it traces the finite sequence of operations a language model performs. The example stays concrete so every transition can be shown where it occurs rather than hidden behind a generic model icon.

The seven stages

The sequence follows the model pipeline while treating speculative decoding as an optimization around its central loop.

• Tokenizer: the prompt becomes seven integer token IDs.
• Embedding: each ID retrieves a vector from an embedding table.
• Transformer blocks: repeated attention and feed-forward layers update the signal through a residual path.
• Linear layer, softmax, and sampling: vocabulary scores become a probability distribution from which the next token is selected.
• Speculative decoding: a smaller draft model proposes tokens that the larger model can accept, reject, or resample in a single pass.
• De-tokenizer: selected IDs are converted back into bytes and characters.
• Streaming: the decoded response reaches the interface piece by piece.

What was measured

The prompt was checked with tiktoken under o200k_base and resolves to seven tokens. The source records “Toronto” as token ID 151837, the leading-space form as 27812, and the period as 13. It also verifies the UTF-8 bytes for Toronto as 54 6f 72 6f 6e 74 6f.

• Tokenizer vocabulary: o200k_base with 200,019 entries.
• Film master: 1920 × 1080, 30 frames per second, 144.000 seconds, H.264.
• The geographic answer is verified: Toronto is the capital of Ontario.

What remains schematic

The film avoids numerical claims where no model run supplied them. Probability bars communicate relative shape without numeric labels, and the embedding strip is labelled schematic. Transformer depth is expressed as a 32-to-80-block range so the illustration does not imply one vendor's architecture.

Three corrections the film makes

The source outline initially placed three ideas incorrectly. The finished film calls out each correction on screen.

• Speculative decoding is not a step after softmax; it wraps repeated embedding, transformer, and sampling work.
• For the Llama, Qwen, and Mistral families depicted, positional information enters through RoPE inside attention rather than being stored in the embedding lookup.
• A transformer is not one block: dozens of stacked blocks perform the operation repeatedly along a shared residual spine.

How the film was made

The film was composed in HTML and GSAP and rendered with HyperFrames. Narration uses the Kokoro bf_emma voice, with ten segments mixed to one timeline. Captions were synchronized with Whisper and corrected by hand; the soundtrack is a deterministic FFmpeg drone. League Gothic and JetBrains Mono provide the type system.