ESSAY · PROJECT 0002
A word becomes a place, and attention does the reading
The second session was the transformer, the architecture behind every chatbot. It has a reputation for difficulty it does not deserve. This page rebuilds it in plain words, no code and no math: a word becomes a place, every word keeps a running notebook, and attention is how each word gathers its context from the words before it, never after.
Where this comes from
- The curriculum: ARENA 3.0, part 1.1: Transformer from Scratch, where you can implement every box on the page.
- For another plain-words pass: The Illustrated Transformer, the classic visual walkthrough.
- The word map, the attention weights, and the layer notes are hand-set and illustrative; the pronoun example is real in spirit: resolving "it" is a standard attention behavior.
- Next in the series: how the model turns its final state into an actual word choice.