ESSAY · PROJECT 0003
Einsum is a contract between dimensions
Entry 1 of the ARENA guide. Part 0.0, prerequisites: the warm-up, meant to be verified rather than studied.
The mental model
Every tensor operation in ARENA is written with einsum, and an einsum string is not notation to parse. It is a contract you can check by eye. Take the line the whole curriculum leans on:
"batch samples features, out features -> batch samples out"
Three rules, readable from the string alone. A dimension that appears in one operand and in the output belongs to that operand only: out comes from the weights. A dimension that appears in both operands and survives into the output was never shared work at all: batch and samples just ride along. A dimension that appears in both operands and vanishes from the output is the work: features is the axis the multiplication contracts over. That is the entire operation, x @ W.T, stated as three ownership rules.
The reason this part exists is that shape errors stop being puzzles once you read them this way. "mat1 and mat2 shapes cannot be multiplied" is the contract being violated, and the einsum string tells you which clause broke. ARENA's quiz makes you earn this once: four candidate strings, one correct contraction, three that leak or lose the wrong axes.
The picture
If you want to build this
One exercise block, quoted from ARENA’s 0.0 page with its own timed budget:
- einops operations (match images): ~45 minutes collectively: indexing, image rearrangement, broadcasting and
rearrange, theneinsumfor outer product, matrix-vector, matrix-multiply, trace and inner product.
Everything else in 0.0 is a self-audit (maths you half-know, tooling you half-set-up), meant to be skimmed against your own gaps, not studied.
What it unlocks
Every later chapter writes its math in einsum. The 1.1 attention implementation, the single hardest exercise of the program's first month, is unreadable without this contract, and trivial with it. That is why ARENA itself calls this part light: verify it, don't redo it, and spend the saved hours on chapter 1.
Go deeper
- The ARENA 3.0 curriculum, chapter 0, part 0.0 — where the exercises on this page live.