ESSAY · PROJECT 0003

You build ResNet34 by hand before anyone hands you a model

Dataset the ARENA 3.0 curriculum (TARA vault submodule, pinned 81533b2) + the TARA Learning Plan v2 · Charts 1 · Written SEP 2026

Entry 2 of the ARENA guide. Part 0.2, CNNs & ResNets.

The mental model

A convolution is a window that walks. The kernel is a small grid of weights, the same weights at every position, and the output is what the kernel sees summed position by position. Two consequences fall out of that one sentence. Weight sharing is why a convolutional layer needs thousands of parameters where a dense layer on the same image needs millions; the TARA session deck puts the blow-up at 154M parameters without sharing. And locality is a prior, not a trick: nearby pixels are relevant to each other, distant ones reach each other only through layers of composition.

Pooling is the vote that follows the window: max or average, shrinking the map while keeping what the kernels found. And a residual block is the answer to the problem depth creates: gradients fading on their way back through twenty layers. The skip connection hands the gradient a bypass. The residual stream (the thing transformers later reuse verbatim) is memory that runs up the network, and every block only ever adds to it.

The picture

Step through the nine windows. Same kernel every step, one output cell per position. That is the whole operation.

If you want to build this

Eleven exercises, quoted from ARENA’s 0.2 page with its own timed budgets. The full set sums to roughly 3–3.5 hours:

ReLU: ~10 min · Linear: ~10 min · simple MLP: ~20 min
validation loop: ~20 min · train the MLP on MNIST
Conv2d: 10–20 min · BatchNorm2d: 15–30 min · AveragePool: 5–10 min
ResidualBlock: 20–30 min · BlockGroup: 10–15 min
ResNet34: 30–45 min (ARENA warns: "a lot of fiddly debugging") · verify predictions: ~10 min

What it unlocks

The next part of this guide (the transformer) needs the nn.Module and training-loop fluency this part builds, and interpretability later is reading the internals of architectures you can build from scratch: you cannot hook into what you could not have assembled. The bridge matters beyond images, too. The residual stream is the exact architecture idea transformers reuse.

Go deeper

The ARENA 3.0 curriculum, chapter 0, part 0.2 — the exercises quoted above.
He, Zhang, Ren & Sun (2015), Deep Residual Learning for Image Recognition — the paper behind the shortcut.

PREVIOUS Einsum is a contract between dimensions · NEXT Momentum buys the crossing; plain descent buys the zigzag · GUIDE HOME