ESSAY · PROJECT 0003
An agent is a loop, not a model
Entry 9 of the ARENA guide. Part 3.4, LLM agents. Route position: program week 8, tagged [core] (§1–3 core, §4 elicitation [light]; ARENA budgets the whole set "up to 2 days"). Reading thread: MacDiarmid et al., natural emergent misalignment and reward hacking.
The mental model
An agent is a model with tools and a loop. The model emits a tool call instead of a
final answer; the harness executes the call and hands back the observation; the model
goes again. Everything ARENA builds this week is that loop wearing different clothes:
an arithmetic agent whose tool is a calculator, then a Wikipedia-game agent whose
tools are link navigation, then chat history management so the loop survives more
than a handful of turns.
Two ideas deserve the week's attention. First, the loop is where control lives: the
harness decides which tools exist, what permissions they carry, and when the episode
ends, which is why agent evaluations can be safety-relevant at all. Second, agents
fail in loop-shaped ways: they revisit, they loop on the same failed call, they
burn tokens. The scaffolding choices (history truncation, max iterations, tool
design) are the difference between an eval and a fortune-burning session.
The picture
The exercises
Quoted from the pinned 3.4 page, with ARENA's own budgets:
- build a simple arithmetic task: 20–25 min
- write
CalculateTool: 10–15 min - build an
ArithmeticAgent: 20–25 min - use eval to run the agent: 5–10 min
- implement
get_permitted_links(): ~10 min - build tools for the WikiGame: 15–20 min
What it unlocks
The capstone's episodes (plotted for week 10 as smoke tests) are exactly this loop
around a market environment. The route's W44 gate assumes you can read an agent
scaffold and name its permissions, and the control chapter next week assumes the
loop exists to be attacked.
Budget and receipts
ARENA's whole-set budget is "up to 2 days"; the listed exercises sum to roughly
1.5–2 hours. The route takes §1–3 as core and defers §4 elicitation to [light].
[TODO: receipts — hours vs budget, WikiGame agent behavior, what broke (fill from the TARA vault when the week closes)]