ESSAY · PROJECT 0003
An eval is a threat model with a scoreboard
Entry 6 of the ARENA guide. Part 3.1, intro to evals.
The mental model
Chapter 3 is the toolkit chapter, and this part exists because an eval is more
than a quiz with grades. The part's actual order of operations, straight from the page: pick a property you worry a model might have, write the **threat
model that says when and how it would matter, turn that into a specification**
(otherwise the eval measures whatever is easy), and only then engineer the questions.
The replication exercise lands early for the same reason: ARENA has you replicate
alignment faking before you write a single question, so you feel what a real eval
result looks like from the inside.
The vocabulary shift is the point. Capability evals ask "can the model do X"; property
or control evals ask "does the model do X under conditions that matter for safety".
Same machinery, different claim. Every exercise feeds a threat model of your own,
which is exactly how the chapter intends it.
The picture
If you want to build this
Quoted from ARENA’s 3.1 page, with its own timed budgets; they sum to roughly 2.5–3.5 hours, and the threat-model and specification exercises are where the hours should go (the optional backoff can slip without guilt):
- retry with exponential back-off (optional): 10–15 min
- replicate alignment faking: 25–45 min
- build threat models: 40–60 min
- design a specification: 45 min
- engineer the eval question prompt: 35–40 min
What it unlocks
Everything downstream is this part's pipeline wearing different clothes: 3.2 fills the
questions, 3.3 runs them, 3.4 points them at agents, 3.5 assumes the model is
adversarial — and whatever evaluation project you build starts with the threat model
you write here.
Go deeper
- The ARENA 3.0 curriculum, chapter 3, part 3.1.
- The reading that pairs with this part: Seredyński et al., on AI agents in algorithmic electricity markets.