ESSAY · PROJECT 0003
Misalignment is measured, not assumed
Entry 11 of the ARENA guide. Part 4.2, science of misalignment.
The mental model
Chapter 3 built instruments; chapter 4 points them at alignment questions and treats
the answers as science: hypotheses, experiments, replications. This part runs two of
the field's live experiments on real models through your own pipeline. Shutdown
resistance: give a model a task, then a shutdown instruction, and measure what it
does when finishing the task and obeying the instruction conflict. Alignment
faking: give a model a scratchpad and a training setup that threatens its values,
and look for the signature pattern of complying now while preserving the option to
defect later.
The frame that makes this science rather than anecdote is in the exercises' shape:
system-prompt variants are the experimental conditions, the main experiment loop is
the trial harness, and the replication exercises connect your numbers to published
ones. A result you cannot reproduce on your own API key is a rumor; this week's job
is turning two famous results into numbers you own.
If you want to build this
Quoted from ARENA’s 4.2 page, with its own timed budgets; they sum to roughly 1.5–2.5 hours of work plus about $10 of API spend (ARENA's cost note for the part). No GPU appears anywhere:
- implement bypass detection: 10–15 min
- integration: complete bash simulator: 15–25 min
- main experiment loop: 25–35 min
- write system prompt variants: 5–15 min
- post-task shutdown control: 15–20 min
- replicate scratchpad alignment faking: 20–30 min
What it unlocks
Any evaluation claim you make will be judged by exactly this standard: conditions,
controls, replications. The part is also where alignment science starts to feel like
science day to day — most of the work is careful experimental plumbing, and the
famous result is the last mile.
Go deeper
- The ARENA 3.0 curriculum, chapter 4, part 4.2.
- Wang et al., on persona features controlling emergent misalignment.