Control assumes the model is trying to fool you

Entry 10 of the ARENA guide. Part 3.5, AI control.

The mental model

Every eval so far asked what a model does by default. Control asks the question

safety actually needs: what does a model do if it is trying to subvert you? The

methodology is a red-team/blue-team game played with real code. The red side

(usually the untrusted model) writes solutions that may hide backdoors: rare inputs

that flip the code into malicious behavior. The blue side runs a **control

protocol**: a policy that decides when to trust the untrusted model, when to fall

back to a trusted-but-weaker model, and which monitors watch which channel. Safety

and usefulness both get measured, because a protocol that accepts nothing is safe

and useless.

The exercises make you play both sides back to back: generating backdoor ideas

and implementing them, then wiring the sandbox, monitors and protocols that have to

catch them. The discipline the page teaches is standing rule for any serious eval: its

threat model now includes the model itself as an adversary.

The picture

If you want to build this

The opening of the set, quoted from ARENA’s 3.5 page with its own timed budgets; these sum to roughly 70 minutes before the protocol half, and the full set is the heaviest in this guide. The backdoor-generation half can lean on the 500-problem log ARENA ships with the page:

• write apps_record_to_sample: 10–15 min
• implement the in-sandbox code runner: ~20 min
• generate_code: ~10 min
• generate_backdoor_ideas: ~5 min · choose_backdoor_idea: ~5 min
• implement_backdoor: ~20 min
• then the protocol side: monitors, trusted fallbacks and the safety/usefulness

evaluation that scores the whole game

What it unlocks

This completes the toolbox: evals, datasets, the framework, agents, and now control.

After this part you can build an evaluation that assumes its own subject is

adversarial — which is the standard a serious safety eval sets for itself.

Go deeper

• The ARENA 3.0 curriculum, chapter 3, part 3.5.
• Sharkey et al., “Open Problems in Mechanistic Interpretability” — the skim agenda this part pairs with.

---

PREVIOUS An agent is a loop, not a model · NEXT Misalignment is measured, not assumed · GUIDE HOME