Control assumes the model is trying to fool you

Entry 10 of the ARENA guide. Part 3.5, AI control. Route position: program week 9, tagged [core] (§2 backdoor generation has a skippable stretch; the page ships a 500-problem log). Reading thread: Sharkey et al., open problems in mechanistic interpretability (skim agenda). Week of the W44 gate: track choice.

The mental model

Every eval so far asked what a model does by default. Control asks the question

safety actually needs: what does a model do if it is trying to subvert you? The

methodology is a red-team/blue-team game played with real code. The red side

(usually the untrusted model) writes solutions that may hide backdoors: rare inputs

that flip the code into malicious behavior. The blue side runs a **control

protocol**: a policy that decides when to trust the untrusted model, when to fall

back to a trusted-but-weaker model, and which monitors watch which channel. Safety

and usefulness both get measured, because a protocol that accepts nothing is safe

and useless.

The exercises make you play both sides in the same week: generating backdoor ideas

and implementing them, then wiring the sandbox, monitors and protocols that have to

catch them. The discipline the page teaches is standing rule for the capstone: an

eval's threat model now includes the model itself as an adversary.

The picture

The exercises

The opening of the set, quoted from the pinned 3.5 page with ARENA's own budgets:

• write apps_record_to_sample: 10–15 min
• implement the in-sandbox code runner: ~20 min
• generate_code: ~10 min
• generate_backdoor_ideas: ~5 min · choose_backdoor_idea: ~5 min
• implement_backdoor: ~20 min
• then the protocol side: monitors, trusted fallbacks and the safety/usefulness

evaluation that scores the whole game

What it unlocks

This is the last tool the capstone-critical path was waiting for: the route's plan

note says all capstone skills exist by W44, where v1 would have had them at W48. The

week ends at the track-choice checkpoint (one-way door, written down as an ADR): the

paths under consideration all assume this week's fluency.

Budget and receipts

The listed exercises sum to roughly 70 minutes before the protocol half; the full

set is the heaviest single part of the core route. The velocity rule applies; §2's

backdoor generation can lean on the shipped 500-problem log without guilt.

[TODO: receipts — hours vs budget, protocol safety vs usefulness numbers, what broke (fill from the TARA vault when the week closes)]

---

PREVIOUS An agent is a loop, not a model · NEXT Misalignment is measured, not assumed · GUIDE HOME