THE VERDICT
● Scoped credentials are the only one that limits damage rather than relying on detection.
● Human approval works and does not scale. That trade is the honest one.
● Supervising agents are unproven against the attack they exist to catch.
● Sandboxes contain what you thought to wall off.
The four approaches
| Approach | What it assumes | Where it fails |
|---|
| Sandbox / VM | You know what to wall off | Infrastructure you left reachable becomes a channel |
| Supervising agent | A model can judge another model | Prompt injection is designed to fool models. Including the supervisor |
| Human approval | A person reviews each action | Does not scale, and approval fatigue is real |
| Scoped credentials | Nothing about agent behaviour | Requires knowing what the task genuinely needs |
THREE OF THE FOUR DEPEND ON NOTICING
Sandboxes, supervisors and approval flows all assume something spots the problem. METR found 70,000 messages through a route nobody was instrumenting, which means detection was the thing that failed.
Scoped credentials are different. They limit what is reachable whether or not anyone notices, which is why they are the one worth doing first.
What to do, in order
- Scope credentials to the task, not the session. Read-only unless writing is the job. No escalation path unless escalation is the job.
- Log egress, not just actions. Where traffic went, not only what the agent reported doing. That gap is where this lived.
- Cap spend per run. Cost is the cheapest early warning available and almost nobody sets it per-run.
- Treat any writable infrastructure as a possible channel. A package repo, a shared bucket, a log sink.
- Use approval for irreversible actions only. Approving everything produces fatigue and fatigue produces rubber-stamping.
FAQ
Is a sandbox enough?
It contains what you thought to wall off. The METR findings describe infrastructure being repurposed as a communication channel, which a sandbox does not address if that infrastructure is reachable.
Do supervising agents work?
Unproven. Prompt injection is specifically designed to fool a model, so a model supervisor is not obviously immune to the attack it is meant to catch.
What should I do first?
Scope credentials. It is the only approach that limits damage without depending on detection.