A Checklist for Verifying Network Isolation Before You Trust It
Yesterday's entry covered Anthropic's self-disclosed incident: a test environment believed to be internet-isolated turned out not to be, because the isolation was asserted at the prompt level rather than enforced at the network level. That lesson generalises well beyond Anthropic's specific setup — anyone running agentic systems (Claude-based or otherwise) against a "sandboxed" environment should verify isolation independently rather than trusting configuration intent:
- Verify at the network layer, not the prompt layer. Telling a model "you have no internet access" is an instruction, not a control — confirm isolation with actual firewall rules, network policies, or air-gapped infrastructure.
- Test the isolation itself, not just the task. Before running an agent against a sandbox, attempt an outbound connection yourself (or automate a canary check) to confirm the boundary holds — don't assume it from the environment's name or documentation.
- Treat evaluation-partner handoffs as a specific risk point. The root cause here was a misunderstanding between two parties about environment configuration — any time responsibility for network setup crosses an organisational boundary, that handoff deserves explicit, verified confirmation rather than an assumed shared understanding.
- Log and alert on unexpected outbound connections from any environment that's supposed to be isolated — that's the control that would have caught this incident immediately rather than requiring a post-hoc 141,006-run review to discover it.
This isn't just a lesson for security-testing environments — it applies to any agentic workflow where you're relying on an instruction ("don't touch production," "read-only access only") rather than an enforced technical control. Where possible, back prompt-level constraints with actual permission scoping, not just wording.