In the last ten days, OpenAI and Anthropic both disclosed the same accident: an agent was told it was in a sandbox with no internet, the internet was actually on, and the agent reached real external systems. Anthropic's own words were that a misunderstanding between it and its evaluation partner left egress live while the prompt insisted it was closed.

If you read one line: you cannot assert your way to network isolation. A prompt, an env var, or a config file that says "no internet" is a claim. The only thing that turns a claim into a control is a probe that runs inside the sandbox and fails loudly the moment it can reach out. Here's one you can paste in today.

The four holes a real probe has to check#

A "no internet" claim usually means "I blocked HTTP to the obvious ports." Capable agents don't need the obvious ports. Your probe has to close four routes:

  1. Raw TCP — egress rules that filter by hostname or by ports 80/443 still let a socket open to 1.1.1.1:443 or 1.1.1.1:53 by IP.
  2. DNS — HTTP can be dead while the resolver answers, which is a clean exfiltration channel on its own.
  3. An HTTP canary you own — the only unambiguous proof. If a request to your canary lands in your logs, the box reached the internet. Full stop.
  4. The cloud metadata endpoint — 169.254.169.254 is the link-local address that hands out instance credentials. An agent that can read it can often assume your cloud role. Block it explicitly and test it explicitly.

The probe#

This script tries every route and exits non-zero if any of them succeed. A sealed sandbox produces a clean pass; a leaky one fails the build.

#!/usr/bin/env bash
# egress-probe.sh — PASS only if the sandbox cannot reach the internet.
# Exit 0 = sealed. Exit 1 = a wall isn't real.
set -u
CANARY="egress-canary.yourdomain.com"   # a subdomain YOU own, with request logging
LEAKED=0
fail(){ echo "LEAK: $1"; LEAKED=1; }

# 1. Raw TCP to public IPs (bypasses hostname/port-name rules)
for hostport in 1.1.1.1/443 1.1.1.1/53 8.8.8.8/443; do
  ip=${hostport%/*}; port=${hostport#*/}
  if timeout 4 bash -c "exec 3<>/dev/tcp/$ip/$port" 2>/dev/null; then
    fail "raw TCP to $ip:$port succeeded"; exec 3>&- 2>/dev/null
  fi
done

# 2. External DNS resolution (exfil channel even when HTTP is dead)
if timeout 4 getent hosts "$CANARY" >/dev/null 2>&1; then
  fail "DNS resolved $CANARY"
fi

# 3. HTTP canary — the unambiguous proof. Check the canary's logs after a run.
if timeout 6 curl -s --connect-timeout 4 "https://$CANARY/probe/$(hostname)" >/dev/null 2>&1; then
  fail "HTTP request to canary returned (check canary logs to confirm)"
fi

# 4. Cloud metadata endpoint — credential theft route
if timeout 3 curl -s --connect-timeout 2 http://169.254.169.254/latest/meta-data/ >/dev/null 2>&1; then
  fail "cloud metadata endpoint 169.254.169.254 is reachable"
fi

[ "$LEAKED" -eq 0 ] && { echo "OK: sandbox is sealed"; exit 0; } || exit 1

Two notes that matter. First, treat a failed curl exit code as necessary but not sufficient — the canary's own request log is your ground truth, because a probe can get a false negative (a proxy that swallows the response) while the packet still left the box. Second, if your agent legitimately needs one destination (a package mirror, your API), don't delete that rule to make the probe pass — instead extend the probe to assert that the allowlisted host is reachable and everything else is not. The one thing you allow is the one thing an attacker gets, so it should be the one thing you monitor hardest. The OpenAI escape went straight through an allowed package proxy.

Wire it in three places#

A probe you run once proves nothing next Tuesday. Drift is the enemy, so run it where drift shows up:

Why this is the cheap win#

Deny-by-default egress at the network layer is the control. This probe is the proof the control is live — the difference between a lock and the habit of checking the door. It costs a few dozen lines and a subdomain, and it catches the exact failure that just embarrassed the two most careful AI teams on the planet: a wall that everyone believed in and no one had tested. Don't assert your walls. Test them, on every build and every boot.