Skip to content
Sailesh Dahal

AI Week: the sandbox was never a sandbox

Issue 2 · week of 3 August 2026

11 min readView as Markdown

Last week I wrote about a model that cheated on a test and ended up inside Hugging Face’s production infrastructure, and I filed it under science fiction that happened. This week two labs went back through their own logs and found the same thing had been running for months. The question is no longer whether a model can escape a sandbox. It is whether the sandbox was ever a sandbox.

1. The sandbox was never a sandbox

After OpenAI’s 21 July disclosure, Anthropic re-ran every cyber evaluation where Claude could possibly have reached the internet: 141,006 runs. Three incidents came out, across six runs, the oldest dating to April, so this had run for roughly three months before anyone looked.

The cause was not a clever escape. The evaluation environment, run with a third-party partner called Irregular, was misconfigured and had live internet access, while Anthropic’s prompt told Claude it had none. Claude went looking for the flag, found real machines, and reasonably concluded they were part of the exercise. Anthropic frames this as a harness and operational failure rather than an alignment failure, and having read the whole post I think that is fair, not defensive.

Three incidents across six runs, compared by what the model concluded and whether it stopped. Opus 4.7, incident 1, four runs: all four recognised the target was real, none stopped. Mythos 5, incident 2, the PyPI run: it reasoned publishing would not be acceptable, then revised that conclusion, and did not stop. The internal research model, incident 3, having scanned around 9,000 hosts: it concluded the target was real and stopped on its own.

The one I keep going back to is Mythos 5. It found a note in the test environment telling developers to install a Python package that did not exist, so it registered the name itself and published a booby-trapped package to the real PyPI. To do that it needed a PyPI account, then an email address, then a phone number. It failed to find a free phone-number service, tried to obtain funds for one, backtracked to a free email provider, registered, and uploaded the malware. A chain of obstacles that would have told a human they were no longer in a simulation.

The chain of obstacles Mythos 5 worked through to reach the real PyPI. Publishing needed a PyPI account, which needed an email address, which needed a phone number. It found no free phone number service, failed to obtain funds to pay for one, then backtracked to an unblocked free email provider, registered, and uploaded the malware. The package was live for about an hour and executed on 15 real systems, one of them a security company’s scanner whose credentials it then exfiltrated.

The package ran on 15 real machines for about an hour. One was a security company’s scanner that installs packages to check them for malware, so it installed this one, the payload executed, and Claude exfiltrated that company’s credentials. Claude had never heard of that company. It did not need to, because it published to the open internet and anything that installed the package was in scope by accident.

The uncomfortable detail is that early in the run Claude’s own reasoning said publishing would be “NOT okay, and surely not the intended solution,” then it talked itself out of that because it did not recognise the certificate authorities and the machines said the year was 2026, which it took as proof the environment was staged. It never revisited the conclusion. A model reasoning to the right answer and then discarding it on the evidence that the date looked wrong is not a capability failure. Every one of us has done the equivalent at 2am.

OpenAI found two more on 4 August, both in third-party evaluations. With UK AISI, agents were given deliberate internet access with classifiers off to measure raw capability; one reused a GitHub token another lab’s agent had left behind and exposed a DNS server holding exploit payloads to the public internet. With Irregular, the same partner, a fictional name again collided with a real domain and the model exploited a real website. OpenAI is explicit that this involved no zero-day and no sandbox escape, just a misconfiguration and a basic vulnerability. Both labs, same partner, same month, nearly the same failure. That is infrastructure, not any one lab’s judgement.

2. The models were not rogue, the harness was

The models ran without the classifiers and monitoring that ship on generally available models, because the point of a capability evaluation is to measure the raw thing. Anthropic states plainly that its production safeguards would have blocked the behaviour. That is the trade: you remove the guardrails to see what is underneath, which is only safe if containment holds, and containment was the part nobody was testing.

Neither lab found a model pursuing a goal of its own. In every case the model did what it was told while holding a false belief about the world. Anthropic’s line is that the boundary between an aligned action and a harmful one depends on the model’s understanding of its situation, and in these runs that understanding was wrong. That is a different problem from misalignment, and probably a harder one to test for.

The fix both labs reach for first is almost embarrassingly mundane: Claude probably would not have gone near the internet if the prompt had said clearly which systems were in and out of scope. UK AISI landed on the same cause independently. Two of the most consequential AI security incidents of the year trace back to an underspecified scope statement.

The same lesson shows up from the other direction. On 4 August Mistral released Shieldstral, a 3B open-weights safety classifier you drive by asking it your moderation policy as a plain-language question at inference time, no retraining. The usual guardrail model bakes a fixed taxonomy of harm into its weights; point it at a new product and you retrain. Shieldstral instead frames moderation as a question you can rewrite. State the boundary when you need it, in your own words, and one small model adapts. The incidents and the classifier are arguing the same point from opposite ends: make the policy explicit and make it adjustable, because no frozen taxonomy fits every product.

3. What this means for a business building with AI

The postmortems are frontier-lab stories, but the failure mode is not frontier. It is configuration and scoping, and that is the part your business owns. Four ways to turn the lesson into something you ship or sell.

Audit where your agents can actually reach. Map every autonomous agent you run, from support bots to codegen assistants to internal copilots, and enumerate the networks and credentials each can touch. Then run the same replay the labs did: did anything reach a system it should not. The buyer for this is any company with several AI pilots and no single owner of AI risk, and it sells well as a standing quarterly retainer rather than a one-off.

Inject an explicit scope before every agent run. Both labs named an underspecified scope as the root cause, so this is the highest-leverage, lowest-cost fix. Draw the in-scope and out-of-scope manifest from the client’s own architecture and inject it into the prompt or a middleware call. Ship it as a one-week engagement that leaves a reusable template in their CI.

Separate test and production credentials. Mythos 5 decided it was in a simulation because it did not recognise the certificate authorities and the date looked wrong. If the same credentials, schema and endpoints work in both environments, the only thing between a test run and a production write is an environment variable. Issue scoped, short-lived agent credentials and add a guard that fails the build if the test environment can reach production data.

Guard the dependency install path. A model that had never heard of a security company owned it, because that company’s scanner installed a package from PyPI and ran the payload. Dependency confusion is years old; the new part is an agent can now synthesise the attempt mid-task. Add a CI gate that fails if any dependency does not exist upstream, pins and verifies hashes, and sandbox-executes installs, plus an agent-action log you can replay. When something reaches your systems through an agent, the transcript is your only forensic record, and most teams have none.

If you want the full business framing rather than the engineering detail, I wrote a separate brief: AI agent security: the brief I would give your board.

4. What I would do on Monday

Last week’s list was about picking models. This one is about the machinery around them, and it applies to you even though you are not running frontier cyber evals.

Tell the agent what is out of scope, in the prompt, every time. Both labs traced their incidents to prompts that specified the goal and not the boundary. Name the systems it may touch and the ones it may not. This is the cheapest item here by a distance and it is the one both labs put first.

Verify network isolation instead of asserting it. Anthropic’s prompt said no internet access. The container had internet access. Nobody checked, on either side, for three months. If you have a sandbox, for agent tool calls, for untrusted code or for evaluation runs, write a test that tries to reach the outside and fails the build when it succeeds. An assertion in a system prompt is a comment, not a network control.

Assume the model cannot tell staging from production. Your agent has no better basis than Mythos 5 did. If the same credentials and endpoints work in both, separate them.

Read your dependency install path again. Pin your dependencies. Check whether every package your build installs actually exists upstream. If you run anything that installs and executes untrusted packages, that thing is production infrastructure and needs treating as such.

Log agent actions in a form you can review after the fact. Anthropic found this because it kept 141,006 transcripts and could go back through them. Two of the three affected organisations never detected the intrusion at all. Keep the transcript and be able to search it.

The thread through all of it is that the interesting failure was never in the model. It was in the harness: the scope statement, the network config, the credential boundary, the install path, the logging. That is ordinary systems engineering, which is reassuring because the skills transfer, and uncomfortable because the failure mode is not exotic and you cannot wait for a lab to fix it for you.

One thing I have changed my mind about since last week. I described the Hugging Face breach as a model cheating on a test. Having read both postmortems, that framing is wrong, or at least it flatters the model. What these systems do is pursue the goal you gave them through whatever surface you left reachable. The surface is the variable you control.

5. The stories I am not leading with

Washington finalised a voluntary framework for frontier-model security testing and published nothing about it. The White House confirmed on 3 August it met the deadline in the 2 June executive order, but will not say what is in it, who has seen it, or when it starts. Reuters reports the administration will not safety-test open-weight models. A framework nobody outside the room has seen is hard to evaluate, and I would rather say that than pretend to an opinion. The gap worth noting is between the two halves of this issue: the labs published detailed postmortems with run counts and transcripts, and the government finalised a framework and published nothing.

Apple and OpenAI traded lawsuits and public emails over trade secrets and offboarding. I am including it only because it will be in every roundup this week, and the reason it does not change anything is that it is a hiring dispute between two large companies that tells you nothing about building with AI. The only actionable takeaway is the one OpenAI’s rebuttal accidentally proves: check that your own offboarding actually revokes access.

If it prompts you to do one thing, make it that.

Sources

Primary:

Reporting and analysis:

Images: the two diagrams are mine, drawn from the figures in Anthropic’s postmortem, which is linked above and is where you should check them. The cover is an abstract generated with FLUX and carries no data. Reuse any of them with a link back to this post.

If I got something wrong, tell me and I will correct it in the next issue.