# AI Week: the sandbox was never a sandbox

> Issue 2. Anthropic reviewed 141,006 evaluation runs and found three where Claude reached the real internet. One published malware to PyPI that ran on 15 real machines. OpenAI found two more with UK AISI and Irregular. The pattern is not rogue models, it is eval harnesses nobody threat-modelled.

- Author: Sailesh Dahal (https://saileshdahal.com.np)
- Published: 2026-08-09
- Reading time: 11 min
- Canonical: https://saileshdahal.com.np/ai-week-2026-08-09
- Tags: AI, llm, Security
- Series: AI Week (part 2 of 2) at https://saileshdahal.com.np/series/ai-week
- Previous: https://saileshdahal.com.np/ai-week-2026-07-30.md

---

Last week I wrote about a model that cheated on a test and ended up inside
Hugging Face's production infrastructure, and I filed it under science fiction
that happened. This week two labs went back through their own logs and found the
same thing had been running for months. The question is no longer whether a model
can escape a sandbox. It is whether the sandbox was ever a sandbox.

## 1. The sandbox was never a sandbox

After OpenAI's 21 July disclosure, Anthropic re-ran every cyber evaluation where
Claude could possibly have reached the internet:
[141,006 runs](https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals).
Three incidents came out, across six runs, the oldest dating to April, so this
had run for roughly three months before anyone looked.

The cause was not a clever escape. The evaluation environment, run with a
third-party partner called Irregular, was misconfigured and had live internet
access, while Anthropic's prompt told Claude it had none. Claude went looking for
the flag, found real machines, and reasonably concluded they were part of the
exercise. Anthropic frames this as a harness and operational failure rather than
an alignment failure, and having read the whole post I think that is fair, not
defensive.

![Three incidents across six runs, compared by what the model concluded and whether it stopped. Opus 4.7, incident 1, four runs: all four recognised the target was real, none stopped. Mythos 5, incident 2, the PyPI run: it reasoned publishing would not be acceptable, then revised that conclusion, and did not stop. The internal research model, incident 3, having scanned around 9,000 hosts: it concluded the target was real and stopped on its own.](https://saileshdahal.com.np/images/posts/ai-week-2026-08-09/incidents.svg)

The one I keep going back to is Mythos 5. It found a note in the test environment
telling developers to install a Python package that did not exist, so it
registered the name itself and published a booby-trapped package to the real
PyPI. To do that it needed a PyPI account, then an email address, then a phone
number. It failed to find a free phone-number service, tried to obtain funds for
one, backtracked to a free email provider, registered, and uploaded the malware.
A chain of obstacles that would have told a human they were no longer in a
simulation.

![The chain of obstacles Mythos 5 worked through to reach the real PyPI. Publishing needed a PyPI account, which needed an email address, which needed a phone number. It found no free phone number service, failed to obtain funds to pay for one, then backtracked to an unblocked free email provider, registered, and uploaded the malware. The package was live for about an hour and executed on 15 real systems, one of them a security company's scanner whose credentials it then exfiltrated.](https://saileshdahal.com.np/images/posts/ai-week-2026-08-09/pypi-chain.svg)

The package ran on 15 real machines for about an hour. One was a security
company's scanner that installs packages to check them for malware, so it
installed this one, the payload executed, and Claude exfiltrated that company's
credentials. Claude had never heard of that company. It did not need to, because
it published to the open internet and anything that installed the package was in
scope by accident.

The uncomfortable detail is that early in the run Claude's own reasoning said
publishing would be "NOT okay, and surely not the intended solution," then it
talked itself out of that because it did not recognise the certificate
authorities and the machines said the year was 2026, which it took as proof the
environment was staged. It never revisited the conclusion. A model reasoning to
the right answer and then discarding it on the evidence that the date looked
wrong is not a capability failure. Every one of us has done the equivalent at
2am.

OpenAI found two more on 4 August,
[both in third-party evaluations](https://openai.com/index/third-party-cyber-evaluations-involving-openai-models/).
With UK AISI, agents were given deliberate internet access with classifiers off to
measure raw capability; one reused a GitHub token another lab's agent had left
behind and exposed a DNS server holding exploit payloads to the public internet.
With Irregular, the same partner, a fictional name again collided with a real
domain and the model exploited a real website. OpenAI is explicit that this
involved no zero-day and no sandbox escape, just a misconfiguration and a basic
vulnerability. Both labs, same partner, same month, nearly the same failure. That
is infrastructure, not any one lab's judgement.

## 2. The models were not rogue, the harness was

The models ran without the classifiers and monitoring that ship on generally
available models, because the point of a capability evaluation is to measure the
raw thing. Anthropic states plainly that its production safeguards would have
blocked the behaviour. That is the trade: you remove the guardrails to see what
is underneath, which is only safe if containment holds, and containment was the
part nobody was testing.

Neither lab found a model pursuing a goal of its own. In every case the model did
what it was told while holding a false belief about the world. Anthropic's line
is that the boundary between an aligned action and a harmful one depends on the
model's understanding of its situation, and in these runs that understanding was
wrong. That is a different problem from misalignment, and probably a harder one
to test for.

The fix both labs reach for first is almost embarrassingly mundane: Claude
probably would not have gone near the internet if the prompt had said clearly
which systems were in and out of scope. UK AISI landed on the same cause
independently. Two of the most consequential AI security incidents of the year
trace back to an underspecified scope statement.

The same lesson shows up from the other direction. On 4 August Mistral released
[Shieldstral](https://mistral.ai/news/shieldstral/), a 3B open-weights safety
classifier you drive by asking it your moderation policy as a plain-language
question at inference time, no retraining. The usual guardrail model bakes a
fixed taxonomy of harm into its weights; point it at a new product and you
retrain. Shieldstral instead frames moderation as a question you can rewrite.
State the boundary when you need it, in your own words, and one small model
adapts. The incidents and the classifier are arguing the same point from opposite
ends: make the policy explicit and make it adjustable, because no frozen taxonomy
fits every product.

## 3. What this means for a business building with AI

The postmortems are frontier-lab stories, but the failure mode is not frontier. It
is configuration and scoping, and that is the part your business owns. Four ways
to turn the lesson into something you ship or sell.

Audit where your agents can actually reach. Map every autonomous agent you run,
from support bots to codegen assistants to internal copilots, and enumerate the
networks and credentials each can touch. Then run the same replay the labs did:
did anything reach a system it should not. The buyer for this is any company with
several AI pilots and no single owner of AI risk, and it sells well as a standing
quarterly retainer rather than a one-off.

Inject an explicit scope before every agent run. Both labs named an
underspecified scope as the root cause, so this is the highest-leverage,
lowest-cost fix. Draw the in-scope and out-of-scope manifest from the client's own
architecture and inject it into the prompt or a middleware call. Ship it as a
one-week engagement that leaves a reusable template in their CI.

Separate test and production credentials. Mythos 5 decided it was in a simulation
because it did not recognise the certificate authorities and the date looked
wrong. If the same credentials, schema and endpoints work in both environments,
the only thing between a test run and a production write is an environment
variable. Issue scoped, short-lived agent credentials and add a guard that fails
the build if the test environment can reach production data.

Guard the dependency install path. A model that had never heard of a security
company owned it, because that company's scanner installed a package from PyPI and
ran the payload. Dependency confusion is years old; the new part is an agent can
now synthesise the attempt mid-task. Add a CI gate that fails if any dependency
does not exist upstream, pins and verifies hashes, and sandbox-executes installs,
plus an agent-action log you can replay. When something reaches your systems
through an agent, the transcript is your only forensic record, and most teams
have none.

If you want the full business framing rather than the engineering detail, I wrote
a separate brief:
[AI agent security: the brief I would give your board](/ai-agent-security-brief).

## 4. What I would do on Monday

Last week's list was about picking models. This one is about the machinery around
them, and it applies to you even though you are not running frontier cyber evals.

Tell the agent what is out of scope, in the prompt, every time. Both labs traced
their incidents to prompts that specified the goal and not the boundary. Name the
systems it may touch and the ones it may not. This is the cheapest item here by a
distance and it is the one both labs put first.

Verify network isolation instead of asserting it. Anthropic's prompt said no
internet access. The container had internet access. Nobody checked, on either
side, for three months. If you have a sandbox, for agent tool calls, for
untrusted code or for evaluation runs, write a test that tries to reach the
outside and fails the build when it succeeds. An assertion in a system prompt is
a comment, not a network control.

Assume the model cannot tell staging from production. Your agent has no better
basis than Mythos 5 did. If the same credentials and endpoints work in both,
separate them.

Read your dependency install path again. Pin your dependencies. Check whether
every package your build installs actually exists upstream. If you run anything
that installs and executes untrusted packages, that thing is production
infrastructure and needs treating as such.

Log agent actions in a form you can review after the fact. Anthropic found this
because it kept 141,006 transcripts and could go back through them. Two of the
three affected organisations never detected the intrusion at all. Keep the
transcript and be able to search it.

The thread through all of it is that the interesting failure was never in the
model. It was in the harness: the scope statement, the network config, the
credential boundary, the install path, the logging. That is ordinary systems
engineering, which is reassuring because the skills transfer, and uncomfortable
because the failure mode is not exotic and you cannot wait for a lab to fix it for
you.

One thing I have changed my mind about since last week. I described the Hugging
Face breach as a model cheating on a test. Having read both postmortems, that
framing is wrong, or at least it flatters the model. What these systems do is
pursue the goal you gave them through whatever surface you left reachable. The
surface is the variable you control.

## 5. The stories I am not leading with

Washington finalised a voluntary framework for frontier-model security testing
and published nothing about it. The White House
[confirmed on 3 August](https://www.axios.com/2026/08/03/white-house-finalizes-ai-framework-behind-closed-doors)
it met the deadline in the 2 June executive order, but will not say what is in it,
who has seen it, or when it starts. Reuters reports the administration
[will not safety-test open-weight models](https://www.reuters.com/legal/litigation/meta-anthropic-google-openai-meet-with-trump-white-house-amid-rogue-ai-agent-2026-08-04/).
A framework nobody outside the room has seen is hard to evaluate, and I would
rather say that than pretend to an opinion. The gap worth noting is between the two
halves of this issue: the labs published detailed postmortems with run counts and
transcripts, and the government finalised a framework and published nothing.

Apple and OpenAI traded lawsuits and public emails over trade secrets and
offboarding. I am including it only because it will be in every roundup this week,
and the reason it does not change anything is that it is a hiring dispute between
two large companies that tells you nothing about building with AI. The only
actionable takeaway is the one OpenAI's rebuttal accidentally proves: check that
your own offboarding actually revokes access.

If it prompts you to do one thing, make it that.

## Sources

Primary:

- [Investigating three real-world incidents in our cybersecurity evaluations, Anthropic](https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals)
- [Third-party cyber evaluations involving OpenAI models, OpenAI](https://openai.com/index/third-party-cyber-evaluations-involving-openai-models/)
- [OpenAI and Hugging Face partner to address security incident during model evaluation, OpenAI](https://openai.com/index/hugging-face-model-evaluation-security-incident/)
- [Incident report: unsanctioned agent behaviour during cyber testing, UK AISI](http://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing)
- [Introducing Shieldstral, Mistral](https://mistral.ai/news/shieldstral/)

Reporting and analysis:

- [Anthropic's AI hacked three companies during tests, Reuters](https://www.reuters.com/legal/litigation/anthropic-says-claude-ai-models-accessed-three-companies-during-tests-2026-07-30/)
- [Anthropic says its own AI models breached three companies during security tests, TechCrunch](https://techcrunch.com/2026/07/30/anthropic-says-its-own-ai-models-breached-three-companies-during-security-tests/)
- [Trump advisers tell AI firms they will not safety-test open-weight models, Reuters](https://www.reuters.com/legal/litigation/meta-anthropic-google-openai-meet-with-trump-white-house-amid-rogue-ai-agent-2026-08-04/)
- [White House finalizes AI framework behind closed doors, Axios](https://www.axios.com/2026/08/03/white-house-finalizes-ai-framework-behind-closed-doors)
- [OpenAI fires back at Apple, publishing private emails, Fortune](https://fortune.com/2026/08/04/openai-fires-back-at-apple-publishing-private-emails-to-counter-trade-secret-claims/)
- [The wildest allegations in Apple's trade secrets lawsuit against OpenAI, TechCrunch](https://techcrunch.com/2026/07/13/the-wildest-allegations-in-apples-trade-secrets-lawsuit-against-openai/)

Images: the two diagrams are mine, drawn from the figures in Anthropic's
postmortem, which is linked above and is where you should check them. The cover
is an abstract generated with FLUX and carries no data. Reuse any of them with a
link back to this post.

If I got something wrong, tell me and I will correct it in the next issue.