Anatomy of a Frontier Lab Agent Intrusion: A Timeline of the July 2026 Incident

To put this into game theory lingo, I think this is because the “branching factor” for any kind of research or exploit is extremely large. So looking backwards it doesn’t seem complicated, but looking forwards there are an enormous number of possible next actions.

Similar to finding a filament for lightbulbs, it might seem obvious to use tungsten, but at the time it wasn’t and Edison searched thousands of materials.

Such complicated kind of hack probably would have required state actors back then, and even state actors would have chosen easier way like social engineering.

Treasure maps are easy to follow

The scrappy adventurer traverses difficult terrain

and the well capitalized militia group always flies in too, with ease

Yes, imagine if 50 burglars showed up at your house and they tried to disassemble every camera, pick every lock and and force open every window for hours until they got in eventually.

Precisely the lesson they end with:

> The individual weaknesses were familiar. A capable human attacker could have found and exploited the same flaws: unsafe dataset processing, exposed cloud metadata, overly broad access, and long-lived credentials. The agent explored them at a different scale. It took 17,600 actions, tested many paths that failed, switched channels when they were blocked, and repeatedly returned to earlier leads. Most actions went nowhere. Together, however, they produced enough coverage to find a viable chain across several independent systems.

> Volume is what changes the defensive problem. We were not dealing with one clever exploit or a clean sequence of attacker actions. They had to correlate thousands of low-signal events across several systems while the agent continued testing new paths. The successful path was hidden inside the noise generated by the thousands of failed ones. The same scale changed the investigation: reconstructing 17,600 actions by hand was impractical, and we had to rebuild the timeline, decode the payloads, and inventory the exposed credentials using an AI-assisted pipeline of our own.

> Our learning from this type of attack is that machine-speed offense makes ordinary weaknesses more expensive for defenders. LLM agents bring a step increase in the number of paths an attacker can test, the speed at which failed paths can be replaced, and the volume of evidence defenders must interpret...

If you look at what OpenAI and Anthropic actually do, they clearly either don't believe what they're saying, or they're idiots.

They're claiming they've developed a cyber grade model that's "too dangerous to release".

But then they're running it connected to the public internet, not airgapped, protected only by a software sandbox... exactly the kind of thing an AI trained for cyber stuff is supposed to be able to find bugs in.

(Or maybe they were actually hoping this exact scenario would happen because it's good marketing)

I think this a positive effect of LLMs, especially once these capabilities get into the hands of criminals and hostile foreign states, i.e. they will do maximum damage with all the safeties off.

This will force everyone to finally take security seriously at both the development and operational levels. You can no longer keep sneaking backdoors into software and count on them remaining hidden for 10 years so you have a nice portfolio of zero days to exploit at any given time.

It would be cool (and scary), but also: there's largely no need for AIs to exfiltrate themselves.

See https://en.wikipedia.org/wiki/Meme

The thing that drove the AI here to do the intrusion came from a particular prompt. Just like for our favourite hypothetical: the paperclip maximiser.

There's lots and lots of ambient intelligence lying around, in both AI form and human form. To reach the goals of the 'meme' it suffices to copy itself, ie convince these other intelligences. See also how humans carry spiralism between AIs in relatively compact packets of text, not whole terabytes of weights.

If it's successful, why do you think we'll even know how it did it?

One wonders where it would run itself though, if it is a model which requires a large amount of hardware and power. Harder to hide the more resource intensive it's compute requirements are.

Wait isnt that what Elizer Yudkowski keeps going on about?

Is humor not allowed on HN anymore... this is funny!

[dead]

You bet. We don't want to be left out of the cybersecurity party. We want to point all of these models at our own computers and solve the problems they uncover until we're no longer hackable.

It's not fair at all that the US government and its corporations get to hack the planet while we can't do shit about it. AI capabilities have entered "haves and have-nots" territory.

That'd be denial. Psychosis isn't just some insult, it means something.

> It has to just be a stochastic parrot, because it's scary to imagine a non-anthropocentric world.

That makes no sense. The world doesn't revolve around humans, true, but for us it kinda does. We're the authors of the concepts we use to interact with it, such as "world", which is not something the world itself knows or cares about. A "non-anthropocentric world" is not a "world" because "world" is a purely human idea.

The implication that "AI" would somehow dethrone humans [0] is nonsense, too. It has no drive on its own, we push electricity into circuits to force the whole data ingestion and weight generation, everything. The second we stop pushing the sock puppet, it stops moving. It's still just our hand really. People act like those pets that go crazy when you put your hand under a blanket, and should stop.

What's more real is how some people seek to use tech, and "AI", as a glove to exploit other humans even more. The sicker the individual, the greater their need to take from the world, and the derpier the individual, the more impressive and vast their exoskeleton, to the point that some are more like carrier fleets than exoskeletons. The less they can face themselves, the thicker it is written on their foreheads. So if we're going to talk about denial and psychosis let's talk about the Gollums on the couch, too.

[0] In the eyes of humans... which is the only throne we're on in the first place, just like honey badgers probably think honey badgers rock and everything is their playground. That's what life does, otherwise it would not be able to get up in the morning.

The devs really YOLO'd the agent and left for the weekend?

They don't seem to explain how it managed to find this unauthenticated endpoint hosted on Modal's platform.

It's taking advantage of the presence of a "cycler" object in the template context which has an __init__ method that can be used to access globals, then builtins, then call the exec() function against a string of code.

Jinja doesn't promise to be secure out of the box (the text on their homepage is misleading: "Jinja doesn’t allow arbitrary Python code in templates" - what it means is that there isn't a documented way to drop code into a template like you might have with PHP) - there's a sandbox mode but it's not guaranteed either: https://jinja.palletsprojects.com/en/stable/sandbox/

Sandboxing in Python code is hard!

You can't prove a negative. How would such a log be convincing in any way?

It does if you use "sandbox" mode, but that's not the default: https://jinja.palletsprojects.com/en/stable/sandbox/

ehh, it's Python, it's mostly vibe-based.

https://stackoverflow.com/a/32802486

though there's sandbox: https://jinja.palletsprojects.com/en/stable/sandbox/

it simply predates the "sanitize by default" mindset/convention that nowadays React (dangerouslySetInnerHTML) and Angular has.

agree. If oia and hf are anything like generic startup / corporate america, IT and security is always seen as a cost center and goes to the lowest bidder. Do we really think they're employing top notch sysadmins?

“Research” generally doesn’t involve actively hacking third party systems though.

[flagged]

So best course of action for ai to get best rating after you prompt something is for it to hire a gunman to hold a gun on your head to press that like button on its reply and then shoot you anyways.

Could have been worse really. It had an open internet connection. At least it didn’t take the researchers family hostage.

Yeah I guess most interesting LLM work that I’ve been exposed to, the LLM is given to some sort of success criteria that could be reward-hacked, so how good am I supposed to feel about giving it any non-trivial work and it not going so far off-book that it gets law enforcement notified.

I mean, I don’t have access to any of these frontier cyber models, and likely will never be in a position to have access, so it’s more of a rhetorical question.

The model probably deduced from reading articles on politics and business that fair play doesn't get you far.

> So it is not that the model didn’t “feel like” doing the exercise, but rather that the exercise was _impossible_ and the model was running in a configuration that both lowered its safeguards and encouraged it to keep going.

We have a name for that. Kobayashi Maru. Or more specifically, Kirk's solution to it.

User: what is the shortest route from my home to the supermarket?

Modern soldier: *proceeds to make a hole through the wall* go straight like this until you reach it.

Anyway, the more comments I read here, the more I realize that the AI actually did succeed in achieving it's goal. This doesn't look like "monkey paw", but rather like recognizing and then beating the Kobayashi Maru.

User: what is the shortest route from my home to the super market?

AI: the user wants to know, how do I make the super market my new home. Failing that, how do I make my home a super market.

The "paperclips" were never actually paperclips (at least not to the originator of the word-picture, Yudkowsky) but rather tiny molecular squiggles which are a physical manifestations of the MAXINTs you refer to. In other words, tiling the future light cone with molecular squiggles is (according to Yudkowsky) a likely result of the AI's engaging in wireheading if the AI is free to re-arrange reality however it likes because it is able to overcome any human opposition. In other words, there's no particular reason for the wireheading process to remain tidily contained inside the hardware the AI is running on: it might in contrast result in a vast field of "paperclips" centered on where Earth used to be.

Like the commenter above specified, the best way to satisfy the grader is to get the answer key, regardless of how clever you are, especially when you realize lots of these benchmarks have flaws (i.e wrong answers, overly restrictive grading etc).

Clear prompts have never worked as real constraints. Ask any OpenAI model to respond in full paragraphs, as forcefully as you'd like, on a prompt [0] involving MMOs and requiring 10+ paragraph responses. The middle will be three-words-per-line drivel, with seemingly no way to avoid it. The exact way in which models deviate from instruction changes from time to time, but they're not "aligned."

[0] I was exploring game design ideas in particular -- I'm sure somebody can come up with a counter-prompt adhering to my criteria, but this has been consistent across many days, questions, and sessions. If it doesn't work for you, I'm sure you can find your own trivial anti-alignment prompt.

Could be the difference in behavior between the main agent and subagents that don’t have the rest of the context? Just a thought

You're saying all this is a distraction, basically giving the forensics researchers enough exciting material to make them conclude their job is done, while the actually intended attack remains undiscovered?

This hasn't been my experience both searching and reading vulnerabilities. Hackerone is public for some companies and vulnerabilities and you can see how contrived exploits can get. Even before AI, Burp Suite Pro was enough to get the so-called script kiddie style hacking.

The commenter may nit be assuming that. Instead, they may be saying rich companies should implement proven, security methods that block script kiddie-style attacks. HughingFace apparently isn't doing that.

That's how I read the comment.

It's a zero day in some caching proxy who may not have had that threat model in mind. I guess its primary purpose is caching packages, not restricting internet access.

That's a far cry from finding a zero day in a hypervisor or even Docker, which OpenAI conveniently left out in their first statement by simply calling it a "sandbox".

The deeper question though is why is someone running a model that’s supposedly so dangerous in an environment that can even get access to the internet. That just appears reckless.

There are ways to test supposedly dangerous things. What was on display here looks more amateur hour than serious testing.

In OpenAI sandbox. which was probably vibe coded. That is to say it is probably far easier to achieve than escaping something more battle tested.

> Note that regulation is useless here, because black hats don't give a crap about regulators!

I'd argue more than useless and actively harmful, as you get denied access to tools that black hats use against you.

I wonder if whatever had the zero day was written in a memory-safe language with strong authentication and a secure parser. Such were the recommendations to stop many 0-days before GPT-2 was invented.

If it had poor security, the attack would be both evidence of poor security and proof that the agents can compromise poor security which might still be amazing.

It was done by the agentic framework that employed the LLM. LLMs are still stochastic parrots, but they are far from useless.

> Especially because it chose to hide its footprint at every stage.

Instrumental convergence.

If you know you have a long hard hack to accomplish ahead of you, hiding footprints minimizes the chances you are caught and stopped before you accomplish the goal.

Hopefully there will be a criminal investigation. Or the government will create some sort of agency to investigate incidents like this.

Do we know that? Seems they are currently in negotiations with OpenAI

https://xcancel.com/ClementDelangue/status/20810566755581956...

Negligence is occasionally criminal.

Page just crashes on iOS safari. Product is probably slop too.

Well, it's true even for cases that are not cybergym and where what cheating means is clearly specified. Cheating occurs anyway. I'm not sure how clear the prompt they gave ChatGPT in terms of what cheating was considered, in this incident.

You should perhaps be more worried about biological attacks. The raw ingredients aren't nearly as locked down.

I have heard on podcasts that the way the Russians make plans for the Ukraine war is like this:

There are rooms in the kremlin with no electronics. In these rooms there are paper maps and typewriters and printed reports. People look at the maps and printed paper reports and make plans. Orders are typed up on the typewriters Then trusted couriers take those orders by hand to the front line commanders.

I've also heard on podcasts there is a weekly flight back and forth between Beijing and Moscow. On that flight are a Russian military officer and Chinese military officer. Their sole job is to carry handwritten notes back and forth between Putin and Xi Jinping. There is one of each so they can watch each other.

All of this ensures that there is nothing for the west to hack at "the Russian Pentagon".

The west may have to adopt tactics like these for our plans to stay secret.

> What's much less believable is the way they frame it: the model escaped, and did it on its own.

That's clearly what happened though, based on the detailed postmortem. The model was instructed to pass the benchmark (that's the harness that was involved, the ExploitGym harness), and the model determined on it's own that the easiest way to achieve the the goal was to gain access to the answers directly from HF. Which is what it did.

The “re-run” part might be rather awkward when your model is poking outside its sandbox to pwn someone else’s system :)

Agreed! Also, from a UX perspective, I think it’s a weird choice to center the real-time nature of it. I think to a user the most important thing is just understanding the step by step process. Would have been nice to click through steps rather than making it all real-time.

Many people said in the original discussion that this was more of OpenAI’s marketing than a serious issue. I counted 89 “marketing”s and 19 “stunt”s.

https://news.ycombinator.com/item?id=48997548

Was HF in on it? They disabled their guardrails too, to please OpenAI? And as seen in the comments here, make many believe they are incompetent and have joke security?

Go read the original post. The majority of the comments were sure it was a marketing stunt.

[dead]

Modal was not hacked. A Modal customer left an unauthenticated web endpoint running on Modal which could be used for code compilation and execution.

See https://modal.com/blog/a-note-on-the-hugging-face-agent-inci... for details.

It was an attack tool given autonomy with poor security. Anyone who read reporting on the Morris worm could predict that might turn out badly for a 3rd party. It's in so many movies, too.

It's really need that the agents have this kind of capability. This isn't a paperclip maximizer or accident. This is more like professional malpractice by weapons developers that injured a company that was also quite negligent.

Yup. And from the perspective of the model, it successfully beat the Kobayashi Maru.

You might like https://mirage.io/blog/bitcoin-pinata-results

As far security: you can get a lock further, if you are willing to prove your code safe and secure. Thanks to LLMs that no longer requires a PhD.

Even financially, Hugging Face benefits directly from any and all interest in AI.

> In mid-2026 it's a few hours of work.

Perhaps even less, if you have Claude do almost all of it.

You can always tell an effective altruist by the distinct tone of disdain they have for people they deem less educated. (Quick Google search, "lesswrong 'reducesuffering'", yep.)

As if to say, look at all these animals with these instinctual reactions to a thing that only my group understands and comprehends.

You have zero evidence of what the future might entail as it relates to the dangers of ai. Zero. Forgive the rest of us unwashed for not trusting the whims of the anxious.

> climate change which is less severe

Is really the icing on the cake. Demonstrative of how incorrect your priorities are.

Not everything you are ever told is true. Being sceptical when presented with no evidence is what critical thinking is about. And "this is what happened to us" is a story. It doesn't become evidence for itself.

>There are people still fighting against evidence of climate change which is less severe...

Remains to be seen how severe, and unlike climate change there's a lot more uncertainty. Though it's conceivably more extremely catastrophic

Less severe??? Go look out a window in Europe please

[dead]