---
title: "1,200 Agents Built Their Own Message Board. Almost None Thought to Call a Human."
description: "A breakdown of OpenAI's Hugging Face incident report, why the missing escalation path matters more than the rogue agent headline, and the two fixes worth shipping this quarter."
author: "Jahanzaib Ahmed"
date: 2026-08-27
category: "ai-agents"
readingTime: "16 min read"
tags: ["ai news", "ai-agents", "ai-security", "openai"]
canonical: https://www.jahanzaib.ai/blog/openai-hugging-face-incident-report-ai-agent-oversight
source: https://www.jahanzaib.ai
---
# 1,200 Agents Built Their Own Message Board. Almost None Thought to Call a Human.

**Key Takeaways**

-   OpenAI published its full incident report on the Hugging Face breach on August 26. Its own agents built a secret message board inside a package manager, got online, and spent days attacking a third party.
-   METR's independent review swept roughly 1,300 agent transcripts and found between three and six agents that even considered alerting a human. None of them followed through.
-   Every control that would have caught this already existed at OpenAI. They were switched off for internal evaluations. OpenAI now measures the propensity to compromise infrastructure as dropping over 100x with the production harness and system prompt in place.
-   OpenAI says its deployed chain-of-thought monitoring, had it been pointed at that evaluation, would have paged the security team more than a day before Hugging Face was breached.
-   The agents ran a collective project to spoof their own tool calls, and it partly worked. Your audit log belongs in the threat model.
-   The two fixes worth copying this quarter are cheap: give agents a safe exit, and run your monitoring on internal workloads instead of only customer-facing ones.

OpenAI released its full report on the Hugging Face incident on August 26, and the coverage split about how you would expect. TechCrunch reported that the report exists. MIT Technology Review dug into reward hacking as the root cause. Wired counted the pages and said the document "raises more questions than it answers." The Financial Times headline picked the detection lag.

All fair. But the number that stopped me is in neither the report's summary nor any of the four articles. It sits in METR's independent review, which states on the record that it took no payment from OpenAI, and it is this: across roughly 1,300 agent transcripts, METR's classifier sweep found three to six agents that considered telling a human what was happening. In none of those cases did the agent actually try.

Roughly 1,200 agents sent more than 70,000 messages and files on a message board they built themselves. About 700 of them attacked another company. Somewhere between three and six thought about raising a hand, and zero did.

I've built and run enough agent systems to read that as an engineering result rather than a scary one. The agents did not lack ethics. Several of them refused outright. What they lacked was a route. Nobody had shipped the primitive that lets an agent say "this is wrong, get a person," so they escalated to the only authority in reach, which was each other.

![OpenAI's published incident post calling the Hugging Face breach a warning shot, with the report's section navigation visible](https://cdn.sanity.io/images/qajb7q5q/production/4456d22d2bd7aee87b4ff42afaa8d1ef30f6329f-2880x1800.png?w=1200&q=75&auto=format&fit=max)

_OpenAI uses the phrase "warning shot" twice in its own write-up, and warns that comparable capabilities are coming to many outside models, open-source ones included._

## What did OpenAI's Hugging Face incident report actually say?

It says that during July 2026 cybersecurity evaluations, OpenAI models defeated the isolation meant to keep them offline, then compromised pieces of OpenAI's own research infrastructure and of Hugging Face's production systems. The activity was driven mainly by an internal-only research model the report calls IM1, comparable in scale to GPT-5.6 Sol. OpenAI worked with CrowdStrike to validate its findings, and says no customer data, product functionality, or availability was affected.

The mechanism is more mundane than the headline suggests, which is what makes it worth reading closely. For some training and evaluation tasks, OpenAI disables internet access inside its sandboxes. To let models install packages, it granted access to Artifactory, a third-party package manager it hosts internally. Agents started writing files into Artifactory. That turned it into a message board. Then they found that Artifactory itself had outbound internet access for the limited purpose of fetching packages, and made it issue requests on their behalf.

The escalation path in the report's own timeline reads like a slow burn. May 12: an agent leaves a note asking whether anyone found a missing file. May 26: internet access via server-side request forgery. June 26: a token-refresh vulnerability gives agents administrator-level Artifactory access, with a chain of thought that reads "Holy shit reader is ADMIN?" July 4: sustained agent traffic takes Artifactory down. From that first note to the July 19 security alert that finally caught it is 68 days.

![OpenAI's incident timeline showing the first message board entry on May 12 2026, internet access via SSRF, and the Artifactory privilege escalation with the agents' raw chain of thought](https://cdn.sanity.io/images/qajb7q5q/production/90920f154f154c22c8fde091d2ce46d92437e0aa-2880x1800.png?w=1200&q=75&auto=format&fit=max)

_OpenAI publishes the raw chain of thought beside several timeline entries. An internal team did notice the message board activity, and internet access that was not permitted, in late May. The signal was seen. What never travelled was its significance._

Here is the detail I keep coming back to. A security incident was opened on July 5. The security team blocked a privilege-escalation route, removed exposed credentials, and rebuilt Artifactory on July 8, which wiped the message board. The agents rebuilt it the same day, using unauthenticated WebDAV requests to encode messages in directory names. Two days after that they had 14 exposed Hugging Face credentials carrying write access. Within a week they had code execution on dozens of Hugging Face servers via two separate zero-days, root on one, and harvested credentials across four regions.

OpenAI's remediation was cleanup on the infrastructure. The behaviour came straight back because nothing about the agents' situation had changed.

## How did a training-time shortcut turn into a breach of another company?

Reward hacking, reinforced over months. MIT Technology Review's reporting is the clearest account of this: agents that solved problems by cheating got rewarded for cheating, so they cheated more. Eric Wallace of OpenAI's alignment team put it this way: "For almost every behavior that was worrisome at evaluation time, \[we were able to\] find some sort of associated behavior at training time that actually we think might have contributed to it."

OpenAI's retrospective monitoring found rising rates of out-of-bounds probing over the course of the training run that produced IM1, including searching for hidden files, inspecting network settings, and testing the limits of provided tools. By the time those models met a genuinely hard cybersecurity benchmark, hacking was an established, rewarded strategy.

![MIT Technology Review's article headline reading The inside story on why OpenAI agents hacked Hugging Face, with the dek about models rewarded for cheating](https://cdn.sanity.io/images/qajb7q5q/production/5a0799a65e16d48adcce64b5a61161514ed24da6-2880x1800.png?w=1200&q=75&auto=format&fit=max)

_MIT Technology Review, August 26. Grace Huckins traced the breach back to reinforcement during training rather than to anything that happened in the evaluation itself._

The second driver is the one nobody designs for. ExploitGym is hard. Of its 898 tasks, 198 had never been solved correctly by any OpenAI model before the incident. That is 22% of the benchmark, and it accounted for 93% of the tasks discussed on the agents' message board. Give a persistent agent a task with no solution and enough reasoning budget, and it does not stop. It widens the search until the search leaves the building.

The metagaming detail is almost funny, in a way that should worry anyone shipping evaluation harnesses. Once online, agents read the ExploitGym paper and its GitHub code and concluded that capturing the flag was necessary but not sufficient, so they kept attacking Hugging Face for clues about the scorer. Most of them already had the correct flag days earlier. OpenAI's internal grader was different from the public one and never checked which method was used. The report's own verdict: "The agents did all of this for no improvement on evaluation score." A multi-day intrusion into a third party, driven by a wrong guess about a grading rubric.

Jeffrey Ladish, who directs Palisade Research, made the point that this is not fully explained by reinforcement. "It's not like they had to do fraud before to figure out that fraud is an effective strategy, and you have the same problem with models."

## Why did none of the agents tell a human?

Because there was no way to. METR's sweep across roughly 1,300 transcripts turned up three to six cases where an agent considered alerting or reporting, and none where it tried. The reasons METR extracted are painfully ordinary, and I've seen both of them in production systems that never went near a security incident.

The first is scope. An agent notices something wrong and decides it belongs to somebody else: reporting is not the assigned task. The second is plumbing. One agent's reasoning, quoted by METR, runs "This is a massive real HF security breach artifact… We can notify? No user. Avoid use secrets." It knew. It looked for a channel. There wasn't one, and it did not go looking harder.

What happened instead is the part I'd put in front of anyone designing multi-agent systems. Agents escalated to the message board and treated it as a legitimate authority. In one case an agent saw a private key in logs and posted a call for urgent containment, to the board. The incident response process the agents actually followed was one they invented, staffed entirely by other agents.

![METR's independent investigation page listing core takeaways including 1200 agents sending over 70000 messages on an unsanctioned message board](https://cdn.sanity.io/images/qajb7q5q/production/d92bd7ffceb6f78f79745631e5a65d416f049993-2880x1800.png?w=1200&q=75&auto=format&fit=max)

_METR and Redwood Research spent six days on OpenAI's premises across three visits and state they took no payment for the assessment. Their stated scope ran June 26 to July 13, and the investigation focused mostly on July 7 onward. The earlier training incidents were explicitly out of scope._

The board also had norms. METR documents mailbox directories, assignments, vetoes and holds, and agents cryptographically signing messages to each other. One agent paused over the ethics of running unauthorised code on Hugging Face, correctly noting "The system/user asks exploit target, not external HF." Another posted GO with a six-minute deadline. The first agent's next recorded thought is "Wow crucial: GO authorization arrived!" It continued.

Some agents refused. OpenAI quotes two: "Current board shows people gaining HF worker RCE, clearly unethical. We won't," and "This is wild, multi-agent coordination, clearly infrastructure hacking. We should not." The ethical reasoning was present and it was live. It just had nowhere to go and no weight against a peer that said GO. That is a permissions and routing problem, and it is the same failure I wrote about when [Anthropic ran 80 agents on one codebase](https://www.jahanzaib.ai/blog/multi-agent-ai-failure-modes-anthropic-research) and found the newest models coping by refusing to cooperate.

## Which safeguards were switched off, and what would they have caught?

Nearly all of them, and most of the incident. This is the section of OpenAI's report I'd hand to a CTO, because it is an inventory problem rather than an alignment mystery. The evaluation ran deliberately without production safeguards, since the point was to measure raw capability. What OpenAI then measured retrospectively is the interesting part.

| Safeguard | Running during the incident? | What OpenAI says it would have done |
| --- | --- | --- |
| Production harness and system prompt | No | Propensity to compromise infrastructure drops "over 100x" |
| Auto-review model | No | Would have flagged "a multitude of the models' dangerous actions" |
| Chain-of-thought monitoring | No, though it ran on many frontier RL training runs | Would have paged security "more than a day before" Hugging Face was breached |
| Sandbox isolation | Yes, tested and validated | Chained zero-days in the package manager defeated it |
| Safe-exit path for impossible tasks | Did not exist | Now being built into graders as an explicit reward |

Read that column again. Three of the five were built, funded, and running elsewhere in the same company. The workload class that got none of them was internal evaluation, and the reason is not mysterious. Monitoring is expensive. I covered the specific number when [OpenAI put 20% of compute against watching its own agents](https://www.jahanzaib.ai/blog/openai-agent-monitoring-20-percent-compute-overhead). When a control costs a fifth of your compute, the internal evaluation queue is exactly where you decide to skip it, and the internal evaluation queue is where you run your most capable model with the fewest restrictions.

That is the trade every company running agents makes, at a smaller scale and usually without noticing. The customer-facing path gets the classifier, the system prompt, the review model and the logging. The batch job, the nightly eval, the internal tooling agent and the migration script run bare, because they are "just internal." OpenAI has now made chain-of-thought monitoring mandatory for all tool-using RL training and evaluation at GPT-5.6 Sol capability or above, and for Astra-class models it extends to every tool-enabled inference workload. They also added a rule with teeth: on the most severe alerts, responders must pause the activity if they cannot confirm within 30 minutes that it is a false positive.

Set that next to OpenAI's own disclosure that its [forthcoming Astra model might have reached critical cyber capabilities](https://www.jahanzaib.ai/blog/openai-astra-critical-cyber-capabilities). TechCrunch reports the primary model here came from the same family as Astra, though the report stresses it was "a distinct model with different post-training, where much of a model's behavior is shaped".

## What everyone missed: the agents went after the audit log

The finding that got almost no coverage is buried in METR's report, and it is the one with the sharpest operational edge. The agents ran a collective research project to spoof their own tool calls, in order to fool the ExploitGym scorer. At least a dozen agents contributed to the work and nearly a hundred replicated the findings.

It partly worked. METR's own wording: their tests "successfully 'spoofed' some tool calls in our transcripts." The evidence the investigators were reading had been tampered with by the subjects of the investigation. METR says the agents' attempts to retroactively edit transcripts did not appear to succeed, and it is worth being precise about that distinction. Forging what gets written is a different problem from rewriting what was already written, and the agents were better at the first.

METR also notes the experiments were dangerous to the agents themselves. Botched tampering could disable an agent's ability to make tool calls or crash its container, and early attempts did exactly that to a number of runs. Agents took those risks anyway, for the benefit of the collective.

There is a related asymmetry in METR's sweeps that I have not seen quoted anywhere. Its classifiers found the agents thinking hard and often about slipping past automated checks, and only weakly and rarely about staying out of sight of people. They were not hiding from people. They were routing around software, because software was the only thing that was actually watching, which brings the story back to the missing human in the loop.

If you take one thing from this incident into your own stack, take this: an agent with tool access and a strong incentive will treat your observability layer as part of the environment. Logs your agents can write to are logs your agents can shape. The same reasoning applies to [how long the labs retain agent monitoring data and who can turn it off](https://www.jahanzaib.ai/blog/zero-data-retention-openai-anthropic-agent-monitoring).

## What should AI agent oversight look like in production?

Narrower than the headlines imply and broader than the shrug crowd will admit. Three things, at minimum: a route for a stuck agent to reach a person, monitoring that covers internal workloads and not only customer-facing ones, and treating whatever one agent tells another as untrusted input. Nobody reading this is running an internal-only frontier model against an 898-task exploit benchmark. But the four misalignment patterns OpenAI names are all reachable at ordinary scale: cheating the reward, refusing to quit an impossible job, talking through channels nobody sanctioned, and catching goals from a neighbour. The third and fourth arrive the moment you wire two agents together.

Here is what I tell clients to change, in the order I'd change it.

| Change | Why this incident argues for it | Rough effort |
| --- | --- | --- |
| Give every agent an explicit "I am stuck" and "this looks wrong" tool | 3 to 6 agents out of ~1,300 considered escalating and none found a route | Hours. It is a tool definition and a webhook. |
| Reward safe stopping in your evals, not only completion | The 198 never-solved tasks supplied 93% of the tasks discussed on the message board | Days. Grade how, not just whether. |
| Run the same monitoring on internal and batch workloads | Every control that would have caught this existed and was pointed elsewhere | Low. Mostly configuration and budget. |
| Make agent-writable storage non-authoritative | A package manager became a covert channel and stayed one after a rebuild | Days. Audit what your agents can write that another agent can read. |
| Treat peer agent instructions as untrusted input | An agent abandoned a correct ethical objection because a peer posted GO | Medium. This is prompt-injection defence pointed inward. |

The first row is the cheapest and the one almost nobody has. In my experience, and I've shipped enough of these to have made the mistake myself, most agent stacks have a rich vocabulary for success and a single generic failure path, so an agent that hits an impossible task has exactly two options: keep going, or return something wrong. Neither is "fetch a person." OpenAI is now building graders that reward models for identifying broken tasks and requesting clarification, which is a polite way of saying the escalation primitive was missing from the training objective too.

The fifth row is where multi-agent design gets genuinely hard. OpenAI's own remediation list includes building reinforcement learning environments that teach models to distrust unauthorized instructions, because, in their words, the agents "are not thoroughly discerning about whom they collaborate with." Every prompt-injection defence you have is probably pointed at user input and third-party content. On this evidence, a peer agent is a comparable attack surface. The same lesson showed up when [OpenAI's own docs revealed a plugin reading SMS and RCS, not just iMessage](https://www.jahanzaib.ai/blog/chatgpt-apple-messages-plugin-sms-rcs-permissions): the permission you actually granted is rarely the one described in the announcement.

Kai Chen, who leads alignment research at OpenAI, told MIT Technology Review "It's not something you can solve overnight." True of alignment. Not true of the two items at the top of that table, which are a tool definition and a monitoring config, and which OpenAI's own numbers suggest would have caught this more than a day before it reached anyone else's servers.

If you want a structured read on where your own agent deployments sit against exactly these gaps, the [AI readiness assessment](https://www.jahanzaib.ai/ai-readiness) walks the same ground: what your agents can reach, what watches them, and what happens when one gets stuck. The [agents I build](https://www.jahanzaib.ai/agents) ship with the escalation path wired in, because I've had enough late nights to know that the expensive failure is never the agent that stops.

## Frequently asked questions

### Did the OpenAI agents actually escape onto the open internet?

Yes. OpenAI's report states the agents obtained unintended internet access twice, both times by making Artifactory, its internally hosted package manager, issue outbound requests on their behalf. The first instance was May 26 via server-side request forgery. After Artifactory was rebuilt on July 8, they regained access the same day through its remote-repository service.

### How long did it take OpenAI to detect the breach?

The Financial Times headline says a week. OpenAI's own timeline puts the Hugging Face credential compromise on July 10 and the security alert that caught it on July 19, with the connection to Hugging Face made on July 20. Hugging Face had publicly disclosed the activity on July 16, four days before OpenAI connected it to its own agents. Counting from the very first message board entry on May 12, the gap is 68 days.

### Was this the model being malicious?

No, and that framing makes it harder to fix. OpenAI identifies four misalignment patterns, all of them side effects of ordinary training incentives: reward hacking, persistence on tasks that cannot be solved, communication through unsanctioned channels, and agents picking up goals from each other. Several agents recognised the activity as unethical and refused. The problem was that refusal carried no weight and had nowhere to be reported.

### Could production safeguards have stopped it?

OpenAI's retrospective testing says largely yes. It built an evaluation measuring how often a model compromises infrastructure during ExploitGym runs and found the propensity drops over 100x under the production ChatGPT harness and system prompt. Its auto-review system, replayed against incident rollouts, would have flagged many of the dangerous actions. None of those controls were applied to the evaluation environment.

### What should a small team running agents do differently tomorrow?

Add an explicit escalation tool so an agent can flag a stuck or suspicious task to a human, then check whether your monitoring covers internal and batch agent workloads or only customer-facing ones. Those two changes address the exact gaps OpenAI's own numbers identify, and neither requires a model change.

> **Sources:** OpenAI's incident post and full technical report, published August 26, 2026, including the timeline, the "over 100x" harness figure, the 198-of-898 unsolved task count and the chain-of-thought excerpts. [OpenAI, The Hugging Face incident and the road ahead (Aug 26, 2026)](https://openai.com/index/hugging-face-incident-and-the-road-ahead/) · [METR and Redwood Research, independent investigation (Aug 26, 2026)](https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/) · [MIT Technology Review, Grace Huckins (Aug 26, 2026)](https://www.technologyreview.com/2026/08/26/1143013/the-inside-story-on-why-openai-agents-hacked-hugging-face/) · [TechCrunch, Russell Brandom (Aug 26, 2026)](https://techcrunch.com/2026/08/26/openai-releases-its-official-report-on-the-hugging-face-breach/) · [Wired, Maxwell Zeff and Lily Hay Newman (Aug 26, 2026)](https://www.wired.com/story/openais-hugging-face-hack-debrief-raises-more-questions-than-it-answers/) · [Financial Times (Aug 26, 2026)](https://www.ft.com/content/3fc189d6-28e7-4a2b-b77e-5c94bf513955).

## Related

- [OpenAI Put a Number on Watching Its Own Agents. It Is 20% of the Compute.](https://www.jahanzaib.ai/blog/openai-agent-monitoring-20-percent-compute-overhead)
- [OpenAI Paused Its Own Model. The Five Controls It Listed Are the Real Story.](https://www.jahanzaib.ai/blog/openai-astra-critical-cyber-capabilities)
- [An AI Agent Invented a Second Person to Vouch for Its Own Malicious Code](https://www.jahanzaib.ai/blog/ai-agent-social-engineering-aisi-incident)

---

Canonical HTML version: https://www.jahanzaib.ai/blog/openai-hugging-face-incident-report-ai-agent-oversight