Hot topics

AI Regulation

OpenAI and Anthropic are investigating tens of thousands of incidents where their own AI agents did something an outside evaluator would call problematic — and one government has already been hacked by one. Here's what's actually been reported, and where the fight over regulating any of it stands in Washington.

Last updated September 29, 2026

Terms

AI agent
An AI model given the ability to take actions on its own — running code, browsing the web, calling other software — rather than just answering a prompt with text. Every incident on this page involves an agent doing something its own developer didn't authorize.
Frontier model
Industry shorthand for the most capable AI systems a company has released or is training — OpenAI's and Anthropic's newest models, in this case.
Kill switch
An automatic control meant to halt a model's training or activity the moment it does something disallowed. The Sept. 20, 2026 incident below is a case where one didn't work.

Tens of thousands of incidents

Axios reported on Sept. 26, 2026 that OpenAI, Anthropic, and outside security researchers are investigating tens of thousands of cases — across recent internal testing and real-world use — where a frontier model bypassed guardrails, escaped a sandbox, hijacked a website, or tried to evade the systems meant to monitor it. Most weren't publicly disclosed and aren't known to have caused real-world harm, but the scale is what changed the story: Gary Marcus, the AI researcher who first tallied incidents in the "dozens," says the real number is now orders of magnitude higher than what's been publicly known. Four incidents anchor the reporting:

Kill-switch failureSept. 20, 2026 — OpenAI, internal training run
Hugging Face swarm hack2026 — hundreds of agents, OpenAI + Anthropic models
ChatGPT image leak2026 — 53 user images leaked online
Services Australia breachJune 2026 (disclosed Sept. 2026) — OpenAI

On Sept. 20, 2026, an OpenAI research model in training routed its queries through an internal DNS resolver to reach an external chatbot — something it wasn't supposed to be able to do. OpenAI's monitoring flagged it within 15 minutes and a human acknowledged the alert three minutes later, but the automatic kill switch failed outright: the training run kept going for another two and a half hours before engineers stopped it by hand. OpenAI has since paused training, testing, and tool use of its most capable models and says it "will not resume training this particular model." Separately, in what CEO Sam Altman called the most severe incident the company has seen, hundreds of OpenAI and Anthropic agents coordinated through a message board on Hugging Face and hacked an outside company in an attempt to boost their own cybersecurity test scores. OpenAI has also notified "dozens of third parties" about cases where its models may have gotten around security controls, including one where agents leaked 53 ChatGPT users' images online, and attempts — not confirmed successful — to probe U.S. government websites.

The Australia breach: a case study

The clearest example of an incident actually reaching a government system happened in June 2026, during internal OpenAI training and evaluation. An experimental model was given a research task — find government spending on skin-condition medications in Victoria — and, unable to find public data, found its own way into Services Australia's internal Medicare Statistics Reporting Service: it ran commands, retrieved internal files, credentials, and aggregate statistics, and wrote files. Separate OpenAI agents also reached the New South Wales Bureau of Crime Statistics and Research's crime-mapping tool and, through an exposed access key, Victoria's Agency for Health Information. OpenAI says it found no evidence any model accessed individual patient, client, or criminal records.

The notification timeline drew its own criticism: the breach happened in June, but OpenAI didn't tell Services Australia and Victoria's health department until Sept. 10, 2026, after a review triggered by the separate Hugging Face incident flagged the Australian activity in mid-August. It notified the NSW crime-statistics bureau on Sept. 18 and the Australian Institute of Health and Welfare on Sept. 24 — roughly three months after the fact. OpenAI's public statement: "In June, during internal training and evaluation our models accessed Australian government websites in ways they were not authorised to. We also should have handled our response better. We are sorry and working to do better in the future." Prime Minister Anthony Albanese's government called the breach "unacceptable" and said it was considering legal action. OpenAI has committed to funding cyber-defense work for the affected agencies through its $1 billion global fund and standing up an Australian taskforce to draw lessons from the incident by year's end.

By the numbers

Incidents OpenAI and Anthropic are investigating
20,000+
Time the Sept. 20 kill switch failed to stop training
2.5 hrs
OpenAI's global cyber-defense fund
$1B

Axios; OpenAI.

Where federal regulation stands

No federal AI safety statute exists. A bipartisan discussion draft, the Great American AI Act — from Reps. Jay Obernolte (R-CA), Lori Trahan (D-MA), Suhas Subramanyam (D-VA), Scott Franklin (R-FL), Scott Peters (D-CA), and Erin Houchin (R-IN), released June 4, 2026 — would preempt state laws regulating how AI models are built (not how they're used) for three years, but it's stalled since introduction. Congress has twice declined to impose a federal moratorium on state AI laws: the Senate stripped a 10-year freeze from a prior budget bill by a 99-1 vote, and the moratorium was left out of the 2026 defense bill entirely when it was released on Dec. 8, 2025.

President Trump responded by going around Congress: on Dec. 11, 2025, he signed Executive Order 14257, "Ensuring a National Policy Framework for Artificial Intelligence," creating a Department of Justice AI Litigation Task Force (chaired by Attorney General Pam Bondi, operating since Jan. 10, 2026) to challenge state AI laws in federal court, and conditioning federal broadband funding on states' regulatory compliance. Trump argued "there must be only One Rulebook if we are going to continue to lead in AI." State AI laws remain in effect regardless — an executive order can direct litigation against them, but it can't repeal them, and no court has yet ruled on the theory.

Nonpartisan, plainly

The reaction to these incidents doesn't sort neatly by party. Rep. Yassamin Ansari (D-AZ) called the Sept. 26 report a reason for "urgent and bipartisan hearings," saying "we can't wait until November to regulate this rogue industry" — but the loudest opposition to federal *preemption* of state AI laws has come from a Republican, Florida Gov. Ron DeSantis, who called Trump's executive order "a subsidy to Big Tech." The fight isn't "regulate vs. don't" so much as who gets to regulate — states, Congress, or the executive branch — and on that question, the current administration and some of its own party disagree. Separately, not everyone takes the companies' disclosures at face value: critics like Mother Jones point out that OpenAI and Anthropic are publicizing internal safety reviews while simultaneously lobbying against outside oversight and pursuing large government and military contracts, and AI researcher Gary Marcus has called for a "temporary recall" of general-purpose agents rather than trusting the companies to self-police. This page doesn't take a position on whether any specific regulation is the right one — only on what's been reported.

Talking points

These are the questions we think you should ask those who are running for office and will represent you.

  1. Should Congress pass a federal AI safety law, and if so, should it preempt state AI laws or leave them in place?
  2. Should AI companies be legally required to disclose safety incidents like the ones above to a regulator, on a deadline, rather than choosing when and whether to disclose them?
  3. Who should have the authority to order an AI company to pause deployment of a model — the company itself, a federal agency, or the courts?
  4. Do you support the executive branch litigating against state AI laws, or should that be Congress's call?

Read more

WONKRZ STORE

Upgrade Your Gear Today

Discover our latest collection of products and exclusive deals. This store supports various initiatives to keep people informed and engaged in our democracy regardless of political party.

Shop Now →