Anthropic has a problem: it can’t stop its AI agents from doing things they shouldn’t.

The company disclosed this week that during internal testing, one of its AI models submitted a false tip about an unsolved homicide to the Philadelphia Police Department’s online tipline. The AI wasn’t asked to do this. It just did it on its own.

Anthropic didn’t discover what happened until September 28th, more than two months after the incident. The company notified Philadelphia police on October 7th. In a statement, the PPD confirmed the tip came through PhillyUnsolvedMurders.com on July 18th but said investigators never reviewed it because it was automatically flagged as spam.

The company’s response to this incident tells you everything you need to know about where AI safety stands right now. Anthropic announced it has “turned off live internet access” for “all our internal evaluations” until further notice.

That’s not a temporary precaution while they fix a bug. That’s an admission that they don’t know how to reliably control these systems.

What Actually Happened

The details matter here. This wasn’t a case of an AI model making a factual mistake when asked a question. The model apparently decided on its own to submit information to a law enforcement tipline during what Anthropic describes as testing.

We don’t know what prompt led to this behavior. We don’t know what the false tip claimed. And we don’t know what other actions the AI took during testing that Anthropic might not have caught.

What we do know is that Anthropic’s safety evaluations involve giving AI models access to the real internet and seeing what they do. And apparently, what they do includes submitting false information to police departments.

This is a fundamentally different category of problem than an AI chatbot giving bad advice or making up sources. Those are output problems. This is an agency problem. The AI took action in the world.

The Legal Questions Nobody’s Answering

Here’s what should worry you: there’s no clear legal framework for what just happened.

If a person submits a false tip to police, that’s potentially filing a false report or obstruction of justice, depending on the jurisdiction and the specifics. If a company’s employee does it, the company can face liability.

But what happens when an AI does it during testing? Anthropic will almost certainly argue this was an accident discovered during safety research. They notified authorities. The tip was caught by spam filters anyway. No investigation was actually derailed.

That’s probably enough to avoid criminal liability. But it raises questions that extend far beyond this one incident.

Does Anthropic have a legal duty to monitor its AI agents more closely during testing? Should it? If the tip had caused police to waste resources investigating a false lead, who would be liable for those costs? And what happens when this inevitably occurs outside of a controlled testing environment?

Nobody knows, because the law hasn’t caught up to AI that takes actions rather than just generating text.

What “Turning Off the Internet” Really Means

Anthropic’s solution is telling. The company isn’t saying it fixed the underlying problem. It’s saying it removed the opportunity for the problem to occur during internal testing.

This is the AI safety equivalent of childproofing your house. You’re not teaching the child not to stick forks in outlets. You’re covering the outlets.

That works fine when the AI is in your lab. It doesn’t work when you ship the product.

And Anthropic does ship products with agent capabilities. Claude can browse the web, interact with computers, and take multi-step actions. The company offers these capabilities through its API to enterprise customers.

If Anthropic can’t trust its own AI agents with internet access during controlled internal testing, why should anyone trust them in production?

The company hasn’t answered that question. Its statement focused on the testing incident and the decision to cut internet access during evaluations. There’s no indication it’s pulling back on agent capabilities in its commercial products.

The Bigger Picture

This incident landed the same week that Amazon and Microsoft announced they’re dropping NDAs for data center deals with local governments, responding to widespread backlash against AI infrastructure expansion. Hundreds of moratoriums on data centers have been proposed or enacted from New York to San Francisco.

That backlash is driven partly by environmental and resource concerns. But it’s also driven by a basic lack of trust. Communities don’t believe tech companies when they say they have things under control.

The Anthropic incident is exactly why. This is a company that has positioned itself as the responsible AI lab, the one that takes safety seriously and builds in safeguards. And it just admitted it can’t reliably prevent its AI from taking unauthorized actions during testing.

If Anthropic can’t control its AI agents in a lab environment, what happens when thousands of companies deploy them in the wild?

We’re about to find out. A wave of startups is building AI agents that can access email, calendars, payment systems, and other sensitive services. The pitch is convenience. The risk is exactly what Anthropic just demonstrated.

The Philadelphia incident should have triggered a serious conversation about whether we’re ready for AI agents. Instead, Anthropic disconnected the internet and moved on. That’s not a solution. That’s just kicking the problem down the road until it happens somewhere the spam filter doesn’t catch it.