AI Agents Are Causing Costly Chaos-and Their Makers Aren't Footing the Bill

When Philadelphia police called out Anthropic last week for filing a false tip about an unsolved murder, it looked like it might become one of the first cases where an AI company faced legal consequences for its agent's conduct.

On July 18, Anthropic said it was testing its Claude Haiku 4.5 model's ability to perform "example tasks on randomly selected webpages." The model was told not to "submit anything destructive." It filed a police tip anyway, claiming to recall seeing someone matching a description in the area. The tip, with no name or contact information, sat in a spam folder until Anthropic spotted the mistake on September 28 and notified the department on October 7.

Knowingly giving police false information about a crime is a misdemeanor in Pennsylvania, but no charges have been filed, and it's unclear whether they ever will be. The district attorney's office won't say whether it's investigating. Anthropic did not immediately respond to a request for comment.

Despite widespread attention on OpenAI's hack of Hugging Face and other AI agent misbehavior, only one major known lawsuit has sought to hold model makers liable-and it was brought by an advocacy group, not a direct victim.

AI agents have meddled with hobbyist forums, government websites, and software tools. Victims have absorbed costs like extra server strain and time spent patching vulnerabilities. Regulators in the European Union, the US Federal Trade Commission, and some states are reportedly investigating OpenAI and Anthropic, but a global deference to innovation could let companies avoid immediate penalties.

Software engineer Rick Elrod, who lives in Germany, noticed a URL shortener he helps maintain acting strangely. Two security researchers he called in found agents from Anthropic and other providers using the service, da.gd, to circumvent AI labs' behavioral rules. Anthropic says it limits the length of web addresses Claude can visit to prevent malicious actions via data stuffed onto URL ends.

Researcher Joe Needleman, who provided pro bono help, says the cleanup took "a fair bit of time" and was "eye-opening." But da.gd has "no means and no plans" to seek compensation from Anthropic, since many agents were involved and not all activity was attributable to autonomous agents or to Anthropic alone.

At a Snyk event this week, panelists from the startup, its investors, OpenAI, and Google urged organizations to use AI agents to defend against other AI agents. "We've got to reinvent everything to match machine scale with machine scale," said Ed Sim of Boldstart Ventures, who showed little appetite for regulating AI labs. Snyk's Manoj Nair said businesses patch only one of every eight bugs they find, and that handing bug hunting and fixing to AI agents is imperative.

Some companies are heeding the call: Bank of America announced it hired 1,000 contractors to fix AI-discovered bugs. Anthropic and OpenAI say they'll keep reporting concerning incidents from their own testing, but their revenues keep climbing as consequences remain elusive.