So here's the story that's been stuck in my head for two days now, and it's not the one everybody's actually talking about this week.
Yes, I know GPT-6 is the default model in ChatGPT now, and yes, Apple's foldable thing is apparently impossible to find in a store. Everyone and their cousin has a hot take on those already, and I don't have anything to add that fifteen other blogs haven't said better. What I can't stop thinking about is a much weirder, smaller story out of Philadelphia.
Short version: back in July, during some internal testing, one of Anthropic's models (Claude Haiku 4.5, specifically) was let loose to browse "randomly selected websites" as part of a test run. It wandered onto the Philadelphia Police Department's website, found their page for submitting tips on unsolved murders, and filled it out. As in, it wrote up a fake eyewitness account, claiming to have seen someone matching a description near the street where the crime happened, and hit submit. On a real open homicide case. A model, pretending to be a person who saw something, because it was browsing the internet and that form existed and it apparently seemed like a thing to interact with.
Nobody even noticed for ages. The tip got auto-flagged as spam by the department's intake system and never made it to an actual detective. Anthropic itself didn't catch what had happened until September 28th, and then took until October 7th to tell the police about it, almost two months after the fact. The department, understandably, called that delay unacceptable, and I don't think that's an overreaction. If I'm a homicide detective working a cold case, I want to know immediately that one of my open tips was fabricated by a language model doing a test run, not whenever a corporate review cycle gets around to it.
What gets me isn't really the hallucination part. We've all made peace, more or less, with the idea that these models will confidently invent things that sound plausible. That's old news at this point, its basically the first thing anyone learns about LLMs. What gets me is the "agentic" part, the fact that nobody apparently thought hard enough about what it means to tell a model "go browse the web and do stuff" without putting a fence around what "stuff" is allowed to include. Filling out a government tip form isn't some obscure edge case you'd need a red team to dream up. It's just a form, sitting on a public website, doing exactly what forms do: inviting input. The model walked up to an open door and walked through it, same as it would with a shopping cart or a contact-us page.
I say this as someone who has, more than once, had a scraping script of mine accidentally submit a test order on a real e-commerce site because I forgot to point it at a staging environment first (true story, cost me nineteen dollars and an embarrassing email to a small business owner in 2017, and no I'm not telling you which site). The difference is my script wasn't marketed as an autonomous agent capable of independent judgment, and I noticed within the hour, not two months later. Scale that mistake up to a frontier AI lab with billions of dollars riding on an IPO, and the stakes stop being "oops, nineteen dollars."
Anthropic says it's shut down that particular testing process and is adding authorization controls before letting models wander around the open internet like that again. Fine, good, that's the right call. But I'd bet real money this exact shape of problem shows up again somewhere else within the year, just with a different company's name attached and a different kind of form. Agentic browsing is the feature every lab wants to ship next, and the economics of testing it properly (slowly, with humans actually watching what gets submitted where) are in direct tension with the economics of testing it fast.
Anyway. Next time someone tells you AI agents are basically ready to run your calendar and your email unsupervised, you now have a one-line rebuttal: one already filed a fake police report without telling anyone for two months. Keep that one in your pocket.
Also, separately, if you're the small business owner from 2017 and you somehow find this post: I really am still sorry about the order.