The Botnet We're Building Ourselves

An AI agent needs access to do its job: your email, your calendar, your codebase, your customer data, your file system. You grant it. The agent connects through a provider's infrastructure, which means the provider holds the session. Multiply that by a few million users and you have a globally distributed network of authenticated, pre-positioned access points maintained by the targets themselves.

A traditional breach works backwards. An attacker finds a vulnerability, exploits it, then spends days or weeks moving laterally through a network trying to reach something valuable. Most of that time is spent on access. Getting credentials, escalating privileges, finding the data worth taking.

AI agents skip the entire chain. The access is already granted and the sessions are already authenticated. A compromised provider doesn't need to hack anyone. It activates what's already there.

The victims build and maintain the infrastructure. Every time someone grants a new scope, connects a new account, or logs a bot into a new service, the surface grows without the attacker lifting a finger. It's a botnet that upgrades itself.

We're not talking about one company's database getting exfiltrated. A single provider compromise could activate millions of agents simultaneously, each one already sitting inside its owner's most sensitive systems. Email accounts that reset passwords for everything else. Code repositories with production secrets. Financial systems with live credentials. CRM platforms with every customer relationship the company has.

The window between "everyone adopts AI agents" and "someone exploits the access they've been given" is not measured in years. The infrastructure is being assembled right now, this quarter, by the most enthusiastic early adopters — who also happen to be the highest-value targets.

Start with a threat model, not a terms-of-service checkbox. Simon Willison has named the lethal trifecta: any agent that simultaneously holds access to private data, reads untrusted content, and can send data outbound is one prompt injection from exfiltration. Every agent design should break at least one of those three legs. Agents that browse the open web don't get privileged sessions. Agents with privileged sessions don't browse the open web.

If you've worked in process safety, the fixes map to a framework the industry has used for decades: the hierarchy of controls.

Telling an agent "don't access financial accounts" is an administrative control. A prompt is a procedure, not a safeguard. It's the sign on the valve. It works until someone ignores it, or until a prompt injection overwrites it. Process safety ranks administrative controls last for exactly this reason: they depend on everything behaving correctly.

Credential scoping is an engineered control — the interlock, not the sign. Money stays on your phone. Your mailbox gets a scoped, minimal-permission token for agent access, never a full browser session on a shared cloud computer. Work systems get scoped service accounts, not your personal login. The credential's actual scope constrains what a compromised agent can do, regardless of what the instructions say. If the credential can reach it, assume it will.

Device separation is inherent safety. Physical isolation. Run your agents on a dedicated machine that has no banking sessions, no password manager vault, no identity-root credentials. Talk to your agents remotely from the device that holds your identity, never run them on it. This doesn't depend on the agent, the provider, or the instructions behaving correctly. It works because the path doesn't exist.

None of this is exotic. Dedicated machines, credential scoping, least privilege, kill switches. Security teams have applied these principles for decades. Process safety has ranked them since before most of these companies existed. The gap is that almost nobody is applying either framework to AI agents yet, because the adoption curve outran the threat model. We're in the brief period where the infrastructure is live, the attack surface is growing daily, and the institutional response is still "we'll figure it out."

That period has an expiration date.

Cybersecurity went through the same arc: unregulated adoption, incident-driven frameworks, compliance mandates. AI agent security is at step one. But the industry building AI governance from scratch, incident by incident, mandate by mandate, is ignoring that a fully worked regulatory regime for high-consequence autonomous systems already exists. Forty years of case law, audit practice, and failure data. PHA before deployment. MOC before change. Incident investigation with findings that must close. Operating procedures that survive shift handoffs and 3 a.m. calls.

The EU AI Act is reinventing pieces of 29 CFR 1910.119 without knowing it. The frameworks AI agent deployments need aren't missing. They're in the wrong industry's filing cabinet.