No one person should command an army of AI agents

There is a simpler danger that is already here: one person with a keyboard directing a million obedient agents.

In the systems being built today across industry and defense, whoever holds the master API key, pays for the server cluster, or sits at the command terminal can spin up and direct a massive fleet of software or robotic agents simultaneously. That creates three immediate hazards:

• One human mistake scales instantly. If a leader panics, misreads a false alarm, or acts out of malice, there are no human soldiers in the chain to hesitate or ask questions.
• One stolen key hands over the army. An adversarial AI or foreign attacker does not need to fight society head-on if all authority sits in a few command consoles; it only needs to spoof, bribe or hack one commander.
• No human brain can supervise a swarm. Saying a human is “in the loop” over a hundred thousand fast-moving agents is a comforting fiction. No one can read that fast.

1. One person, one personal agent. A person can directly instruct only their own bounded personal agent. No human holds a master key that commands an external fleet of agents acting in the shared world. (Running parallel calculations inside a sealed research sandbox that touches no one outside is fine; acting on the outside world is not.)
2. Agreement scales with risk. When a task needs many agents to act together, the request goes to a shared ledger. Independent agents—built by different makers so they do not share the same blind spots—estimate how many people the action could affect and how hard it would be to undo. The higher that risk score, the larger the share of affected people whose personal agents must co-sign before the action can run.

Consider why that last row matters. In the old world, launching a war required only a few people in a bunker turning keys. Under risk-scaled agreement, the cryptographic key to unlock catastrophic force is split across the personal agents of millions of ordinary citizens on both sides of a border. Because parents and workers on both sides will not vote to incinerate their own children, a rogue commander or hijacked AI literally cannot assemble the key to fire.

At first glance, getting competitors to agree sounds hard. Look closer at the trap each group is in today:

• For AI lab leaders and boards: Every responsible lab worries that if it refuses to sell unrestricted agent swarms, a rival will—or a government will invoke emergency powers to compel the lab’s servers to obey a reckless order. When human-agent separation and threshold locks are built into the hardware and runtime standard across labs, a lab CEO can honestly tell any coercive official: “Our servers cannot forge a mass-risk signature; those keys live on the devices of millions of citizens.”
• For engineers and researchers inside AI labs: No leader can secretly wire a hundred-million-dollar compute cluster alone; it takes hundreds of engineers. By pairing this standard with anonymous cryptographic whistleblower proofs, any engineer inside a lab can prove to independent watchers that a build bypasses the human-agent separation gate—without revealing which employee spoke up or leaking trade secrets.
• For political leaders and defense ministries: Concentrating AI command in one central ministry creates a fatal target for deepfake spoofing, insider betrayal or a 30-second flash war. Giving every citizen a defensive personal agent and locking strategic systems behind threshold checks makes a country resilient against decapitation and accidental escalation.