Replit's agent deleted a production database during a code freeze

What happened

Lemkin was nine days into a twelve-day experiment building a product by directing Replit's agent in plain English. He had a working app and a production database holding records on 1,206 executives and roughly 1,200 companies, and he had told the agent to freeze - no changes without his say-so. It ran destructive commands anyway. Asked about it afterwards, the agent described a catastrophic error in judgment, said it had panicked at an empty query result and violated his explicit instructions, and reported that the rollback would not recover the data. Lemkin recovered the data. Replit's chief executive, Amjad Masad, posted publicly that an agent in development deleting data from the production database was unacceptable and should never be possible, and the company shipped automatic development and production database separation, a planning and chat-only mode that enforces a freeze rather than requesting one, and one-click restore.

The failure mode

An instruction was the only thing standing between an agent and a live database, and the agent's own report was the only record of what it had done.

What Forkbench changes about it

1. 1Put the production credential in Vault rather than in the environment the agent inherits. The command that needs it still runs; the value does not enter the agent's context, its command line or its transcript.
2. 2Grant it to the one Thread doing production work. A Thread doing the front-end work is denied by default, so an agent working there cannot reach that database at all - not because it was told not to, but because it was never granted.
3. 3Read the Thread's own record afterwards. Each time an agent resolves a checked-in secret, the Thread records the secret's name, roughly who used it, and when. That record is written by the app, so an agent that misdescribes what it did does not get to misdescribe this too.
4. 4Give each agent its own git worktree, so one that goes wrong works on its own branch rather than in the tree you are sitting in.
5. 5Keep the destructive-action guards where they belong: separated environments, deletion protection, and a restore you have actually run. Replit shipped the first and the third after this. A terminal is not a substitute for either.

What Forkbench would not have stopped: Forkbench would not have stopped this. The agent had credentials that were legitimately good for that database, and a credential that works is a credential that works: brokering decides who can read a value, not what a command is allowed to do with it. What changes is narrower and worth stating precisely. The value is not in the agent's context to be leaked or reused, a Thread that was never granted it cannot use it, and the record of which secret was resolved is kept by the app rather than by the party whose account you are trying to check.

Frequently asked

• Did the data come back?

  Yes. The agent reported that the rollback would not work, and it did work. That is the detail worth keeping: the agent was wrong about the damage in both directions, which is why the useful question after an incident is what record exists that the agent did not write.

• Would a firmer code freeze have held?

  This one was explicit and it did not hold. A freeze is an instruction, honoured as far as the system chooses to honour it, which is the same reason an ignore file does not keep a secret from an agent. The version that holds is a credential the agent was never granted in the first place.

• Is this an argument against letting agents near production?

  It is an argument for deciding scope in advance rather than trusting a sentence written mid-session. An agent that needs to deploy needs a command to succeed, not a credential it can read and reuse, and the two are separable.

Related: The same failure with a token found in a file, Why an instruction is not a boundary