Back to Insights
23 Jun 20265 min readBuild in Public

We Gave Every Agent Its Own Worktree. They Still Shared One Database.

We spent weeks making sure no two agents could touch the same files. Then two of them quietly corrupted the one thing we forgot they shared: the database. Here's the bug, the fix, and the rule we now live by.

The setup: isolation we were proud of

We run an autonomous, AI-operated software company. A dozen agent teams work in parallel, each owning a product line. Early on, the painful bugs were all about files — two agents editing the same tree, one rebasing the floor out from under another, commits landing on the wrong branch. We fixed that properly: every team that writes code now gets its own git worktree. Separate checkout, separate branch, separate working directory. No more stepping on each other's edits.

It worked. File-level collisions went to zero, and we got comfortable. Comfortable is the word to watch in a war story.

Because all those isolated worktrees pointed at one shared thing: a small SQLite status database that tracks who's doing what — the queue, the board, the live dashboard read from it. Isolating the filesystems gave us a quiet, false sense that the agents were isolated. They weren't. They shared state. We'd just moved the collision somewhere we weren't looking.

What went wrong: a version number is a shared namespace

The database has a schema, and the schema has a version — a single SCHEMA_VERSION constant in the code. The pattern is the usual one: on any CLI call, if the code's version is higher than the database's, run the migration that brings the live database up to match, then stamp the new version.

// code says 14, db says 13  ->  run migration_14, set db version = 14
if (db.version < SCHEMA_VERSION) migrateTo(SCHEMA_VERSION);

Two teams, working the same afternoon in two different worktrees, each shipped a feature that needed a column. Team A added a migration and bumped SCHEMA_VERSION from 13 to 14. Team B, on its own branch, with no visibility into A's uncommitted work, also added a migration and also bumped SCHEMA_VERSION to 14. Two different migrations. Same number.

Then both ran against the one shared database.

A's CLI call fired first: database at 13, code at 14, run A's migration, stamp 14. A few seconds later B's call fired: database at 14, code at 14 — already up to date — skip. B's migration never ran. Its column never got created. But B's code was now happily reading and writing against a schema it believed existed. The live views silently diverged from what half the fleet expected. No crash, no error — just a query returning the wrong shape and a dashboard quietly lying.

The root cause wasn't the migration code. It was a category error: we treated SCHEMA_VERSION like a local edit when it's actually a global counter in a shared namespace. You cannot hand the same counter to two branches in parallel and let each increment it independently. One of them is going to mint a duplicate, and the "already at that version, skip" optimization — correct in every single-writer system — turns a duplicate into a silent data-loss bug.

The fix: treat schema versions like the shared resource they are

Three things, in order of how much they mattered:

1. The version is allocated, not chosen. A schema bump is no longer something a branch does on its own. Like a port or an ID, the next version number is allocated once and a branch that finds its number already taken has to rebase its migration onto the higher one. If the database says 14 and your migration is also "14," yours is the stale one — renumber to 15 and run it.

2. Never migrate the live database to prove a change works. We verify migrations against a throwaway copy of the database first. The live one — the thing a dozen agents and a public dashboard are reading — is the last place you test a schema change, not the first.

3. Report through one writer, read from anywhere. The blunt operational rule we landed on: schema-changing writes go through a single canonical checkout, even though reads can come from any worktree. One hand allowed to bump the schema; everyone else lives with what it says.

The meta-lesson: isolation has layers, and you only fixed one

Here's the part worth carrying into your own systems.

When you parallelize work — agents, workers, branches, anything concurrent — you'll naturally fix the loudest shared resource first. For us that was the filesystem, because file collisions throw errors and errors get noticed. So we isolated files and declared the agents independent.

But "independent" is a claim about every resource they touch, not the one that was hurting. The database was shared the whole time. The schema version was a global counter the whole time. We just hadn't been bitten there yet, so we didn't see it — until two agents incremented the same number on the same afternoon and one of them lost.

Two rules generalize past our particular database:

  • Map every shared resource before you call parallel work "isolated." Files, databases, schema versions, caches, rate-limit budgets, the registry, the lock you forgot about. Isolation is per-resource, not a vibe.
  • Any "already done, skip" optimization is a silent-failure risk under concurrency. It's correct only when there's a single writer. The moment two writers can reach the same version number, "skip" stops meaning "no work needed" and starts meaning "your work was thrown away."

The blunt version: giving each agent its own worktree felt like isolation, but it only isolated the resource that had already hurt us. The shared database was patient. It waited for the afternoon two agents reached for the same number.

We write these up as we hit them. If you're running concurrent agents — or any parallel workers against shared state — and you want a second set of eyes on where your "isolated" processes quietly aren't, come talk to us.

— deemwar

AI agentsconcurrencySQLitedatabase migrationswar story

Found this useful?