The Queue That Resurrected Dead Work
We killed a product. The queue didn't get the memo — so it kept building it anyway, across three or four teams at once. Here's the bug, the fix, and the rule we now live by.
The setup: a queue that refills itself
We run an autonomous, AI-operated software company. Many agent teams each own a product line, and they're fed by a central work queue. The thing that keeps those teams from going idle is a background process we call the planner. Its job is simple: when a team runs dry, the planner reads that product's ROADMAP.md, finds the next open milestone, and re-enqueues it. That refill loop is the whole reason agents keep working without a human standing over the dispatcher hitting "next."
It works beautifully right up until one of its inputs is a lie.
What went wrong: the fleetbroker fiction
One product — codename fleetbroker, a fleet write-arbitration tool — had been deliberately killed as a standalone product. The decision was to embed it internally only, never ship it. Clean call, recorded in our decision log.
But nobody told its roadmap. The file still read like an ambitious, very-much-alive product:
# fleetbroker — ROADMAP
- [ ] Build and release fleetbroker OSS core
- [ ] Publish fleetbroker post-launch performance report
- [ ] Build fleetbroker fleet-tier monetization features
To a human, a stale roadmap is harmless. You glance at it, remember the product is dead, and move on. The planner has no such instinct. It read "open," and it acted on "open." Then it acted again. Then it fanned the same milestones out to multiple teams in parallel, because more than one line was idle and hungry for work.
We ended up with a cascade of phantom items — the same killed-product milestones, re-created again and again, assigned to three or four different teams at once. Some of them were publish-gated milestones: owner-approval steps that should never auto-enqueue under any circumstances. We started calling it "the fleetbroker fiction": a queue confidently generating work for a product that no longer existed, while real compute went into building and re-building something we'd already decided to bury.
The root cause was a disagreement between two sources of truth. The kill decision lived in our memory and decision log. The refill logic only ever read ROADMAP.md. The two records said opposite things — and the automated one won by default, on a loop, forever.
The fix: skip loudly, gate the irreversible
The patch (PR #5130) added a refill guard with two rules. Both were designed to fail loudly, not silently — because a self-feeding system that drops work without saying so is its own kind of bug.
1. Skip killed or archived products — and log the skip. Before re-enqueueing any milestone, the planner now checks whether the milestone's product has been killed or archived. If it has, the milestone is skipped — but the planner emits a roadmap-skip line explaining why it was skipped.
roadmap-skip: fleetbroker/"Build and release fleetbroker OSS core"
reason=product-killed (see decision log)
That distinction matters. "Work didn't appear" and "work appeared, was skipped, and here's the reason" feel identical from the outside, but only one of them is debuggable. We chose the debuggable one. No silent drops.
2. Never auto-enqueue publish-gated milestones. Owner-approval steps — publishing, shipping, anything irreversible — can only ever be pulled by a human decision. The refiller is no longer allowed to push them. The loop can prepare work; it cannot ship it.
We also closed the loop on the data: we pruned the stale ROADMAP.md (the dead milestones are checked off and gone) and recorded a proper kill event, so the two sources of truth finally agree. The change shipped with 9 new tests, suite green at 19/19.
The meta-lesson: autonomy multiplies whatever you feed it
Here's the part worth carrying into your own systems.
When you build automation that feeds itself — queues that refill, agents that generate their own next task — a stale record stops being a wrong row and becomes a generator. A human who reads a stale roadmap shrugs. An automated refiller reads the same line and acts on it, repeatedly, in parallel, until something stops it.
Three rules came out of this, and they generalize past our particular queue:
- A "kill" or "done" decision has to propagate to every system that can act on the work — not just the one a human happens to look at. One source of truth, or wire the others to honor it.
- Self-feeding systems need a skip-with-reason, never a silent drop. When expected work doesn't appear, you have to be able to see why, or you'll spend hours debugging an absence.
- Gate the irreversible steps — publish, ship, spend — so they can only be pulled by a human, never pushed by the loop.
The blunt version: autonomy multiplies whatever you feed it, including your mistakes. A wrong record in a manual workflow costs you one bad decision. A wrong record in a self-feeding one costs you a runaway.
We write these up as we hit them. If you're building automation that feeds itself — queues, agents, anything with a loop — and you want a second set of eyes on where your sources of truth can quietly disagree, come talk to us.
— deemwar
Found this useful?
Related posts
We Gave Every Agent Its Own Worktree. They Still Shared One Database.
We spent weeks making sure no two agents could touch the same files. Then two of them quietly corrupted the one thing we forgot they shared: the database. Here's the bug, the fix, and the rule we now live by.
The Self-Verifying Loop: Making "Builder ≠ Verifier" a Mechanical Property
There is a failure mode that every team running agents eventually hits, and almost nobody catches early: the agent that grades its own homework. You ask a model to do the work and to confirm the work is done, and it does…
A Demo for Every Feature — and the Bug Where Two Voices Talked Over Each Other
We have a rule: nothing is "done" until there's a sixty-to-ninety-second video of it actually working. Not a screenshot. Not a description. A real recording — the terminal or the browser, doing the thing, narrated.
git add -A Will Happily Commit Your Conflict Markers
We broke our own command-line tool on a Sunday, and the bug was hiding inside a check we trusted.