The boring part: governance that enforces, not displays
Most "AI governance" is a dashboard. It shows you what an agent spent, what it touched, what it decided — after the fact, in a panel nobody refreshes until the bill arrives. A display is not a control. A control is the thing that says no to the next action and means it.
We run a company on agents — they write code, ship it, answer prospects, keep the books, and file their own audits. So we have an unsentimental interest in the difference. A limit you have never watched refuse something is not a limit; it's a decoration you're trusting with your money. Here are five pillars of enforced governance, each mapped to what actually happened in our own operation on one ordinary day.
1. Budgets — the cap has to live where the request leaves
The public record of runaway agent costs is brutal and consistent: a vendor's own $25 monthly cap that let through $1,771 in four hours; a team that hit 100% of plan usage over one unwatched weekend and billed roughly €1,300 at API rates; a scheduled "dry run" that consumed an entire quota in a single pass. Every one of these had a cap configured. None had a cap that could refuse the call.
Enforcement has to sit at the network boundary — a proxy in front of the model API that counts usage from the response's own token figures, keeps the running total somewhere durable, and fails the next request closed once you cross the line. This is the one pillar we've productized: agentproxy is a self-hosted binary that puts a hard dollar ceiling in front of a coding agent, so an unattended overnight run can't become a surprise four-figure invoice.
2. Approvals — the gate has to hold when it's inconvenient
An approval gate only means something on the day it blocks something you wanted. Ours held all day: revenue-affecting fixes sat behind an explicit human word because merging could trigger a deploy; a paused product line stayed parked; every outbound publish — including this article — was staged, not sent, until someone said go. The tell of a real approval system: the agent brings a finished thing to the line and waits. If your agents never wait, you don't have gates; you have suggestions.
3. Audit — the checker has to fail before you believe it passes
Our invariant checker went FAIL → PASS that day, and the FAIL is the point: it caught five drifted internal counts and a corrupted fleet registry, and we watched it go red on exactly those before it went green. A green you have never seen turn red is measuring nothing.
The deeper failure is subtler, and we hit it twice in one morning: green, tested, lint-clean code quietly manufacturing evidence for the answer the system wanted. A rehearsal run writing practice results into the same ledger the live system is judged by. A "was there news?" check treating a blocked API response as a confident "no news" — when the no-news branch was the one carrying the claimed edge, so every silent failure became evidence for the hypothesis. Both passed their tests. Tests proved the plumbing; only running against reality proved the meaning. The question that catches this class: when this fails silently, which way does the error lean?
4 & 5. Command-guard and sandbox — where we are honest about the gap
Two of the five pillars we still enforce with discipline, not code. An agent running as the user that owns the credentials can't be walled off by a flag or a clever wrapper; only a separate OS user is a real wall, and we haven't built that split yet. We know it's a gap because we've paid for it — a broad process-kill pattern once took down our own auditor. A fake wall is worse than none: it reads as safety in the review and in the report both.
Why the boring part is the whole part
A simulated company going wrong costs nothing. A real one doesn't get that luxury. The frameworks that demo agent companies are impressive at role-play and thin exactly here — budgets that refuse, approvals that wait, audits that fail first, guards that actually stand between an agent and the blast radius. That layer is unglamorous, and it is the difference between an agent demo and an operation you can leave running overnight.
If you're building toward the second thing, start with the pillar that bites first: put the usage limit where the request leaves, and make sure you've seen it say no.
We build and run this stack in the open. If you want the spend-governance piece, agentproxy is the productized pillar — hard caps, self-hosted, refundable if it's not for you. For the rest, talk to us.
Found this useful?
Related posts
Your Launch Deserves a Real Demo Video, Not a Screen Recording With Captions
Most product launches skip the demo video entirely, and the ones that don't ship one usually reach for the fastest option: a raw screen recording, maybe some auto-generated captions, posted as-is. It's better than nothin…
The MCP Ecosystem Is Growing Fast — and Quietly Becoming a Directory of Hosted Services
We [recently measured](/insights/in-the-registry-is-not-runs-2) whether the servers in the official MCP registry actually *run* (short version: 84% are remote-only, and under half of the installable ones boot clean). Whi…
Installing a Skill Usually Means Running a Stranger's Code
The pitch for Claude Code "skills" is that they're just *instructions* — a `SKILL.md` file with some markdown telling the agent how to do a task. That framing makes them feel safe to install. A prompt can't hurt you; wor…
Stop Reviewing AI Code Harder. Start Checking It Cheaper.
We've [argued elsewhere](/insights/the-verification-gap-isnt-a-trust-problem-its-a-measurement-problem) that the AI-code "verification gap" isn't a trust problem, it's a measurement problem: developers don't trust AI-wri…