Sample report: Agent secrets audit of deemwar's own setup

Prepared by deemwar as a sample of the fixed-price "Agent secrets audit" deliverable. The subject is our own working environment, audited the same way we audit any engagement. Findings are written as classes of leak path and where they live. No credential values, names of live keys or working exploits appear in this report.

Scope

One developer Mac running a fleet of AI coding agents (the coding agent, plus other agent CLIs) around the clock; the company operations repository; one private object-storage bucket used to snapshot agent memory and state; and the agent transcripts and logs written by those sessions. Out of scope: other organisations' systems, cloud consoles, and third-party SaaS beyond the tokens held on the machine.

Method

1. Map every path by which a credential can reach an agent's context window, and so reach the model provider: inherited environment, shell output, process listings, files read, tool results, logs and transcripts.
2. Scan stored transcripts and logs for live values with a secret scanner, one file per invocation. (An earlier batched run silently scanned nothing, because the scanner takes one file per call; a guard we have not seen fail proves nothing.)
3. For each credential an agent can reach, list what it unlocks and whether any other system shares the same value.
4. Test each guard by making it fail: a guard that has never been seen to refuse proves nothing.

Findings, ranked by severity

1. Critical: every agent shell inherited every API key. The login shell profile sourced a file of API keys, so every agent process carried all of them in its environment. One agent ran an environment dump filtered by a keyword and printed a messaging-bot token straight into a model context. Five such env-dump leaks happened within 48 hours. Fix (applied): the shell profile no longer loads keys; they live in a broker that injects them only into the one child process that needs them (see the worked example). Forbid env, printenv and set dumps in agent instructions.

2. Critical: live values already sat in stored transcripts. The scan of about 11,700 transcript and log files found real production values: database passwords, signing keys, storage keys, registry tokens, and cloud, Git hosting and model-provider keys. Fix: rotate everything found, then prove each old value is dead upstream (a 401 against the old value, not merely a new value existing). Treat transcripts as secret-bearing data: they are excluded from backups and sync unless they have been scanned.

3. High: one token reused across unrelated systems. Revoking one leaked Git hosting token also broke a container-registry push, and would have broken the next deploy, because the same value was registered in three places. Fix: one credential per consumer. Before any revocation, ask the broker where the value is used, fingerprint-match every stored entry, and line up replacements first, unless the token is being actively abused.

4. High: process listings print secrets. Listing processes with full arguments (ps aux, pgrep -fl) shows command lines and, on some setups, environment content, so a routine "is it running?" check can expose values. Fix: agents count processes (pgrep -f <name> | wc -l) and never list them with arguments. Daemons take secrets from the broker, not from argv.

5. Medium: the backup guard failed open. The state-snapshot script (ceo-state.sh push) refuses to upload when its scanner finds a credential. But it never checked the scanner's own exit status, so a crashed scanner printed nothing, which counted as "no hits", and an unscanned snapshot could have been uploaded. Separately, real refusals went unnoticed for 15 days because nothing was watching them. Fix (applied): a scanner failure now refuses the push, with a test using a malformed config; files are excluded only while their hash matches a pinned value; a scheduled push alerts on refusal and records a last-good time that is checked daily.

6. Low: secret-shaped test fixtures. Detection fixtures and vendored libraries trip the scanner and train people to ignore it. Fix: keep them in a named hold list until each one is proven synthetic.

Worked example: brokering and redaction

Two layers, so a value never enters a context window even when an agent makes a mistake.

• Broker (sec). Secrets live in sec, not in files or the shell. An agent writes a command with a placeholder, such as sec exec -- curl -H "Authorization: Bearer {{API_TOKEN}}" ..., or sec run NAME -- cmd. The broker fills the value inside the child process only, and scrubs the value, plus its base64, hex and URL-encoded forms, out of everything the command prints. sec seal <file> --check tells you whether a file holds a live value before it is sent, committed or backed up. sec where NAME lists every placement recorded in sec; with a fingerprint match on top, that is what makes finding 3 safe to fix.
• Context mask (veilpipe). Installed into the agent harness, veilpipe swaps any known secret value for a $NAME placeholder before the model sees tool output. The agent can still use $NAME in commands, and the shell expands it at execution. Canary test: the model saw only the placeholder, and the transcript held zero copies of the value. On a real environment dump, 68 of 82 values were hidden; the 14 left were 13 IDs, usernames, IPs, paths and links, and one credential that a user rule for *TOKEN, *SECRET, *PASSWORD, *_PW and *API_KEY variables then caught.
• Standing rules that back them up. No secret value in a prompt, file, log or command output. Agents never read vault directories, *.env files or credential files unless the owner names that file for that task. Never list process arguments. Every backup or push passes a scanner that also fails on its own failure.

Residual risk

• veilpipe covers only harness sessions started after it was installed. Other agent CLIs on the machine rely on the broker and the rules alone.
• An old copy of one rotated key remains in a local file on disk; the key itself is proven revoked.
• Pattern-based masking misses credentials with unusual shapes, so the transcript scan should run on a schedule, not once.

The engagement

Agent secrets audit: 35 hours, USD 3,150 fixed, 1 week. We produce this same report for up to 3 repositories, 1 CI system and 1 agent setup, and apply the broker and mask to one repository as the worked example. You get written findings ranked by severity, a fix plan, a 45-minute readout call, and one re-check after 30 days.

Get a quote: https://deemwar.com/consulting?utm_source=insights&utm_medium=sample-audit