The Cheapest AI-Code Check You Can Ship Today

We keep arguing that the fix for AI-written code isn't more trust or more review — it's making the check cheap enough that skipping it makes no sense. That's easy to say in a blog post. This one is the opposite: here's a real check, in full, that you can paste into your repo in the next five minutes. It catches the single most common and most dangerous AI-code mistake — a dependency that doesn't exist.

The problem, briefly: LLMs hallucinate package names, 19.7% of the time in one study, and the names they invent are predictable enough that attackers pre-register them ("slopsquatting"). When your agent confidently adds requests-oauth-helper to requirements.txt and it turns out a stranger registered that name last week, "it installed fine" is exactly the wrong signal. The check is dead simple: does every dependency actually exist on the registry? That's a lookup, not a judgment — perfect for CI.

The check (20 lines, no dependencies but curl)

#!/usr/bin/env bash
# check-deps.sh — fail CI if a dependency doesn't exist on the registry.
# Usage: ./check-deps.sh requirements.txt   (or package.json)
set -uo pipefail
f="${1:-requirements.txt}"; bad=0
exists() { # exists <ecosystem> <name>
  if [ "$1" = pypi ]; then url="https://pypi.org/pypi/$2/json"; else url="https://registry.npmjs.org/$2"; fi
  code=$(curl -s -o /dev/null -w '%{http_code}' "$url"); [ "$code" = 200 ]
}
case "$(basename "$f")" in
  package.json) eco=npm;  names=$(python3 -c "import json;d=json.load(open('$f'));print('\n'.join({**d.get('dependencies',{}),**d.get('devDependencies',{})}))");;
  *)            eco=pypi; names=$(grep -oE '^[A-Za-z0-9._-]+' "$f");;
esac
for n in $names; do
  if exists "$eco" "$n"; then echo "  ok   $n"
  else echo "  MISSING  $n  (not on $eco — hallucinated or unregistered)"; bad=1; fi
done
[ "$bad" -eq 0 ] && echo "✓ all dependencies resolve" || echo "✗ found dependencies that don't exist"
exit $bad

Run it on a manifest your agent wrote and it does exactly what you'd hope:

$ ./check-deps.sh requirements.txt
  ok   requests
  ok   numpy
  MISSING  requests-oauth-helper  (not on pypi — hallucinated or unregistered)
✗ found dependencies that don't exist
$ echo $?
1

It exits non-zero on a miss, so CI fails. That's the whole point — the build goes red before the hallucinated package reaches anyone, and no human had to notice.

Wire it into CI (copy-paste)

# .github/workflows/depcheck.yml
name: dep-exists
on: [pull_request]
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: every dependency must resolve on the registry
        run: bash check-deps.sh requirements.txt

Drop check-deps.sh in your repo, add that workflow, done. Every PR — human or agent — now has to prove its dependencies exist before it can merge. It runs in a couple of seconds and needs no secrets, no install, no service.

Make it a little less naive (optional)

The 20-line version answers "does it exist," which catches the pure hallucination. Two small upgrades catch the squat, where the name exists but isn't what the agent thinks:

• Flag near-zero-download packages. A real package with single-digit monthly downloads is probably not the well-known library the agent meant. For npm: curl -s https://api.npmjs.org/downloads/point/last-month/<pkg> and warn under a threshold.
• Flag near-misses to popular names. reqeusts → requests is a one-character typo of a hugely popular package; an edit-distance check against a short list of top packages catches the confusion ones.

Neither changes the shape: a lookup, a comparison, a verdict. No model, no judgment, no review.

Why this is the template, not just one check

The dependency check matters on its own, but the reason to ship it first is that it teaches the move. Look at what made it work: the failure mode (a non-existent dependency) is structural — there's an objective, machine-answerable question ("is this on the registry?") with a yes/no answer. Almost every AI-code failure worth worrying about has that shape:

• A committed secret? A regex over the config.
• An agent grant set that forms a data-exfiltration path? Classify the tools, check for all three legs.
• A skill that bundles a script it tells the agent to run? List the executables, grep the SKILL.md.
• Instruction files that have drifted apart? Diff them.

Each is the same twenty-lines-and-a-CI-step pattern: find the structural question, answer it deterministically, fail the build. You don't need a platform or a vendor for any of them — you need to notice that the thing you were going to review is actually a thing you can check, and then spend the twenty minutes once.

The verification gap closes one cheap check at a time. Here's the first one, ready to paste. Add it before lunch and the next hallucinated dependency your agent writes never makes it past the PR.

→ Talk to us if you want a fuller set of these checks wired around your agents.

---

The check-deps.sh script above is included and was run as shown (it really does exit 1 on a missing package). It's intentionally minimal — curl + the registry, no tool to install — so you can read every line before trusting it. The hallucination/slopsquat figures it defends against come from our earlier write-up and the USENIX 2025 study cited there.