🚢 ShipCheck

A pre-commit safety net for people who ship code written by AI agents.

Your AI agent changed something. Before you commit, ShipCheck tells you — in plain language — what changed, where it's risky, and what to test.

---

The problem

You asked Claude Code / Cursor / Lovable to "add login" or "fix the checkout bug." It edited five files, deleted one, added a package, and left a TODO. You can't read the diff. Do you hit commit?

ShipCheck reads the diff for you and answers four questions a non-engineer can act on:

1. What changed — every file the agent touched, added, renamed, or deleted.
2. Where it might be risky — secrets, deletions, DB wipes, dependency typo-squats, auth/payment code… graded 🔴 / 🟠 / 🟡 with exact file:line.
3. What to test — a concrete, do-it-yourself checklist.
4. What to ask your agent — plain-language follow-up questions.

The core checks are deterministic and 100% offline — no API key, no code leaves your machine.

Quickstart

Requires Python 3.11+ and git.

pip install -e .          # installs the `shipcheck` command
cd ~/your/project
shipcheck                 # check what you haven't committed yet

That's it. No config, no account, no key.

macOS + Python 3.14: if shipcheck says ModuleNotFoundError: No module named 'shipcheck' right after install, that's a known setuptools/3.14 quirk — run make dev (it fixes it) or see Troubleshooting.

What it looks like

ShipCheck
Scope: uncommitted changes (working tree vs last commit)

(1) What changed
    [+ new]      app/payments.py  +3 -0
    [- deleted]  legacy.py        +0 -1
    [~ modified] requirements.txt +2 -1

    3 file(s) changed, +5 / -2 lines total

(2) Where it might be risky   🔴 1 · 🟠 3 · 🟡 1
    🔴 secrets  app/payments.py:1
       Possible Stripe live secret key hardcoded in an added line.
         <Stripe live secret key redacted>
    🟠 dependencies  requirements.txt:2
       `reqeusts` looks like a typo of the popular package `requests` — possible typo-squat.

(3) What to test before you ship
    [ ] Make one test purchase from start to finish and check the amount charged is correct.
    [ ] Make sure no password or key got committed; if one did, change it and remove it.

(4) Questions to ask your AI agent
    ? You deleted `legacy.py`. Was that intended, and can it be undone if it was a mistake?
    ? You added new packages — are they the official, widely-used ones, not look-alikes?

ShipCheck is a helper, not a guarantee. When in doubt, ask a human you trust.

Usage

shipcheck                     # check uncommitted changes (the common case)
shipcheck --since main        # check everything changed since the 'main' branch
shipcheck --since HEAD~3      # ...or since 3 commits ago
shipcheck -C ~/code/my-app    # check a different project folder
shipcheck --explain           # add a plain-language "In plain words" summary
shipcheck --md > report.md    # save a shareable Markdown report
python -m shipcheck           # works without installing too

Run shipcheck --help for the full list. Secrets found in the diff are never echoed — ShipCheck reports the location and redacts the value.

Works with Cursor, Claude Code, Lovable & more

ShipCheck reads your change from one of three sources:

Your setup	Command
A git repo	shipcheck
A diff from a pipe or file	git diff | shipcheck --patch -  ·  shipcheck --patch changes.diff
An exported folder (not a repo)	shipcheck --dir ./export

Cursor / Aider — they edit your real git repo, so just run shipcheck in the project after the agent makes changes. To check before accepting them, pipe the working diff: git diff \| shipcheck --patch -.

Claude Code — same: run shipcheck after it finishes, or wire it into a Stop hook. To review only what one session touched, note the commit you started from and run shipcheck --since <that-commit>.

Lovable / v0 / bolt.new — these generate a project in the browser. Export or download it, then point ShipCheck at the folder: shipcheck --dir ~/Downloads/my-lovable-app. It treats every file as new and scans them all for secrets, stubs, and risky code.

Anything else — if you can produce a unified diff (git diff, a .patch file, or even a copy-paste), shipcheck --patch - will read it.

Try it right now on the bundled examples (one of which would drop a database):

shipcheck --patch examples/01-drops-the-database.patch

What it checks

All checks are deterministic and offline (regex + simple logic, no LLM). Each finding has a severity (🔴 high / 🟠 medium / 🟡 low), a plain-language message, and a file:line location.

Category	Catches
destructive	deleted files, DROP/TRUNCATE/DELETE FROM, DB migrations, rm -rf, rmtree / dropDatabase / FLUSHALL
secrets	API keys/tokens (AWS, GitHub, Stripe, Google…), private keys, KEY="…" credentials, committed .env files — values never echoed
sensitive_paths	payment / auth / permission / user-data paths, and outbound network calls
dependencies	newly added packages, plus typo-squat warnings (edit-distance to popular names)
scope	oversized changesets, binary files, generated / lock / minified files
stubs	TODO / FIXME, NotImplementedError / todo!(), empty function bodies

Adding a detector is adding one file → see CONTRIBUTING.md.

Plain-language explanations (--explain)

shipcheck --explain adds an "In plain words" summary grouped by what a non-engineer actually feels (money, login, what users see, risky stuff).

• No API key → an offline, deterministic heuristic summary. Always works.
• With a key → an LLM writes it. Set ANTHROPIC_API_KEY or OPENAI_API_KEY and install the SDKs: pip install "shipcheck[ai]".

The LLM is kept honest: the prompt contains only the real diff and the deterministic findings; the model must cite real file names; the output is labeled as an AI summary to cross-check against the (authoritative) risk findings.

🔒 Privacy: with a key set, --explain sends your diff to that provider. Without --explain, or with --no-llm, nothing leaves your machine. Config via SHIPCHECK_AI_PROVIDER / SHIPCHECK_AI_MODEL — see .env.example.

Shareable reports (--md)

shipcheck --md prints a clean GitHub-flavored Markdown report (tables + checkboxes) you can paste into a PR description, an issue, or a Slack message:

shipcheck --md > report.md
shipcheck --since main --explain --md | pbcopy   # straight to your clipboard

How it works

ShipCheck shells out to git diff (working tree vs the last commit, or vs --since <ref>), folds in new untracked files, parses the unified diff into added lines with real line numbers, and runs a handful of fast, local detectors. No code leaves your machine unless you opt into --explain with a key.

Development

make dev      # create .venv, editable-install with dev deps, verify
make test     # run the pytest suite
make run      # run shipcheck against the current directory

Project layout:

shipcheck/
  cli.py          # argparse entry point -> `shipcheck`
  gitutils.py     # thin wrappers around the git CLI
  diff.py         # parse git output into a structured DiffSummary
  analyze.py      # orchestrate detectors + checklist + optional narration
  checklist.py    # deterministic manual-test list + questions for the agent
  narrate.py      # optional LLM "in plain words" summary (+ heuristic fallback)
  report.py       # render the report for the terminal / as Markdown
  risks/          # deterministic risk detectors — one module per category
    base.py         destructive.py  secrets.py  sensitive_paths.py
    dependencies.py scope.py        stubs.py
tests/            # pytest suite (incl. tests/risks/ per detector)

Troubleshooting

ModuleNotFoundError: No module named 'shipcheck' after pip install -e . (macOS + Python 3.14). setuptools marks its editable .pth file as hidden, and Python 3.14's site.py skips hidden .pth files — so the install "succeeds" but can't be imported. (Doesn't affect Linux/Windows or Python ≤ 3.13.) Fix:

make dev   # does this for you
# or, manually, after installing:
chflags nohidden .venv/lib/python*/site-packages/__editable__*.pth

Roadmap

• Optional AI-written, plain-language explanations (--explain).
• Shareable Markdown reports (--md).
• --strict — exit non-zero on 🔴 findings (for git hooks / CI).
• --json output.
• More language- and framework-aware risk rules.

Contributing

PRs welcome — especially new risk detectors (it's one file). See CONTRIBUTING.md.

License

MIT