feat(cli): modular CLI with SQL query support over all published tables

[ekim1394]

Summary

Turns the single-file spicy-regs CLI into a modular package that can download and query the full published dataset, structured so outside contributors can add a command by copying one template module and registering it in one place.

New commands (all backed by a shared DuckDB engine in src/spicy_regs/cli/engine.py):

• spicy-regs query "SQL" — run SQL over every published table. --source auto (default) reads each table from your local download when present and streams from the public R2 bucket otherwise, so it works with zero setup. --format table|json|csv, --max-rows, --output FILE.
• spicy-regs tables / spicy-regs describe TABLE — discover tables and column schemas.

Upgraded commands:

• download — now covers all 18 published tables + the manifest (--tables, --all), streams with a tqdm progress bar, writes atomically (.tmp + rename), and skips files whose size already matches the bucket (previously a stale file was silently kept). --types is kept as a hidden back-compat alias.
• stats / sample / search / agencies — rebuilt on the shared engine with the same flags, so they now also work directly against R2 without downloading first, and search pushes filtering down to DuckDB instead of loading whole files into memory.

Extensibility: each subcommand is one module implementing register(subparsers) + run(args); cli/_registry.py::COMMANDS is the single line to touch when adding one. The table list is imported from data_dictionary.TABLES (single owner, already sync-tested against the MCP server). A "Add a CLI command" recipe was added to CONTRIBUTING.md.

Notes:

• src/spicy_regs/cli.py (module) is replaced by src/spicy_regs/cli/ (package); the spicy-regs entry point string in pyproject.toml is unchanged. No in-repo code imported the old module.
• mcp_server.py is deliberately untouched (Vercel sync-copy test); the engine mirrors its view-per-table + graceful-skip + jsonify patterns instead of sharing code with it.
• A possible follow-up: point the Claude plugin skill's standalone query script at spicy-regs query --format json.

Test plan

• uv run pytest — 401 passed (43 new CLI tests: parser round-trips, engine view resolution incl. partitioned comments, atomic/skip/404 download behavior via httpx.MockTransport, end-to-end command output in table/json/csv)
• uv run ruff check . and uv run ty check — clean
• uv run pytest -m integration -k cli — live R2 describe passes
• Manually exercised against the live bucket: download --tables agency_stats twice (second run skips as up-to-date), describe feed_summary --source r2, query "SELECT agency_code, docket_count FROM agency_stats ORDER BY docket_count DESC LIMIT 3", plus tables, stats, sample, search, agencies against local fixtures
• uv run spicy-regs-dict check — data-dictionary gate still green

Checklist

• My change is scoped to one concern
• I added or updated tests for new behavior
• I updated docs (README / CONTRIBUTING / module READMEs) if relevant
• CI is green

🤖 Generated with Claude Code

https://claude.ai/code/session_01NYT7oKodm42fs1RNmafM4f

---

Generated by Claude Code