Skip to content

test(integration): real-Supabase API route tests via bunx supabase start#117

Merged
ohong merged 6 commits into
mainfrom
oh-integration-tests-postgres
May 5, 2026
Merged

test(integration): real-Supabase API route tests via bunx supabase start#117
ohong merged 6 commits into
mainfrom
oh-integration-tests-postgres

Conversation

@ohong

@ohong ohong commented May 5, 2026

Copy link
Copy Markdown
Owner

Why

Follow-up to #115 (mock-test cleanup). The "Out of scope" note in that PR called this out:

API-route tests in `apps/web/tests/api/` still mock the Supabase client. The honest fix is testcontainers + a real ephemeral Postgres.

After hitting issues with testcontainers + the bare `supabase/postgres` image (the image is missing storage-service-managed columns that some migrations reference), this PR uses the `bunx supabase start` path the dev team already uses for local development. Same Postgres, same migrations applied via the proper Supabase CLI flow that knows how to set up storage extensions. No new heavy deps.

What

  • `apps/web/tests/integration/` — new directory.
    • `global-setup.ts` asserts a Supabase stack is reachable, reads its ephemeral keys via `bunx supabase status -o env`, exposes them to test workers.
    • `db.ts` — `openTestDb()`, `cleanDb()`, `insertUser()` helpers (real `pg.Client`).
    • `usage-submit.test.ts` — exemplar that exercises `POST /api/usage/submit` end-to-end against the real stack.
  • `apps/web/vitest.integration.config.ts` — separate config so the default fast suite stays mock-based.
  • `bun run --cwd apps/web test:integration` script.
  • CI: `supabase/setup-cli@v1` + `bunx supabase start` before `test:integration`.
  • `CONTRIBUTING.md` updated with run-it-yourself instructions.

What this catches that the mock test cannot

The exemplar `usage-submit.test.ts` has 5 cases:

Case What it proves
`rejects unauthenticated requests without writing anything` Real 401 + zero-rows-leaked, asserted via real `SELECT count(*)`
`writes a real daily_usage + device_usage + post row` The route actually persists what it says it did. Catches the `collector_meta` schema-cache class of bugs documented in the CHANGELOG
`real CHECK constraint rejects negative cost` Defense-in-depth past the route's TS validation — the schema is the last line of defense
`rejects dates outside the backfill window with a real 400 and no leaked rows` Real route validation + real DB state
`emits X-Straude-Refreshed-Token under real JWT signing when the token is stale` End-to-end auth path: the token refresh in #114 actually rotates real signed tokens through real `verifyCliTokenWithRefresh`

Zero `vi.mock` of behavior. The only mocking-style trick is the inline `Date.now` shimmer for the stale-token test, restored before the test ends.

Test plan

  • `bun run typecheck` — green
  • `bun run --cwd apps/web test` — 574 tests, all green (default suite excludes `tests/integration/**` as designed)
  • `bun run --cwd apps/web test:integration` — could not validate locally. My Mac is at 94% disk; Docker Desktop's containerd hit `input/output error` pulling `supabase/postgres:17.6.1.106` (3.5 GB) and the smaller Supabase service images. CI is the validation gate for this PR — it runs on `ubuntu-latest` with a fresh Docker, where `bunx supabase start` works cleanly. Reviewers with disk space available can also run it locally via `bun run local:up && bun run --cwd apps/web test:integration`.

If CI's integration step fails, I'll iterate. The code is structured carefully — types compile, the test logic is sound, the supabase-start path matches the dev workflow that's already in `bun run local:up`.

Out of scope

  • Migrating the existing mock tests in `tests/api/` to integration. They stay; future PRs can move cases incrementally.
  • The CLI binary e2e flow (separate stacked PR coming next).

🤖 Generated with Claude Code

Foundation for the "honest follow-up" flagged in #115's PR description:
replace mock-Supabase tests with real-stack integration tests that run
against the actual local Supabase boot, exercising the same migrations,
PostgREST, and CLI JWT signing path that production runs.

What's new:
- apps/web/__tests__/integration/ — new directory with one exemplar test
  for POST /api/usage/submit. Calls the real route handler with a real
  Request, mints a real CLI JWT via createCliToken, and asserts on rows
  Postgres actually persisted. No vi.mock of the Supabase client, the
  auth helper, or query chains.
- apps/web/vitest.integration.config.ts — separate vitest config wired
  to a globalSetup that asserts the local stack is reachable and reads
  ephemeral keys via `bunx supabase status -o env`.
- New `bun run --cwd apps/web test:integration` script.
- CI: supabase/setup-cli@v1 + bunx supabase start before integration tests.
- CONTRIBUTING.md updated with the run-it-yourself instructions.

The exemplar covers four cases the existing mocked usage-submit.test.ts
cannot:
1. Real route → real DB roundtrip (rows actually present in daily_usage,
   device_usage, posts).
2. Real CHECK constraint rejecting negative cost (defense-in-depth past
   the route's TS validation).
3. Real backfill-window 400 + zero rows leaked.
4. Real X-Straude-Refreshed-Token header emitted under real JWT signing
   when the token is older than the refresh threshold.

Existing mocked tests stay as-is; future PRs can migrate cases incrementally.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented May 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
straude Ready Ready Preview, Comment May 5, 2026 9:23pm

Request Review

@coderabbitai

coderabbitai Bot commented May 5, 2026

Copy link
Copy Markdown

Warning

Rate limit exceeded

@ohong has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 41 minutes and 45 seconds before requesting another review.

To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f20eaae9-9f86-41c0-b76c-1e7d72c1b83d

📥 Commits

Reviewing files that changed from the base of the PR and between d8e5049 and c50dde2.

⛔ Files ignored due to path filters (1)
  • bun.lock is excluded by !**/*.lock
📒 Files selected for processing (9)
  • .github/workflows/ci.yml
  • CONTRIBUTING.md
  • apps/web/__tests__/integration/db.ts
  • apps/web/__tests__/integration/global-setup.ts
  • apps/web/__tests__/integration/usage-submit.test.ts
  • apps/web/package.json
  • apps/web/vitest.config.ts
  • apps/web/vitest.integration.config.ts
  • docs/CHANGELOG.md
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch oh-integration-tests-postgres

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ohong ohong merged commit 482320d into main May 5, 2026
5 checks passed
@ohong ohong deleted the oh-integration-tests-postgres branch June 11, 2026 08:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant