Skip to content

test: replace mock-heavy CLI tests with pure unit + real-I/O tests#115

Merged
ohong merged 2 commits into
mainfrom
oh-replace-mock-tests
May 5, 2026
Merged

test: replace mock-heavy CLI tests with pure unit + real-I/O tests#115
ohong merged 2 commits into
mainfrom
oh-replace-mock-tests

Conversation

@ohong

@ohong ohong commented May 4, 2026

Copy link
Copy Markdown
Owner

Stacks on top of #114. Rebase target should change to main once #114 merges.

Why

Audit of the test suite from #114's PR review:

503 of 794 tests (63%) live in files that mock something.
Tests like first-run.test.ts mock node:fs entirely — they only verify the shape of our calls, not the actual behavior. They'd pass while the real production path silently fails.

Same pattern in api.test.ts (stubbed globalThis.fetch) and ccusage-install.test.ts (mocked everything around the install decision).

What changed

Three of the worst offenders are now honest, plus one preventive extraction.

1. first-run.test.ts — real fs

Before: vi.mock("node:fs") and assertions on the call shape (e.g. "we called writeFileSync with this path").
After: mkdtempSync + real I/O. New tests assert:

  • Actual file mode (0o600 for the marker, 0o700 for the dir)
  • Real EACCES on a chmod 0o500 dir
  • mkdir -p semantics for nested config dirs
  • Round-trip between isFirstRun and markFirstRun

Required a tiny additive change: isFirstRun(configDir?) and markFirstRun(configDir?) accept an optional override (default = CONFIG_DIR). Production behavior unchanged.

2. api.test.ts — real http server

Before: vi.stubGlobal('fetch', mockFetch).
After: http.createServer in beforeAll, planned-response queue, request recording. Every test exercises the real:

  • fetch HTTP/1.1 wire format
  • Authorization header serialization (`Bearer ${token}`)
  • JSON request body
  • Real response-header reading via res.headers.get()
  • The X-Straude-Refreshed-Token rotation and 401-retry paths

Only remaining mock is auth.saveConfig (we don't want test runs writing to ~/.straude/config.json). That's a true boundary — captured-and-asserted, not faked-and-forgotten.

3. ccusage-install.test.ts — pure helper extracted

Extracted pickInstallCommand({ hasBun }): { cmd, args, manager } as a pure function. The decision logic (`bun add -g ccusage` vs `npm install -g ccusage`) is now unit-tested in pick-install-command.test.ts with zero mocks. The orchestration test in ccusage-install.test.ts is trimmed to branches that genuinely depend on process state (PATH, isatty).

4. resolvePushDateRange — pure helper extracted (preventive)

The complex date-range logic in pushCommand (six branches: `--date` / codex repair / `--days` / smart-sync / fresh install / future date / boundary) is now a pure function. New resolve-push-date-range.test.ts covers all six branches with zero mocks.

Stats

Metric Before After
CLI test count 240 265
Pure unit test files 4 7
Mocks in first-run.test.ts 1 (`node:fs`) 0
Mocks in api.test.ts 2 + global fetch stub 2 (boundary only)
Mocks in ccusage-install.test.ts 4 4 (smaller scope)

Test plan

  • `bun run --cwd packages/cli test` — 265 tests, all green
  • `bun run typecheck` — green
  • `bun run --cwd packages/cli build` — green

Out of scope (real follow-ups)

  • API-route tests in `apps/web/tests/api/` still mock the Supabase client. The honest fix is testcontainers + a real ephemeral Postgres. Larger CI infra change — separate PR.
  • `flows/cli-push-flow.test.ts` and `flows/web-import-flow.test.ts` claim to be flow tests but mock everything. Honest fix is Playwright against a real Next.js + local Supabase. Separate PR.
  • The flaky `authenticated-100ms.test.tsx > messages optimistic send` perf test (1018ms vs 1000ms budget) flagged in fix(cli): unblock activation — ccusage install, EPIPE, silent reauth, backfill filter #114 is unrelated to this work.

🤖 Generated with Claude Code

Three of the worst mock-only tests are now honest:

1. first-run.test.ts — was 100% vi.mock("node:fs"). Replaced with
   mkdtempSync against the real filesystem; exercises actual 0o600/0o700
   mode bits, real EACCES on read-only dirs, and the round-trip between
   isFirstRun + markFirstRun. Adds an optional configDir param to both
   helpers (default = CONFIG_DIR), so production behavior is unchanged.

2. api.test.ts — was vi.stubGlobal("fetch", …). Replaced with a real
   http.createServer in beforeAll; every test now exercises the real
   fetch / Authorization-header serialization / JSON body / response-
   header reading, including the X-Straude-Refreshed-Token rotation and
   the 401-retry path. The only remaining mock is auth.saveConfig
   (boundary, not faked behavior).

3. ccusage-install.test.ts — extracted the install-command decision into
   a new pure helper pickInstallCommand({ hasBun }) and unit-tested it
   with zero mocks. Trimmed the orchestration test to branches that
   genuinely depend on process state (PATH, isatty).

Also extracted resolvePushDateRange() from pushCommand as a pure
function, unit-tested across all six branches (--date / codex repair /
--days / smart-sync / no last_push_date / future date / boundary).

CLI test count: 240 → 265 (+25 pure unit tests, 7 mock-only tests retired).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented May 4, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
straude Ready Ready Preview, Comment May 5, 2026 9:02pm

Request Review

@coderabbitai

coderabbitai Bot commented May 4, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f10a55cb-2125-42b7-9f46-34b730f6fef1

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch oh-replace-mock-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ohong ohong changed the base branch from oh-cli-activation-fixes to main May 5, 2026 21:20
@ohong ohong merged commit bdeec64 into main May 5, 2026
5 checks passed
@ohong ohong deleted the oh-replace-mock-tests branch June 11, 2026 08:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant