Skip to content

test: align suite with the updated testing skill guideline - #863

Merged
zgeoff merged 12 commits into
mainfrom
test/align-guideline-drift
Aug 11, 2026
Merged

test: align suite with the updated testing skill guideline#863
zgeoff merged 12 commits into
mainfrom
test/align-guideline-drift

Conversation

@zgeoff

@zgeoff zgeoff commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Description

Aligns the test suite with the updated testing skill guideline across four waves, from mechanical matcher fixes through higher-value factory/snapshot/authorisation cleanups.

  • Wave 1: toEqualtoStrictEqual, describe-free titles, dropped a duplicate test.
  • Wave 2: try/catch/finally narrowed onto toThrowWithMessage/onTestFinished; hand-rolled env restore → updateEnv (needed registerBunTestCleanup() wired into libs/data/db/test-setup.ts); ad hoc polling → shared waitFor/polled drain, retry-drain timeouts tightened to just above the configured delay.
  • Wave 3: ~15 app-web route-handler tests moved off isRedirect/instanceof Response guards onto rejects.toMatchObject/invariant; waitFor(() => getBy*)findBy*; the continue-here click now waits on the avatar query cache (the render util exposes its queryClient); seeded @msw/data instead of hand-built responses.
  • Wave 4: inlined factory-fed contract tests, regenerated replay checkpoint snapshots (verified byte-for-byte against the old literal), closed activity authorisation gaps, swapped OTel harnesses for createInMemoryMetrics, secrecy tests assert the secret is absent from the message.
  • Production fix: findCurrentSimVersion breaks a deployedAt tie on engineHash, matching find-latest-release's tiebreaker pattern.

Testing

  • bun run typecheck passes
  • bun run test passes
  • bun run lint passes
  • New tests added for new functionality

Context

Deferred to follow-up work, per the guideline-alignment plan:

  • Module-level fixture inlining (~35 files)
  • game-lib setState→setter conversions (23 sites)
  • Unique-key isolation cleanups
  • Contracts accept-side result.data sweep (57 sites)
  • libs/data/db database-creation leak redesign
  • Weak-assertion rewrites
  • Disposing the env-booted service in create-*-service.test.ts: the factories expose no db handle or dispose hook, so real teardown needs a factory API change (the email variant's queue.stop() is the one real disposal available today)
  • 3 fixed-delay setTimeout sites: create-worker-demux.test.ts:65 waits out an eviction sweep interval with no pollable signal; create-worker-runtime.test.ts:153 and start-writer-election.test.ts:77 prove an absence after a fixed window

zgeoff added 9 commits August 11, 2026 02:26
toEqual to toStrictEqual, describe-free "it " test titles, and a
duplicate test removed, per the updated testing skill.
Converts try/catch narrowing to toThrowWithMessage/toMatchObject
rejection assertions, try/finally connection teardown to
onTestFinished, hand-rolled env restore to updateEnv (registering
registerBunTestCleanup in @vers/db's preload so it takes effect),
a module-level polling helper to the shared waitFor, and two
job-queue sleep-then-drain waits to a polled drain.
Replaces isRedirect .catch() ternaries and instanceof Response
throw-guards with direct rejects.toMatchObject assertions and
invariant() narrowing; converts waitFor(() => getBy*) to findBy*
where the check is a DOM query; seeds the @msw/data store instead
of hand-crafting service responses; replaces the sole idle-worker
raw-store write with its sanctioned setter; drops a satellite-store
cleanup the preload's registerZustandReset already owns; and moves
factory tests to full-shape toStrictEqual pairs.
Replaces factory-fed contract-schema tests with inline literal
payloads and issue-path assertions, pins the checkpoint-hash golden
test to toMatchInlineSnapshot, and replaces run-replay-segment's
369-line hand-written checkpoint fixture with a regenerated inline
snapshot verified against the deleted literal's content.
Binds each service's env.DATABASE_URL boot test to its factory's own
teardown (stopTelemetry, or the job queue's stop for email) so the
booted service no longer leaks past the test. Adds the missing
cross-actor and anonymous-caller authorisation pairs for
advanceActivity and getLatestActivityProgress, and splits
trackActivityProgress's mislabeled "foreign or missing" test into
its two actual cases.
Replaces hand-rolled OTel meter-provider harnesses with the shared
createInMemoryMetrics, routes sentryHandle test writes through
setSentryHandleForTesting with capture-and-restore, and drops the
checked-in JWT keypair for getTestJWTKeyPair.

Production change: findCurrentSimVersion's deployedAt ordering gets
a createdAt tiebreaker (sim_versions has no id/sequence column, so
createdAt is the closest analog to release-registry's id
tiebreaker) plus a covering test, following the testing guideline's
timestamp-ordering rule.
bun types .rejects/.resolves chains as synchronous, so awaiting them
trips await-thenable and no-confusing-void-expression; the fixes in
prior waves awaited these to force ordering against a following
cookie/db read. Switches those sites to the documented drain-then-
assert idiom (await promise.catch(() => {}) before the unawaited
rejects assertion) and un-awaits the sites nothing depends on.
Also braces two waitFor callbacks in build-router.test.ts that
returned a void expression as an arrow shorthand.
createdAt defaults to now() pinned to transaction start, so two rows
written in one transaction tie on it too and the ordering stays
arbitrary. engineHash is sim_versions' actual primary key and gives a
genuine total order, matching the pattern in find-latest-release. The
tie test now asserts the winner by engineHash instead of hand-setting
createdAt.
Removing the hand-rolled otel harnesses left sdk-metrics with no
importer in product-analytics and service-activity, and knip flags
both as unused. Both packages already use createInMemoryMetrics from
test-utils instead.
@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 09cdd8cf-4e54-48c9-968a-ebd958e3a46c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The pull request modernizes tests across the web app, contracts, libraries, and services. It replaces manual asynchronous handling with direct rejection and query assertions, uses persisted and shared fixtures, adds authorization coverage, and strengthens replay snapshots and factory checks.

Changes

Web test synchronization and redirect handling

Layer / File(s) Summary
Route redirect and async assertions
apps/web/src/routes/...
Route tests assert rejected redirect promises directly and wait for side effects before reading cookies or database state. Form and activity tests use asynchronous Testing Library queries.
Client fixtures and UI utilities
apps/web/src/lib/..., apps/web/src/test-utils/...
Client tests create persisted sessions and users. Activity rewards use persisted avatar data. The render result exposes its QueryClient, and UI tests synchronize with query state.

Contracts, services, and shared test infrastructure

Layer / File(s) Summary
Contract and database assertions
contracts/..., libs/data/db/...
Contract tests use explicit payloads and boundary fixtures. Database tests use strict equality and shared environment and cleanup helpers.
Service authorization and metrics coverage
services/activity/..., services/email/..., services/avatar/..., services/keys/...
Activity tests add cross-owner and anonymous access cases. Metrics, email delivery, key validation, and service cleanup tests use shared utilities and direct assertions.
Shared test utility adoption
libs/service/..., libs/testing/..., scripts/...
Tests replace fixed delays, manual cleanup, embedded JWT keys, direct Sentry mutation, and manual environment changes with shared polling and lifecycle utilities.

Replay test coverage

Layer / File(s) Summary
Replay snapshots and factory contracts
services/replay/...
Replay tests use complete inline snapshots. Factory tests assert complete default objects and explicit overrides. Replay iteration descriptions were renamed without logic changes.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

  • zgeoff/vers#860: Covers the testing practices applied across this refactor, including direct redirect assertions and asynchronous query usage.
  • zgeoff/vers#591: Relates to shared metrics infrastructure and removal of manual OpenTelemetry test setup.
  • zgeoff/vers#568: Relates directly to the replay worker tests and replay factory coverage updated here.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the primary change: aligning the test suite with updated testing guidelines.
Description check ✅ Passed The description directly explains the testing guideline updates, production fix, validation results, and deferred work.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/align-guideline-drift

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 75 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread contracts/email/src/email-contract.test.ts
Comment thread services/activity/src/handlers/get-latest-activity-progress.test.ts
zgeoff added 2 commits August 11, 2026 04:48
…tion

Remove the no-op stopTelemetry disposals (the factories expose no real
teardown), restore regex-style messages with secret-absence assertions in
the keys parsers, gate the continue-here click on the avatar query cache
via the render util's exposed queryClient, drop a redundant findByRole,
tighten retry-drain timeouts, and align replay factory test titles.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
libs/testing/client-test-utils/src/orpc/build-contract-mock.test.ts (1)

99-116: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Assert the error code so the test matches its title.

The title claims the rejection surfaces as a defined ORPCError. The assertion now checks only data.reason, so any rejection value that carries that data shape passes. Include the code in the matched object to keep the defined-error contract covered.

🧪 Proposed assertion
-  expect(client.getSecret({})).rejects.toMatchObject({ data: { reason: 'missing-session' } });
+  expect(client.getSecret({})).rejects.toMatchObject({
+    code: 'UNAUTHORIZED',
+    data: { reason: 'missing-session' },
+  });

Replace UNAUTHORIZED with the code declared for this procedure in secretContract.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@libs/testing/client-test-utils/src/orpc/build-contract-mock.test.ts` around
lines 99 - 116, Update the rejection assertion in the mock-thrown typed-error
test to also match the error code declared for getSecret in secretContract,
replacing the hardcoded UNAUTHORIZED expectation with that contract-defined code
while preserving the existing data.reason check.</code>

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/web/src/routes/-game/satellite-stack.test.tsx`:
- Around line 1-3: Restore test cleanup in the satellite registration setup by
importing and using onTestFinished with removeSatellite for the avatar-viewer
registration in satellite-stack.test.tsx. Ensure cleanup runs after each test so
registered satellite state cannot leak or conflict with later tests.

In `@contracts/avatar/src/avatar-data-schema.test.ts`:
- Around line 6-34: Replace the inline valid AvatarData payloads in the affected
tests with the package’s faker-defaulted create-mock-* factory, overriding mode
to trade and self_found respectively. Preserve the existing assertions and keep
the invalid hardcore payload inline.

In `@services/replay/src/dispatch/run-replay-segment.test.ts`:
- Around line 533-909: Consolidate the replay-output assertions in the relevant
test into one inline snapshot golden record. Capture the shared expected output
once, then have both the remote-dispatch and traceparent result assertions
compare against that record, preserving the parity checks without duplicating
the full snapshot.

---

Outside diff comments:
In `@libs/testing/client-test-utils/src/orpc/build-contract-mock.test.ts`:
- Around line 99-116: Update the rejection assertion in the mock-thrown
typed-error test to also match the error code declared for getSecret in
secretContract, replacing the hardcoded UNAUTHORIZED expectation with that
contract-defined code while preserving the existing data.reason check.</code>
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 836e2aae-156d-4004-a48d-676b4633fdc0

📥 Commits

Reviewing files that changed from the base of the PR and between acbbad0 and 853da84.

⛔ Files ignored due to path filters (1)
  • bun.lock is excluded by !**/*.lock, !bun.lock
📒 Files selected for processing (70)
  • apps/web/src/lib/activity/use-activity-rewards.test.tsx
  • apps/web/src/lib/auth/run-logout.test.ts
  • apps/web/src/lib/avatar/require-active-avatar.test.ts
  • apps/web/src/lib/forms/use-form-submit.test.tsx
  • apps/web/src/lib/rpc/clients/session-existence-client.test.ts
  • apps/web/src/lib/rpc/clients/session-refresh-client.test.ts
  • apps/web/src/lib/rpc/clients/user-client.test.ts
  • apps/web/src/routes/-account-2fa-verify/load-two-factor-setup.test.ts
  • apps/web/src/routes/-account-2fa-verify/verify-two-factor-setup-handler.test.ts
  • apps/web/src/routes/-account-change-email/run-change-email.test.ts
  • apps/web/src/routes/-account-change-password/run-change-password.test.ts
  • apps/web/src/routes/-account/run-disable-two-factor-auth.test.ts
  • apps/web/src/routes/-activity/activity-panel.test.tsx
  • apps/web/src/routes/-avatar-create/run-avatar-create.test.ts
  • apps/web/src/routes/-forgot-password/forgot-password-form.test.tsx
  • apps/web/src/routes/-forgot-password/run-forgot-password.test.ts
  • apps/web/src/routes/-game/nav-rail.test.tsx
  • apps/web/src/routes/-game/playing-elsewhere-notice.test.tsx
  • apps/web/src/routes/-game/satellite-stack.test.tsx
  • apps/web/src/routes/-login-force-logout/run-force-logout.test.ts
  • apps/web/src/routes/-login/login-form.test.tsx
  • apps/web/src/routes/-login/run-login.test.ts
  • apps/web/src/routes/-onboarding/onboarding-form.test.tsx
  • apps/web/src/routes/-onboarding/run-onboarding.test.ts
  • apps/web/src/routes/-reset-password/reset-password-form.test.tsx
  • apps/web/src/routes/-reset-password/reset-password-handler.test.ts
  • apps/web/src/routes/-signup/run-signup.test.ts
  • apps/web/src/routes/-signup/signup-form.test.tsx
  • apps/web/src/routes/-verify-otp/verify-otp-form.test.tsx
  • apps/web/src/routes/-verify-otp/verify-otp-handler.test.ts
  • apps/web/src/test-utils/factories/create-mock-avatar.test.ts
  • apps/web/src/test-utils/factories/create-mock-user.test.ts
  • apps/web/src/test-utils/render.tsx
  • contracts/activity/src/activity-contract.test.ts
  • contracts/activity/src/build-checkpoint-hash.test.ts
  • contracts/avatar/src/avatar-data-schema.test.ts
  • contracts/email/src/email-contract.test.ts
  • libs/data/db/package.json
  • libs/data/db/src/create-db.test.ts
  • libs/data/db/src/test-support/resolve-test-db-target.test.ts
  • libs/data/db/test-setup.ts
  • libs/data/sim-registry/src/find-current-sim-version.test.ts
  • libs/data/sim-registry/src/find-current-sim-version.ts
  • libs/service/jobs/src/create-job-queue.test.ts
  • libs/service/product-analytics/package.json
  • libs/service/product-analytics/src/metrics/record-delivery-failure.test.ts
  • libs/service/service-runtime/src/create-service.test.ts
  • libs/service/service-runtime/src/report-unexpected-error.test.ts
  • libs/service/service-runtime/src/start-error-reporting.test.ts
  • libs/service/service-utils/src/utils/create-token-verifier.test.ts
  • libs/testing/client-test-utils/src/orpc/build-contract-mock.test.ts
  • libs/testing/service-test-utils/src/bun/strategies/create-schema-test-db.test.ts
  • scripts/src/utils/require-env-var.test.ts
  • services/activity/package.json
  • services/activity/src/handlers/advance-activity.test.ts
  • services/activity/src/handlers/get-latest-activity-progress.test.ts
  • services/activity/src/handlers/track-activity-progress.test.ts
  • services/activity/src/metrics/record-terminal-transition.test.ts
  • services/activity/src/test-utils/factories/create-mock-checkpoint-batch.test.ts
  • services/avatar/src/handlers/get-avatars.test.ts
  • services/email/src/build-router.test.ts
  • services/email/src/create-email-service.test.ts
  • services/keys/src/parse-roll-key-roots.test.ts
  • services/keys/src/parse-scope-secret-roots.test.ts
  • services/replay/src/dispatch/run-replay-segment.test.ts
  • services/replay/src/test-utils/factories/create-mock-activity-row.test.ts
  • services/replay/src/test-utils/factories/create-mock-chain-row.test.ts
  • services/replay/src/test-utils/factories/create-mock-encounter-node.test.ts
  • services/replay/src/test-utils/factories/create-mock-replay-segment.test.ts
  • services/replay/src/worker/run-replay-iteration.test.ts
💤 Files with no reviewable changes (3)
  • services/activity/package.json
  • libs/service/product-analytics/package.json
  • services/activity/src/test-utils/factories/create-mock-checkpoint-batch.test.ts

Comment thread apps/web/src/routes/-game/satellite-stack.test.tsx
Comment thread contracts/avatar/src/avatar-data-schema.test.ts
Comment thread services/replay/src/dispatch/run-replay-segment.test.ts Outdated
…oldens

Seed the owner's activity so the ownership filter is what rejects, and
replace two duplicated replay snapshots with an in-test parity comparison
and an outcome-kind assertion.
@zgeoff
zgeoff merged commit e17d60f into main Aug 11, 2026
6 checks passed
@zgeoff
zgeoff deleted the test/align-guideline-drift branch August 11, 2026 00:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant