test: Add CI for npm-installed CLI#5542
Draft
backspace wants to merge 2 commits into
Draft
Conversation
The boxel-cli suite only ever called command functions in-process inside the monorepo, where pnpm's nested node_modules masks how the package behaves once installed from npm. `boxel parse` shipped broken — glint silently resolved nothing under npm's hoisted node_modules layout — despite a fully green suite. Add a subprocess-driven harness whose only per-context variable is the binary under test (BOXEL_CLI_BIN): - tests/helpers/run-boxel.ts drives the CLI as a subprocess (argv + env + stdin -> stdout/stderr/exit). - tests/helpers/integration.ts seeds a JWT profile on disk so a spawned CLI authenticates via a temp HOME with no Matrix round-trip. - scripts/run-cli-suite.ts installs the target and runs a test command against it: `pnpm pack` (resolves catalog:/workspace: specifiers) then `npm install` the tarball for real hoisting, or `npm install` from the registry for the published artifact. Wire two contexts into CI: the PR gate runs the suite against the packed npm install (test:cli:tarball), and the publish workflow smoke-tests the just-published registry artifact (verify-unstable / verify-stable). Add parse fixtures covering the reported resolution gaps plus a deliberate type error that proves glint actually ran, and port smoke.test.ts onto the harness. This is the test/CI half of CS-12083. The parse fix is a follow-up, so the parse fixtures are expected to fail against the installed binary until it lands. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Rewrite the tests/integration suite to drive the installed `boxel` binary via runBoxel instead of calling command functions in-process, so the same tests exercise the CLI as installed from npm (the tarball PR gate and the published post-release smoke). - Add createTestRealmViaCli(home): create a realm through `boxel realm create` and read its URL back from the profile the CLI wrote to disk. - Each test seeds a profile on disk (createTestHome + setupTestProfile / setupJwtTestProfile), runs the command under test via runBoxel, and verifies realm/DB/checkpoint state in-process via reloadProfile(home). - Text content rides stdin; binary content is staged to a temp --file. - Commands with --json are asserted via res.json(); others via stdout (the CLI drops ANSI colors when stdout is not a TTY) and exit code. A few tests stay in-process by necessity, coexisting with the subprocess tests in the same files: white-box cases that mock/spy the internal fetch (request-body shape, injected non-2xx responses) can't cross the process boundary, and realm-watch is the long-running `watch start` daemon plus the RealmWatcher internal API, which a subprocess can't drive without weakening assertions. Not run locally (needs the realm-server + Postgres + Matrix stack); eslint clean across the suite. Expect CI iteration to settle argv and output-assertion details. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.