feat(server)!: jspm.io direct vendor (Rails-style no-build)#89
Open
vivek7405 wants to merge 61 commits into
Open
feat(server)!: jspm.io direct vendor (Rails-style no-build)#89vivek7405 wants to merge 61 commits into
vivek7405 wants to merge 61 commits into
Conversation
…s and false positives
The vendor scanner picked up server-only imports from contexts that
never reach the browser, generating spurious vendor pipeline work
on packages that can't be browser-bundled.
Four tightenings:
1. route.{ts,js,mjs,mts} and middleware.{ts,js,mjs,mts}: server-
only by file-router convention. New isServerOnlyFile() helper
joins these to the existing .server.{ts,js,mjs,mts} suffix
check. Imports of @prisma/client, ws, etc. in these files no
longer enter the vendor pipeline.
2. test/ and tests/ directories: tests are server-only by webjs
convention. Their imports of test frameworks and DB clients
shouldn't generate browser vendor entries.
3. `import type X from 'pkg'` statements: TypeScript type-only
imports are erased at compile time, never reach the runtime.
The IMPORT_RE has a (?!type\s) negative lookahead. Catches
real-world false positive in api/chat/route.ts which imports
'ws' as type-only.
4. Imports inside /* … */ block comments and // line comments:
JSDoc examples (cn.ts in @webjsdev/ui, etc.) frequently show
'import x from clsx' in a code block. The scanner now strips
comments before pattern-matching.
Four new tests:
scanBareImports: skips route.ts and middleware.ts
scanBareImports: skips test/ and tests/ directories
scanBareImports: skips import type statements
scanBareImports: skips import strings inside comments
These bug fixes were salvaged from the closed PR #87
(feat/no-build-vendor-drops-esbuild). Re-applied directly on top
of main's esbuild-on-demand vendor.js instead of cherry-picking,
because the closed branch had drifted too far for a clean apply.
1155 tests pass.
Source-level companion to the existing erasable-typescript-only
tsconfig-flag rule. Scans every .ts / .mts file under the app for
the four constructs the framework's type-stripper rejects at
request time:
- enum declarations (any of enum / const enum / declare enum,
with uppercase first letter to avoid matching variables
literally named 'enum')
- namespace blocks containing value statements (let/const/var/
function/class). Type-only namespaces, which ARE erasable,
are intentionally allowed
- constructor parameter properties (public/private/protected/
readonly modifier directly before a parameter name)
- import = require (TypeScript-style CommonJS import)
Each violation reports file, line number, the construct name, and
a concrete fix. Skips node_modules, dist, build, .next, .git, and
any folder starting with underscore (the framework's _private
convention).
Why both rules ship enabled by default:
erasable-typescript-only catches the tsconfig case (flag missing
or off). It's the early-warning path; if the flag is set, the
TypeScript compiler flags violations in your editor before they
reach the runtime.
no-non-erasable-typescript catches the source case (offending
syntax that slipped past tsconfig, or files written before the
flag was added, or third-party packages that publish raw .ts).
It's the late-warning path; runs at commit time via
webjs check.
Together, an app gets two independent defenses for the same class
of violations. Either alone is incomplete: the tsconfig flag does
nothing if the user disables it; a source scan alone doesn't help
during editing.
Six new tests: one positive case per construct (asserts the rule
flags it), one negative case (clean erasable .ts file passes),
and one scope test (node_modules and _private folders are
correctly skipped).
All 1164 tests pass.
…Rails-style no-build)
Replaces main's Vite-style optimizeDeps esbuild-on-demand vendor
pipeline with the Rails 7 + importmap-rails posture: bare-specifier
npm imports resolve via importmap to jspm.io CDN URLs and the
browser fetches the bundle directly from jspm.io. The webjs server
does not proxy, cache, or bundle vendor packages.
This is the strictest "no build" architecture for a no-build
framework. Nothing bundles on the user's machine, ever. esbuild
leaves @webjsdev/server's dependencies entirely.
vendor.js rewrite (259 -> 310 lines, but most of that is doc):
- scanBareImports (kept): walks user source for bare imports
- extractPackageName, isServerOnlyFile, stripComments (kept):
same precise scanning rules
- resolvePackageDir (new): walks require.resolve's path back to
the package root, handles npm workspace hoisting
- getPackageVersion (new): reads node_modules/<pkg>/package.json
version field
- jspmGenerate (new): POSTs an install list to api.jspm.io/generate
with provider=jspm.io, env=[browser,production,module], returns
the resolved importmap fragment. In-memory cached by sorted
install-list key. 10s timeout with AbortController. Logs (does
not throw) on API failure so server boot still succeeds; vendor-
importing pages get "unresolved bare specifier" errors in the
browser until api.jspm.io is reachable.
- vendorImportMapEntries (rewritten, now async): scans bare imports,
resolves versions from node_modules, calls jspmGenerate, returns
the importmap fragment
- clearVendorCache (kept): drops the jspmCache so file-watcher
rebuilds re-resolve URLs
REMOVED entirely:
- bundlePackage (was the esbuild bundler call)
- serveVendorBundle (was the /__webjs/vendor/* response handler)
- vendorCache (was the in-memory esbuild-output cache)
- VENDOR_CACHE_MAX constant
- import { build } from 'esbuild' (no longer needed at runtime)
dev.js:
- removed /__webjs/vendor/* URL handler (browser bypasses webjs
server entirely for vendor URLs; goes straight to jspm.io)
- vendorImportMapEntries call sites now await the async result
- import list cleaned up
index.js exports:
- removed bundlePackage, serveVendorBundle
- added getPackageVersion, jspmGenerate
Tests:
- dropped tests for removed APIs (bundlePackage, serveVendorBundle)
- new tests for getPackageVersion (resolution + null fallback)
- new tests for jspmGenerate (empty input, real call, cache hit,
order-independent cache key); network-gated via
WEBJS_SKIP_NETWORK_TESTS env
- vendorImportMapEntries tests use the async signature
- dev-handler /__webjs/vendor/* test replaced with one asserting
the path 404s (no local handler)
- 1160/1160 pass
Why jspm.io over esm.sh:
- Years of uptime track record; esm.sh has had documented
downtimes and maintenance windows
- Institutional backing: 37signals (Silver), CacheFly (CDN
infrastructure sponsor), Socket, Framer (Bronze). Rails ecosystem
dependency creates downstream pressure for continued operation
- status.jspm.io for incident transparency
- Standards-first maintenance by Guy Bedford (TC39 contributor
on ESM, import maps, HTML spec)
- Matches Rails 7 + importmap-rails default exactly
Why the JSPM Generator API rather than naive URL construction:
- jspm.io's bare-package URL (https://ga.jspm.io/npm:dayjs@1.11.13)
returns text/plain metadata, not JavaScript. Browser execution
would fail with SyntaxError
- The correct entry path (e.g., /dayjs.min.js) varies per package
and must be resolved by the Generator
- Same call importmap-rails makes at pin time; webjs makes it at
server boot
What 'no build' means in this PR:
- User runs no build command
- User writes no build config
- User's source IS the deploy artifact (.ts files served via
Node's stripTypeScriptTypes)
- No bundler invocation on user's machine, ever
- No esbuild in framework deps
Breaking changes:
- @webjsdev/server.bundlePackage removed
- @webjsdev/server.serveVendorBundle removed
- /__webjs/vendor/* URL paths no longer handled by the server
- Apps now require api.jspm.io reachability at server boot to
populate the vendor importmap
Pairs with the scanner improvements (commit fcf2692) and the
no-non-erasable-typescript lint rule (commit 83e77a9) already
cherry-picked onto this branch. Together they constitute the
Rails-aligned no-build vendor architecture.
…stence
Adds `webjs vendor pin`, `webjs vendor unpin`, `webjs vendor list`
plus the runtime layer that reads the committed pin file in
preference to a live api.jspm.io call. Matches Rails' importmap-rails
pin workflow exactly, including --download for offline bundle
vendoring.
Layered on top of the existing jspmGenerate / vendorImportMapEntries
machinery; doesn't replace it. The runtime preference order is:
1. Read .webjs/vendor/importmap.json (committed pin file)
2. Fall back to live api.jspm.io/generate (if no pin file)
So apps that never run pin still work (boot-time API call, in-memory
result, same as before). Apps that run pin commit a small JSON config
and shed the boot-time dep on api.jspm.io.
Two modes mirror Rails:
Default: importmap.json holds resolved jspm.io CDN URLs. Browser
fetches direct from ga.jspm.io. Only importmap.json is
committed (a few KB).
--download: also downloads each bundle from jspm.io to
.webjs/vendor/<pkg>@<version>.js. importmap.json holds
local /__webjs/vendor/ paths. Server handler serves the
bundles from disk. Both importmap.json and bundle files
are committed (offline-capable production, CSP-friendly,
audit-friendly).
Auto-prune handles three orphan scenarios uniformly:
Update dayjs@1.x.js bundle removed when version bumps to 2.x
Delete bundle + importmap entry removed when import is dropped
Mode swap default <-> --download cleans up the other mode's files
Pin is idempotent with respect to the current source tree. Run twice
in a row with no source change = no-op. Switch modes = clean directory.
Three CLI commands matching Rails' pattern:
webjs vendor pin [--download] auto-discovers, resolves, writes
webjs vendor unpin <pkg> removes one entry + bundle if any
webjs vendor list shows pinned packages with sizes
Skipped Rails commands and the rationale:
pristine subsumed by 'pin --download' (always overwrites)
json 'cat .webjs/vendor/importmap.json' already works
outdated 'npm outdated' covers installed versions
update 'npm install pkg@latest && webjs vendor pin' is the flow
audit 'npm audit' covers most cases; vulnerability-data
integration is its own project
Pin is intentionally manual (no predev/prestart auto-run). Auto-pin
would cause silent churn in the committed importmap.json file as
jspm.io re-resolves entries or transitive deps drift. Rails takes
the same posture: bin/importmap pin is always developer-invoked.
dev.js: vendor URL handler restored for --download mode (serves
files from .webjs/vendor/). In default mode the handler still 404s
but the browser never requests these URLs (importmap routes direct
to jspm.io). resolveVendorImports replaces vendorImportMapEntries
at the call site, layering pin-file preference over the live API.
vendor.js new exports: pinAll, unpinPackage, listPinned,
readPinFile, resolveVendorImports, serveDownloadedBundle. Existing
exports (jspmGenerate, vendorImportMapEntries, etc.) unchanged.
index.js re-exports all of the above from the server's main entry
so the CLI can use them.
Tests for the new pin layer come in the next commit (this one is
the implementation; tests stay green via the existing API surface
which is unchanged for callers that don't touch the new functions).
1155/1155 tests pass on this commit.
…st, serve, prune) 12 new tests, network-gated where they hit api.jspm.io: - pinAll default: writes importmap.json with jspm.io URLs - pinAll --download: writes importmap.json + bundle files locally - pinAll prune: removes orphan bundle files from prior pins - pinAll mode switch (--download to default): removes leftover bundles - unpinPackage: removes entry from importmap.json - unpinPackage: returns removed:false for non-existent package - listPinned: parses jspm.io URLs and extracts versions - listPinned: returns empty array when no pin file - resolveVendorImports: prefers pin file over live API call - serveDownloadedBundle: rejects path-traversal filenames (../, /, .., non-js) - serveDownloadedBundle: serves real file from .webjs/vendor/ - serveDownloadedBundle: missing file returns 404 makeTempAppWithSource() helper creates an isolated tmp app dir with symlinked node_modules so getPackageVersion / pinAll's createRequire chain finds installed packages. 1160 to 1172 tests pass.
…commands README.md item 'DX' previously claimed esbuild bundled vendor packages (Vite-style optimizeDeps). After the PR #89 architectural change, vendor packages resolve through importmap to jspm.io URLs at runtime; webjs's server doesn't bundle them. Updated to describe: - jspm.io as the vendor resolution mechanism - webjs vendor pin / unpin / list commands - --download mode for offline-capable production - .webjs/vendor/importmap.json as the committed config artifact AGENTS.md invariant 10 (TypeScript must be erasable) now mentions both lint rules: - erasable-typescript-only (existing): checks the tsconfig flag - no-non-erasable-typescript (new in this PR): scans source for the four offending patterns even if the flag is unset esbuild's TS-strip fallback documentation stays accurate; only the vendor-pipeline esbuild claim was wrong post-jspm.io.
Companion to the previous commit. README.md's 'DX' bullet previously claimed esbuild bundled vendor packages (Vite-style optimizeDeps). After PR #89's architectural change, vendor packages resolve through importmap to jspm.io URLs at runtime; webjs's server doesn't bundle them. Updated to describe: - jspm.io as the vendor resolution mechanism - webjs vendor pin / unpin / list commands - --download mode for offline-capable production - .webjs/vendor/importmap.json as the committed config artifact - new no-non-erasable-typescript lint rule esbuild's TypeScript-strip fallback documentation stays accurate; only the vendor-pipeline esbuild claim was wrong.
packages/server/README.md and AGENTS.md previously described vendor as 'Vite-style optimizeDeps backed by esbuild'. After PR #89, vendor resolves through jspm.io at runtime; the server doesn't bundle. Updated to: - README.md vendor bullet: jspm.io resolution, pin commands, --download mode, .webjs/vendor/importmap.json - AGENTS.md vendor.js row: jspm.io flow + pin file preference + --download local-bundle serving - AGENTS.md check.js row: mention no-non-erasable-typescript alongside no-json-data-files
…eployment pages docs/app/docs/no-build/page.ts had the most outdated content: the entire 'Bare specifiers' + 'Why auto-bundle' sections were built around the old Vite-style optimizeDeps esbuild pipeline. Replaced with the jspm.io direct architecture description plus a new section on `webjs vendor pin` (default + --download modes). Also updated: - Importmap example (URLs now show jspm.io shape with @Version) - Cache-invalidation section (versioned URLs explanation) - Dev vs prod table (vendor resolution row) - deployment/page.ts vendor URL paragraph (URL pattern + jspm.io cache headers + --download bundle headers) Kept the no-build philosophy framing intact; just updated the mechanism (jspm.io CDN-direct instead of local esbuild-on-demand). Rails alignment is more explicit now, since the new architecture matches importmap-rails posture exactly. Pages still pending update next commit: docs/app/docs/typescript/page.ts (only TS-fallback mention; accurate) docs/app/docs/getting-started/page.ts (TS-fallback mention; accurate)
…io architecture ETags and Cache Headers section described `/__webjs/vendor/<pkg>.js` URLs with hash-based immutable caching. After PR #89, vendor URLs are either jspm.io URLs directly (default mode) or local `/__webjs/vendor/<pkg>@<version>.js` paths (after `webjs vendor pin --download`). Updated the paragraph to describe both modes and their cache header sources.
Adds 7 new end-to-end tests that spawn the actual webjs CLI binary against a temp app directory and exercise the full pipeline: - list with no pin file → reports 'No pin file' - pin → writes .webjs/vendor/importmap.json with jspm.io URLs - list with pin file → shows pinned packages + URLs - unpin <pkg> → removes entry from importmap.json - unpin <not-pinned> → reports 'not in pin file' - pin --download → writes bundle files alongside importmap.json - unknown subcommand → exits 1 with usage message These complement the existing per-function unit tests in packages/server/test/vendor/vendor.test.js by verifying the CLI surface: argument parsing, stdout shape, exit codes, --download flag handling. Network-gated where they hit api.jspm.io (4 of 7 tests). Skip via WEBJS_SKIP_NETWORK_TESTS=1 in air-gapped CI. Test file lives at test/vendor-cli/vendor-cli.test.mjs (new directory; consistent with the test/<feature> layout convention already used by test/serialization/, test/scaffolds/, etc.).
Header comment still described the old 'Vite-style optimizeDeps'
mental model. Updated to describe the actual flow:
- vendor entries come from resolveVendorImports
- reads committed .webjs/vendor/importmap.json if present
- else calls api.jspm.io/generate once at boot
- browser fetches direct from jspm.io (default) or local
/__webjs/vendor/ paths (after webjs vendor pin --download)
Cosmetic doc cleanup; behavior unchanged.
vivek7405
force-pushed
the
feat/jspm-direct-vendor
branch
from
3757ebb to
245ebe7
Compare
…peline scanBareImports now preserves the full specifier instead of dropping to the root package name. vendorImportMapEntries + pinAll splice the version into the specifier (pkg@version/subpath) before calling jspm.io's Generator API, which resolves each subpath via the package's exports field. For --download mode, bundleFilenameWithSubpath encodes the filesystem-safe filename: 'dayjs', '1.11.13', '/plugin/utc' becomes 'dayjs@1.11.13__plugin__utc.js'. The __ separator stays reversible. End-to-end verified against api.jspm.io: scanner finds the subpath, generator resolves it correctly, importmap emits the right entry. Tests: 1 new scanBareImports test for subpath preservation; 1180 tests pass total. Limitation: jspm.io errors for subpaths the package's exports field doesnt declare. Most well-maintained packages declare their subpaths; legacy packages may not. Same behavior as before for those cases (missing importmap entry, browser surfaces error).
…path version parsing Two real bugs found during deep PR review: 1. Scaffold .gitignore was ignoring .webjs/ entirely, including .webjs/vendor/. Users running 'webjs vendor pin' would write the importmap.json + downloaded bundles into a gitignored directory. git add wouldn't pick them up. Production deploys would never see the pin file. The entire 'webjs vendor pin' feature silently defeated for every scaffolded app. Fix: keep .webjs/ ignored (still right for any future tooling caches), but add !.webjs/vendor/ to un-ignore the vendor subdirectory. Mirrors Rails treating config/importmap.rb + vendor/javascript/ as committed. 2. listPinned's version parser for local --download URLs was treating the __subpath segment as part of the version. For '/__webjs/vendor/dayjs@1.11.13__plugin__utc.js', it returned version '1.11.13__plugin__utc' instead of '1.11.13'. Cosmetic bug in 'webjs vendor list' output for subpath imports. Fix: after slicing off the .js suffix, split on '__' to separate version from subpath. Version is everything before the first __, not the whole tail. New test 'listPinned: parses subpath URLs and extracts versions (not subpath as version)' plants a subpath entry + bundle file and asserts the parsed version is '1.11.13' not '1.11.13__plugin__utc'. 1181 tests pass (was 1180).
Same bug as the scaffold .gitignore fix: webjs vendor pin writes
.webjs/vendor/importmap.json, which would be silently ignored. Fix
applied to:
- .gitignore (repo root): affects webjs's own monorepo apps
(website, docs, ui-website) which inherit from this file
- examples/blog/.gitignore: the reference example app
scaffold template fix was in the previous commit. Three .gitignore
files now share the same pattern: .webjs/ ignored, .webjs/vendor/
un-ignored.
…avior
Previous test asserted the /__webjs/vendor/* path is 'unhandled (no
local vendor proxy)'. That was outdated: in --download mode the
server DOES handle that URL via serveDownloadedBundle, returning a
real file or 404 if missing. The old test was passing for the wrong
reason (handler returned 404 because no .webjs/vendor/ file existed).
Updated to three tests that accurately cover:
1. 404 when no bundle file on disk (with hint to run vendor pin
--download in the error body)
2. 200 + correct content-type when a real bundle is present
3. Path-traversal rejection (400 or 404, both safe)
1180 -> 1183 tests pass.
…h production images Companion to the earlier .gitignore fixes. The repo's .dockerignore excluded all .webjs/ directories from the Docker build context. Even if the user committed .webjs/vendor/importmap.json (after the .gitignore fixes), the Dockerfile's COPY statements would exclude it because Docker uses .dockerignore, not .gitignore. Result: production images would not contain the pin file. Server would fall back to live api.jspm.io calls on every cold start, defeating the deterministic-deploy property the pin file provides. Fix: keep **/.webjs ignored generally, but un-ignore **/.webjs/vendor and its contents. Same pattern as the .gitignore changes. Trio of related fixes in this audit: - packages/cli/templates/.gitignore (scaffold) - .gitignore + examples/blog/.gitignore (monorepo apps) - .dockerignore (Docker context)
…resolved value
Two concurrent rebuilds during dev (chokidar firing twice quickly,
or two simultaneous server startups in tests) would each hit the
check-then-set race: both see jspmCache.has() return false, both
issue an HTTP request to api.jspm.io, both call cache.set with
their own result. Wasteful, and the second-to-complete write
clobbers the first (deterministic but redundant).
Standard Promise-cache pattern fixes it: store the Promise
immediately, before awaiting the fetch. Concurrent callers with
the same install list share the in-flight request and resolve
together.
Also: on failure, drop the cache entry so retries can succeed.
Without this, a transient api.jspm.io error would poison the cache
with {} forever (or until the process restarted).
1183 tests still pass.
Strict CSP with script-src 'self' blocks the jspm.io script tag, so vendor imports fail to load. This wasn't documented anywhere. Added a 'Content Security Policy (CSP) and vendor packages' section to the deployment doc with the two mitigations: 1. Allow jspm.io in CSP: add https://ga.jspm.io to script-src 2. Switch to --download mode: bundles served from same origin, 'self' alone sufficient Suitable scenarios per mode: jspm.io default for typical apps, --download for compliance / air-gapped / strict-CSP environments.
…S.md
Scaffolded apps had no documentation about webjs vendor pin. AI
agents working in those projects wouldn't know about it. They'd
either: (a) call api.jspm.io on every server boot indefinitely, or
(b) forget about the vendor pipeline entirely and write fetch()
calls for npm packages.
Added a focused section after the Database section:
- Standard npm install workflow
- webjs vendor pin for production (writes committed pin file)
- webjs vendor pin --download for offline/CSP-strict scenarios
- webjs vendor list / unpin commands
- Why pin is intentionally NOT in predev/prestart (would cause
silent churn in committed importmap.json)
Cross-references docs.webjs.com Deployment > CSP section for the
strict-CSP discussion.
Previous 'fix' was wrong. Verified with git check-ignore. Pattern was: .webjs/ !.webjs/vendor/ This silently does NOT work. Per the gitignore man page: 'It is not possible to re-include a file if a parent directory of that file is excluded.' The .webjs/ rule excludes the directory itself; git won't even traverse into it to evaluate the !.webjs/vendor/ exception. git check-ignore reports .webjs/vendor/importmap.json as ignored, despite the apparent exception. Correct pattern: .webjs/* !.webjs/vendor/ !.webjs/vendor/** The .webjs/* glob excludes contents of .webjs/ but not the directory itself. The two un-ignore rules cover both the vendor subdirectory entry AND its recursive contents (including nested files like .webjs/vendor/some-pkg/inner.js if --download bundles ever land in subdirectories). Verified with git check-ignore on a temp repo: .gitignore:14:!.webjs/vendor/** .webjs/vendor/importmap.json .gitignore:12:.webjs/* .webjs/cache/ts.bin .gitignore:14:!.webjs/vendor/** .webjs/vendor/sub/nested.js Vendor files are correctly un-ignored; future cache files stay ignored. Same pattern applied to .dockerignore for the same reason. Docker uses gitignore-like syntax with the same parent-exclusion gotcha. Affected files: packages/cli/templates/.gitignore (scaffold) .gitignore (repo root) examples/blog/.gitignore .dockerignore Embarrassing miss in the previous audit pass: I 'fixed' the gitignore but didn't verify with git check-ignore. The user caught it.
Verifies via `git check-ignore` that .webjs/vendor/importmap.json is not accidentally ignored. The common mistake is simplifying the three-line exception pattern (`.webjs/*` + `!.webjs/vendor/` + `!.webjs/vendor/**`) back to `.webjs/`, which silently breaks `webjs vendor pin`: gitignore semantics excludes the parent first, after which no child negation can re-include anything. Skipped when the directory is not a git repo or has no .gitignore. Strengthens inline comments in both .gitignore files to call out the hazard and point at this rule.
Adds inline warnings to examples/blog/.gitignore and .dockerignore matching the scaffold template, plus a paragraph in the scaffold's AGENTS.md vendor section explaining why the three-line pattern is structurally load-bearing and pointing at the lint rule that catches regressions.
Tooling lives in dot-prefixed directories (.opencode/, .claude/,
.github/, .husky/, .vscode/) and root-level config files
(web-test-runner.config.js, vitest.config.ts, tailwind.config.mjs). It
imports packages the browser will never load (test runners, AI tool
plugins) that legitimately cannot resolve via jspm.io. With these
specifiers in the install batch, api.jspm.io/generate returns 401 and
the entire importmap silently empties, breaking legitimate user deps
like dayjs.
Skip ALL dot-prefixed directories during the walk and any file matching
*.config.{js,ts,mjs,mts,cjs,cts} at any depth. Adds two new tests
covering the new exclusion behavior.
api.jspm.io/generate returns 401 with an error body when ANY package in the install batch fails to resolve (e.g. a transitive subpath that isn't exported). The previous batched call collapsed the entire importmap on a single failure, silently dropping legitimate user deps like dayjs and breaking pages in the browser with bare-specifier errors. Split jspmGenerate into per-install calls running in parallel via Promise.all. Cache keys are now individual install specs, so concurrent rebuilds with overlapping deps still share work. Failure logs name the offending package and surface jspm.io's error reason. Regression test plants a known-bad install alongside a known-good one and asserts the good one still resolves.
The framework already extracts the CSP nonce from incoming Content-Security-Policy headers and applies it to other inline scripts (env shim, boot, suspense), but importMapTag was bare. Strict-CSP apps using script-src 'nonce-...' policies silently lost the entire vendor pipeline: browser blocked the unsigned importmap tag, every bare-specifier import failed. importMapTag now takes a nonce option and emits nonce="..." when provided. ssr.js threads opts.nonce through alongside the publicEnvShim call. Matches the pattern Turbo's test fixtures use.
Browsers require crossorigin on cross-origin modulepreload, else the preload is ignored or double-fetched (defeating the optimization). Same-origin preloads must NOT carry the attribute for the same reason in reverse. Vendor packages resolved to jspm.io URLs are the new common case after the per-package vendor pipeline lands. Today vendor URLs flow only through the importmap (not preload), so this fix is preventative: if a future change adds vendor URLs to the preload set or a user lists a CDN URL in metadata.preload, the modulepreload now does what it claims. Exports preloadCrossOriginAttr for unit testing; covers cross-origin, same-origin path, and same-origin URL with /__webjs/vendor/ prefix.
`webjs vendor pin` now computes a SHA-384 hash for every resolved
vendor URL and writes it into `.webjs/vendor/importmap.json` under
a new `integrity` key:
{
"imports": { "dayjs": "https://ga.jspm.io/.../dayjs.min.js" },
"integrity": { "https://ga.jspm.io/.../dayjs.min.js": "sha384-..." }
}
Default mode fetches each bundle solely to hash it (bytes not written
to disk). --download mode hashes the bytes it already downloads.
resolveVendorImports returns both maps. setVendorEntries(imports,
integrity) stores them in the importmap module. buildImportMap emits
the integrity field per the browser importmap-integrity spec
(Chrome 132+, Safari 18.4+). Modulepreload tags get
`integrity="sha384-..."` when the URL has a known hash.
Older pin files lacking the integrity field still load (treated as
empty integrity map). Live-API mode skips integrity entirely; users
who want SRI run `webjs vendor pin`.
Updates the resolveVendorImports unit test for the new return shape.
Adds 5 tests covering: - readPinFile returns integrity when present - readPinFile is backwards-compatible (no integrity field on old format) - sha384Integrity returns deterministic sha384-<base64> strings - pinAll default mode writes integrity field with sha384 hashes - pinAll --download mode integrity matches on-disk bytes byte-for-byte Also fixes a latent hazard in makeTempAppWithSource: it symlinks the repo's node_modules into the temp dir, so a `sourceFiles` entry like `node_modules/picocolors/package.json` would clobber the real picocolors package (the entry resolved through the symlink and rewrote the real package.json with a 2-line stub, breaking every test that needed picocolors locally). Helper now refuses paths under `node_modules/`. The previously-corrupted picocolors was reinstalled.
Mirrors the same change in the main web-test-runner.config.js. The blog-e2e config is a separate file because it boots the blog dev server on :3456 first and proxies /__blog/* requests; both configs needed the same plugin substitution.
deployment/page.ts, typescript/page.ts, getting-started/page.ts, and no-build/page.ts described the esbuild fallback that's now gone. Rewrote each passage to state: - Only erasable TypeScript is supported (matches the new dev.js behavior after dropping the fallback). - Non-erasable syntax now fails at strip time with a 500 pointing at the no-non-erasable-typescript lint rule. - "esbuild-with-sourcemap pipeline" comparison line in typescript.ts rephrased to "bundler-with-sourcemap pipeline" since esbuild is no longer the local point of comparison. - "esbuild fallback available" line in deployment.ts replaced with "buildless end-to-end: no bundler or transpiler at deploy time". Scaffold + lint-rule message updates follow in the next commit.
The erasable-typescript-only rule's description (in check.js) and the four scaffold-template files (AGENTS.md, CONVENTIONS.md, .cursorrules, .windsurfrules, .github/copilot-instructions.md) all said "the dev server falls back to esbuild + inline sourcemap" for non-erasable TS. That's no longer true: the fallback is gone in this PR, and the dev server now returns a 500 pointing at the no-non-erasable-typescript lint rule. Updated prose in all listed files to reflect the new behavior plus the buildless-end-to-end framing. Remaining 3 files (templates CONVENTIONS.md + examples/blog CONVENTIONS.md + examples/blog copilot-instructions.md) follow in the next commit.
… test comments Final esbuild-mention cleanup. Mirrors the previous commit's prose update across the 3 remaining files: - packages/cli/templates/CONVENTIONS.md - examples/blog/CONVENTIONS.md - examples/blog/.github/copilot-instructions.md All now say: "the dev server fails at strip time and returns a 500 pointing at the no-non-erasable-typescript lint rule. webjs is buildless end-to-end and has no bundler fallback." Also tidies three stale esbuild-reminiscing comments in packages/server/test/dev/dev-handler.test.js (test name "esbuild- stripped types" -> "types stripped"; "esbuild may rewrite" comment replaced with note about stripTypeScriptTypes preserving source; section header "missing esbuild path" -> "tsResponse cache path"). After this commit, no executable code in the repo references esbuild outside of changelog history.
Final esbuild-cleanup pass for top-level docs (README, root AGENTS, packages/server/AGENTS, agent-docs/typescript, agent-docs/advanced). agent-docs/advanced.md had a stale description of the old vendor pipeline (esbuild-based bundling at /__webjs/vendor/<pkg>.js). Rewrote the section to describe the new jspm.io-direct posture: bare specifiers resolve through api.jspm.io to CDN URLs, the browser fetches directly from ga.jspm.io, `webjs vendor pin` commits resolved URLs + SHA-384 integrity hashes for reproducible deploys, `--download` caches bundles locally for air-gapped / strict-CSP deploys. Mirrors what dev.js + vendor.js actually do now. README, root AGENTS invariant 10, packages/server/AGENTS module map, and agent-docs/typescript all said non-erasable TS "falls back to esbuild". Updated each to say it now returns a 500 pointing at the no-non-erasable-typescript lint rule, since webjs is buildless end-to-end with no bundler fallback. After this commit no executable code or current documentation references esbuild; remaining hits are in changelog/ (historical) and blog/strip-types-not-esbuild.md (the blog post itself, which narrates the journey of why esbuild went away).
Two more spots not caught in the earlier batches: a second README
mention saying "esbuild stays as a per-file fallback" and a second
no-build docs page bullet describing the now-removed fallback as
shipping "inline sourcemaps and roughly 3x wire bytes".
Both rewritten to match the new behavior: dev server returns a 500
naming the file and pointing at the no-non-erasable-typescript lint
rule, since webjs is buildless end-to-end with no bundler fallback.
Remaining executable-code esbuild mentions are now either
intentionally-negative ("No esbuild" in web-test-runner.config.js)
or describe-the-new-behavior ("there is no longer an esbuild" in
dev-handler.test.js). Historical changelog/ + the strip-types blog
post stay as-is.
Two real CSP / vendor-correctness gaps surfaced by comparing against
importmap-rails (rails/importmap-rails packager) and hotwired/turbo:
1. jspmResolveOne now sends `flattenScope: true` in the
api.jspm.io/generate request. Without it jspm.io returns transitive
ESM deps in a separate `scopes` field (e.g. react-dom imports
`scheduler`, returned as `scopes: { "https://ga.jspm.io/":
{ "scheduler": "..." } }`). Webjs only consumes
`result.map.imports` so the transitive would silently never reach
the browser importmap, breaking the page with an unresolved
bare-specifier error. importmap-rails has always sent this. Verified
via direct curl: simple packages (dayjs, clsx, picocolors) produce
identical output either way, so existing tests are unaffected; react
+ react-dom go from `imports: { react, react-dom } + scopes: {...}`
to `imports: { react, react-dom, scheduler }`.
2. ssr.js documentParts now emits `nonce="..."` on every
`<link rel="modulepreload">` tag when a CSP nonce is in scope. Under
strict CSP (script-src 'nonce-...') the browser also gates
modulepreload by the same policy; without the attribute the preload
is blocked. Rails applies nonce on every preload tag for the same
reason (importmap-rails javascript_importmap_module_preload_tags).
Out of scope for this PR but filed as task #36: dynamic-script
nonce propagation in the client router (Turbo's
`<meta name="csp-nonce">` pattern, needed for strict CSP with client
nav). Affects strict-CSP users only.
Strict-CSP apps (script-src 'nonce-...') were broken on client-side
navigation: the client router copies scripts from the new page's
fetched head, but the new page's nonce is per-request and doesn't
match the original page-load nonce the browser's CSP cache holds.
Every dynamic script after the first navigation was blocked.
Server (packages/server/src/ssr.js):
- Emits <meta name="csp-nonce" content="${nonce}"> in the head when
the request CSP has a nonce. Standard Turbo convention. Existing
CSP nonce extraction (getNonce) feeds this.
Client (packages/core/src/router-client.js):
- getCspNonce() reads the meta tag (not cached: cheap querySelector,
avoids test/late-insert pitfalls). Mirrors turbo/src/util.js.
- cloneScriptWithCorrectNonce(source) copies non-nonce attributes
from the source, applies the cached page-load nonce via
setAttribute, re-emits textContent. Used by all three dynamic
script-creation sites (addNewHeadElements, mergeHead,
reactivateScripts).
- outerHTMLForDiff(el) strips the nonce attribute from script
outerHTML for head-diff comparison so per-request nonces don't
cause every script to look "changed" on every nav (would duplicate
scripts indefinitely). Mirrors turbo's elementWithoutNonce.
Tests:
- ssrPage: meta csp-nonce emitted when CSP header has nonce, absent
otherwise.
- addNewHeadElements: dynamic script gets the meta nonce, not the
source-page nonce.
- addNewHeadElements: head diff ignores nonce-only differences (no
spurious re-add / duplicate scripts).
End-to-end correctness for strict-CSP apps now: importmap nonce
(c3ce2ae), modulepreload nonce (d453e3e), and client-router dynamic
scripts (this commit). The full CSP story works across initial load
and client navigation.
Two correctness findings from re-comparing webjs against Rails+Turbo source. 1) outerHTMLForDiff now strips nonce from ANY element, not just SCRIPT. modulepreload <link> tags carry nonce after the recent CSP fix; without this, per-request nonces on link tags would cause head-diff to flag them as "changed" and append a duplicate preload on every navigation. 2) Added cloneElementWithCorrectNonce for non-script head elements that carry nonce (modulepreload links being the common case). Same pattern as cloneScriptWithCorrectNonce: copy attributes, substitute the page-load nonce for the source's per-request nonce. applySwap's `else` branches (non-script head clones) now use this instead of plain cloneNode(true). 3) applySwap now triggers a full page reload (location.href = href) when the incoming document's importmap differs from the current page's. Importmaps are immutable once applied, so partial swap after a deploy with bumped vendor pins would leave stale URLs in place and silently break module resolution. Mirrors Turbo's tracked_element_mismatch reload behavior, specifically applied to importmaps (Rails uses data-turbo-track=reload on the importmap script for the same effect). Replaces the prior console.warn-only handling in addNewHeadElements (which left the user on a broken page). Threads `href` through performNavigation -> fetchAndApply -> applySwap; popstate cache restores pass href=null to keep revalidation soft (the cached page already had the matching importmap). Tests: 3 obsolete importmap-warning tests removed/rewritten; 2 new integration tests assert hard-reload-on-mismatch + skip-on-match.
Discovered while testing scaffolds: when pinAll attempted to resolve
packages and every one failed (e.g. dayjs@1.11.21 was too new and
jspm.io's CDN hadn't indexed it yet), pinAll wrote
.webjs/vendor/importmap.json with `{ "imports": {} }`. On next boot,
readPinFile returned a truthy file with empty imports; the
live-API-fallback path (which runs when readPinFile returns null)
was shadowed. Browser ended up with no vendor entries and every
bare-specifier import silently broke.
pinAll now detects "installs attempted > 0 AND pins resolved == 0"
and returns { failed: true, attemptedInstalls: [...], pins: [],
pruned: [], downloaded: 0 } WITHOUT writing the pin file. The CLI
prints a clear error explaining the failure, lists the attempted
installs (per-package errors already logged loudly by
jspmResolveOne), and exits non-zero. Next boot falls back to live
API resolution, which may have recovered by then.
Regression test in packages/server/test/vendor/vendor.test.js
plants a fake-pkg-xyz-no-such-version@99.99.99 (unknown to
jspm.io), runs pinAll, asserts result.failed AND no pin file
written.
CSP nonce edge cases (8 new tests in router-client.test.js): - mergeHead: applies meta csp-nonce to scripts created during full body swap - addNewHeadElements + mergeHead: nonce-only diff on <link> tags does not duplicate preloads (regression check for the recent fix that strips nonce from any element type in outerHTMLForDiff) - reactivateScripts: applies meta csp-nonce to body scripts re-emitted after a full body swap SRI edge cases (3 new tests in vendor.test.js): - readPinFile + resolveVendorImports: integrity is keyed by FINAL URL (post-rewrite), so --download mode integrity keys on /__webjs/vendor/... not on the original jspm.io URL. Regression check for subpath integrity propagation. - readPinFile: tolerates extra fields in pin JSON (forward-compat for future fields like resolver, generatedAt, _comment). - importMapTag: integrity field omitted from JSON when empty, present when populated. Matches importmap-rails behavior. All 1212 tests pass.
… tags publicEnvShim already escaped `</` to `<\/` when emitting JSON inside its inline script. importMapTag and the boot script in wrapHead did NOT, relying on raw JSON.stringify. A string value containing `</script>` would close the script element early and let any HTML after it execute as fresh content. Defense-in-depth attack surface: small but real (vendor URLs from a maliciously-crafted package or specifier names from compromised source). Extracted a shared jsonForScriptTag helper into a new script-tag-json.js module (avoids the ssr.js <-> importmap.js circular import). The helper escapes `</`, U+2028, and U+2029. Applied to importMapTag, publicEnvShim, and the boot module-imports script in wrapHead. ASCII-only source: literal chars are constructed via String.fromCharCode at runtime so the file itself stays parseable under every JS dialect. Tests: importMapTag with a URL containing `</script><img ...>` asserts no early-close sequence survives. Separate test plants U+2028 and U+2029 in URLs and asserts they encode to / . All 1212+ existing tests still pass.
Two adversarial-pass findings: 1. serveDownloadedBundle echoed the user-supplied `filename` into its error response bodies. Block-comment close (`*/`) was incidentally blocked by the existing path-traversal check (rejects `/`), but defense-in-depth says don't echo input at all. Switched the input validation to a strict allowlist regex `^[A-Za-z0-9@._-]+\.js$` matching the framework's own filename-generation scheme (pkg@version[__subpath].js, scope--name forms). The error response bodies are now fixed strings, no echoes. 2. The strip-ts 500 response in dev.js leaked the full filesystem path AND Node's error message (which can include source snippets) to the browser. Fine in dev, not fine in prod. Split the response: dev keeps the verbose form (developer sees offending construct + path), prod returns a terse "Check server logs" message and writes the full detail to console.error for the operator. Lint catches non-erasable TS at edit time so this path only fires if the user has misconfigured; the prod terseness is defense-in-depth for that edge case. Tests: - Existing serveDownloadedBundle path-traversal test still passes (new regex covers the same rejection set plus more). - New test asserts the prod-mode strip-ts response leaks neither appDir nor Node's error message but still mentions "Check server logs" so the operator can find the diagnostic.
Two adversarial-pass findings on pin file handling.
1. readPinFile accepted any value type in `imports` and `integrity`.
A hand-edited or malicious pin file with non-string values
(numbers, objects, nulls) would land structurally invalid entries
in the served importmap and break browser-side module resolution
for the whole page. Now: validates `imports` is a plain object,
filters non-string keys/values, and rejects integrity values that
don't look like SRI hashes (`sha(256|384|512)-...`). Also rejects
the file entirely when `imports` parses to something other than
an object (string, null, array, number, boolean).
2. unpinPackage left an empty `{ imports: {} }` pin file behind when
the last pin was removed, shadowing the live-API fallback (same
anti-pattern pinAll guards against). Now: deletes the pin file
entirely when imports becomes empty. Also strips the integrity
entry for the unpinned URL (previously left orphan integrity).
CLI test updated to assert file removal instead of file presence
with empty imports.
Tests:
- readPinFile: corrupt JSON, non-object imports (5 variants),
non-string values filtered, integrity-not-SRI-shape filtered.
- unpinPackage: keeps file when other pins remain, strips
only-the-targeted entry's integrity, deletes file when last pin
removed.
The client router's importmap-mismatch hard-reload (commit f375af8) compares the served textContent of the importmap script tag. With unsorted keys, two deploys with identical vendor pins but different filesystem iteration order (e.g. after a file rename) would produce different JSON byte sequences and trigger an unnecessary full page reload on every nav until the user's tab caught up. buildImportMap now sorts both `imports` and `integrity` keys before serializing. Same logical content always produces byte-identical output regardless of insertion order. Regression test plants the same logical importmap twice with different insertion orders and asserts byte-identical JSON output.
Chokidar fires rebuild on every relevant file change with an 80ms debounce. If two file edits arrive within ~80ms but each rebuild takes >80ms (jspm.io fetch easily takes 100-500ms), both rebuilds run concurrently and whichever finishes LAST wins. Failure mode: rebuild #1 starts with the file state before edit B. Rebuild #2 starts (debounced) with the post-B state. If #1's jspm.io fetch is slow and #2 is fast, #2 calls setVendorEntries first with fresh data, then #1 calls it with stale data, leaving the dev server serving a permanently-stale importmap until the next rebuild. Fix: chain rebuilds onto a sequential promise so the next rebuild waits for the previous to finish. Also adds a monotonic token: a rebuild's setVendorEntries call is no-op if a newer rebuild has already been queued. The token is defensive belt-and-suspenders; serialization alone would suffice. No new tests (the race is a timing window that needs real chokidar events to exercise; serialization is provable from the code shape).
The rule previously only checked .webjs/vendor/importmap.json. A .gitignore that allows the JSON manifest but blocks bundle files (common pattern: a broader rule like `*.js` at root) would still silently break `webjs vendor pin --download`: bundles never reach production, the importmap routes to `/__webjs/vendor/<file>.js`, server returns 404, page breaks. Now probes both: - .webjs/vendor/importmap.json (the manifest) - .webjs/vendor/sample-pkg@1.0.0.js (a representative bundle name) Adds a regression test that plants a `*.js` rule alongside the correct `.webjs/*` + exception pattern and asserts the rule fires with a message that mentions the bundle file probe.
ssrPage's error-boundary branch (route.errors) and the default fallback both went through wrapInDocument without `nonce` in opts. The error response still emits boot scripts (moduleUrls includes page + layouts on the error-boundary path) plus the meta csp-nonce tag, both of which need the request's nonce to pass strict-CSP enforcement. Without it, the error page itself fails to load any JS and subsequent client-side nav uses an empty nonce (since the meta csp-nonce tag is absent). Same gap in ssrNotFoundHtml. Fix: extract the request's nonce once via getNonce(opts.req) at the start of the error-handling block, thread it into every wrapInDocument call. ssrNotFoundHtml gets the same treatment. Regression test: ssrPage with a page that throws + a request CSP nonce, asserts the 500 response carries the meta csp-nonce tag.
…de tests Three adversarial-pass findings: 1. listPinned's version-extraction regex /\/npm:[^@]+@([^/]+)\// could not handle scoped packages: the scope's leading `@` (in `/npm:@scope/name@1.2.3/...`) didn't satisfy the `[^@]+` requirement. Scoped packages in `webjs vendor list` showed version `(unknown)`. New regex: /\/npm:(?:@[^/]+\/)?[^@/]+@([^/]+)\//. Tests cover `@scope/name@1.2.3`, `@hotwired/turbo@8.0.0`, plain packages, and malformed URLs. 2. Scanner stress tests added (5 new): CRLF line endings, UTF-8 BOM at file start, unterminated string literal (mid-edit user state), deeply nested 25-level dirs, multi-MB file. All pass without crashing or excessive time. 3. jspmGenerate failure-mode tests (7 new): fetch rejection, 5xx response, non-ok with JSON error body extracting detail, non-ok with non-JSON body, 200 with missing map.imports, 200 with map.imports as non-object, 200 with malformed JSON. All return empty map without throwing. All 71+ vendor tests pass.
Browser-level testing with playwright under a strict CSP (script-src 'nonce-...' 'self' https://ga.jspm.io) surfaced that inline scripts written by USER code (the scaffold's layout.ts has a theme-detection script for first-paint flicker prevention) were being blocked by the browser. The framework's nonce flow reaches framework-emitted scripts (importmap, env shim, boot, modulepreload, client-router dynamic scripts) but user-authored inline scripts in pages / layouts / metadata routes have no public API to read the nonce. New `cspNonce()` export from `@webjsdev/server` reads the nonce from the in-flight request's CSP header via the existing AsyncLocalStorage request context. Usage: import { cspNonce } from '@webjsdev/server'; return html`<script nonce="${cspNonce()}">...</script>`; Returns '' when no nonce in CSP (empty attribute, browser ignores) and '' outside a request (module-top-level safe; no throw). Scaffold template updates to USE this helper follow in a separate commit since the scaffold work overlaps with in-flight Docker / compose changes I don't own.
User layouts / pages / metadata routes need to call cspNonce() on
inline `<script>` tags to pass strict CSP. They cannot import from
@webjsdev/server because layout / page modules also load on the
browser (for side-effect component registration), and server-only
deps (node:async_hooks, etc.) crash there.
Fix: move cspNonce to @webjsdev/core (browser-safe). On the browser
it returns ''. On the server, @webjsdev/server's context module
calls setCspNonceProvider at load time, wiring the real reader that
pulls from AsyncLocalStorage and parses the request's CSP header.
The previously-shipped @webjsdev/server `cspNonce` export now
re-exports from core, so existing imports still work.
Scaffold layout.ts emission updated to import cspNonce from
@webjsdev/core. The actual application of `nonce="${cspNonce()}"`
to the theme-init `<script>` tag follows in the next commit (kept
separate so the diff is reviewable).
Found by: playwright + Chromium under
`script-src 'nonce-...' 'self' https://ga.jspm.io` reported the
scaffold's theme-detection inline script as a CSP violation. The
framework's nonce already reaches framework-emitted scripts
(importmap, env shim, boot, modulepreload, client-router dynamic
scripts); this commit closes the user-script gap.
The scaffold's RootLayout emits an inline `<script>` for theme detection (matchMedia + localStorage read for first-paint flicker prevention; the script must be inline + synchronous to beat the first paint). Without a nonce, strict CSP (script-src 'nonce-...') blocks it. Now uses `cspNonce()` from @webjsdev/core (the isomorphic helper landed in the previous commit). When no CSP nonce is in effect, the attribute is empty (browser ignores). When strict CSP is on, the inline script signs correctly. Verified end-to-end with playwright + Chromium against a fresh scaffold under `script-src 'nonce-...' 'self' https://ga.jspm.io; object-src 'none'`. Previously reported "Executing inline script violates the following CSP" violation is now gone; page boots clean with zero CSP violations.
Each of the 4 apps (examples/blog, website, docs, ui-website) has a
RootLayout that emits inline `<script>` tags for theme detection
(localStorage read for first-paint flicker prevention) plus the
Google Analytics gtag init scripts. Under strict CSP these get
blocked unless nonce-signed.
All 4 layouts now:
1. Import `cspNonce` from `@webjsdev/core`.
2. Read it once at the top of the layout function: `const nonce = cspNonce();`.
3. Apply `nonce="${nonce}"` to every inline `<script>` AND to the
`<script async src=".../gtag/js">` external loader (so cross-origin
GA script also passes script-src enforcement when the user
configures CSP).
When no CSP nonce is in scope the attribute renders empty
(browser ignores it), so existing non-CSP deployments are unaffected.
Companion to the scaffold's layout.ts emission (commit 816b9f8) so
the in-repo apps follow the same pattern users will see in their
own scaffolded code.
When a Suspense boundary settles during streaming SSR, the framework
emits a `<template data-webjs-resolve="...">` plus a fallback inline
`<script>window.__webjsResolve&&__webjsResolve("...")</script>` for
browsers without MutationObserver. The fallback was missing the
nonce attribute, so strict-CSP enforcement blocked it.
Found by playwright + Chromium browsing the blog under
`script-src 'nonce-...' 'self' https://ga.jspm.io`: the homepage
streams a suspended comment thread, the resolution script fired the
"Executing inline script violates the following Content Security
Policy directive" violation in the browser console.
Fix: thread `nonce` from ssrPage through streamingHtmlResponse into
the per-resolution script chunk. When no nonce is in scope the
attribute is omitted entirely (same as before).
Regression test mocks a request with a CSP nonce, asserts every
emitted `<script>...__webjsResolve...</script>` chunk carries the
matching nonce.
Verified end-to-end: blog now reports zero CSP violations under
strict policy. Same applies to any user app with Suspense or
loading.js boundaries.
Adversarial probe of readPinFile surfaced two real security gaps. 1. A malicious pin file could carry a `javascript:` or `data:` URL in its imports map. Both reach the served importmap as-is. The browser's importmap spec explicitly accepts data: URLs, so an attacker who lands such a commit ships code execution to every visitor on the next deploy via a single-line pin diff. A user reviewing the diff might miss it (looks like a "vendor URL"). 2. Newlines or other control characters in imports keys serialize to escape sequences in the served JSON. Not directly exploitable today (browser parses fine, client-router compares textContent verbatim) but lets attacker-controlled content reach log output and downstream tooling. Fix: tighten readPinFile's filter to accept only URLs starting with `http://`, `https://`, or `/` (matching what `webjs vendor pin` itself emits). Drop keys containing any C0 control character (`\x00-\x1F`) or `\x7F` (DEL). Tests cover: - javascript: / data: / blob: / file: / ftp: schemes all rejected - http(s) and root-relative URLs preserved - keys with \n, \r, low-ASCII control chars dropped
Adversarial probe of /__webjs/vendor/<file>.js found that POST/PUT/DELETE/PATCH requests returned 200 with the bundle body. Vendor bundles are read-only static content; non-GET/HEAD makes no semantic sense and matches the standard static-file behavior. Fix in dev.js: check the method before delegating to serveDownloadedBundle. Non-GET/HEAD returns 405 with an `allow: GET, HEAD` header. HEAD returns the GET headers with an empty body (per HTTP semantics). Tests cover: POST/PUT/DELETE/PATCH → 405, HEAD → 200 with empty body.
The static /public/* handler did `join(appDir, path)` without checking that the resolved absolute path stays under `appDir/public/`. Node's URL parser normalises raw `..` segments, but it does NOT decode percent-encoded characters. webjs then runs `decodeURIComponent` on the parsed pathname AFTER URL parsing. Combined, this lets a `/public%2F..%2Fsecret%2Fsecret.svg` URL survive URL parsing as `/public%2F..%2Fsecret%2Fsecret.svg`, decode to `/public/../secret/secret.svg`, match the `/public/` branch, and have `join` resolve to `appDir/secret/secret.svg` (outside public/). Fix: after computing `abs = join(appDir, p)`, verify `abs.startsWith(appDir + path.sep + 'public' + path.sep)`. Reject with 404 if not. Tests in /tmp covered raw `..`, `%2E%2E`, percent-encoded slash, double-encoded forms. Only the `%2F..%2F` (encoded slash) variant exploited the /public/ branch specifically; the others fall through to URL-parser normalisation and hit the user-source branch, which is a separate appDir-wide-exposure concern filed as task #53.
The documented broadcast(path, data) example was silently a no-op: the WS upgrade handler never called registerClient, so pathClients stayed empty and broadcast() walked an empty Set. User code that followed the docs example saw zero messages reach any peer. Hook registerClient(url.pathname, ws) into the upgrade callback so every upgraded socket joins the per-path Set automatically. The existing close handler in registerClient handles unregistration. Regression test covers the full path: two clients upgrade against a route.js WS endpoint, broadcast() reaches both, clientCount tracks open + closed transitions correctly.
The lint message and fix hint referenced lib/prisma.ts, but every scaffold creates lib/prisma.server.ts (with the .server. infix that triggers the source-protection guardrail). Users following the fix hint hit a missing-file error.
Two latent bugs in the in-memory store: 1. ttlMs = NaN slipped past the truthiness check and produced an entry with expiresAt = null (no expiration). Code computing TTL from arithmetic (e.g. Date.parse() - Date.now() on a bad input) could silently lock entries in forever. Tighten to a finite, positive Number predicate. Infinity, 0, negative, and non-number all fall back to the documented "no TTL" path. 2. increment() mutated the entry value in place without re-inserting the key, so its Map insertion-order position stayed at its original (oldest) slot. Hot rate-limit buckets got evicted ahead of less-active keys, defeating the LRU intent. Re-insert on every increment to keep hot keys at the recent end. Tests cover NaN, Infinity, 0, negative ttl, plus the hot-bucket-survives-eviction case.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces main's Vite-style
optimizeDepsesbuild-on-demand vendor pipeline with the Rails 7 + importmap-rails posture exactly: bare-specifier npm imports resolve via importmap to jspm.io CDN URLs and the browser fetches the bundle directly from jspm.io. The webjs server does not proxy, cache, or bundle vendor packages.This is the strictest "no build" architecture for a no-build framework. Nothing bundles on the user's machine, ever.
How it works (Rails-aligned)
At server boot:
scanBareImportswalks user source for bare-specifier imports (import dayjs from 'dayjs')getPackageVersionreads each package's installed version fromnode_modules/<pkg>/package.jsonjspmGeneratePOSTs the resolved install list (['dayjs@1.11.13', '@hotwired/turbo@8.0.0', ...]) tohttps://api.jspm.io/generatewithprovider=jspm.io,env=['browser', 'production', 'module']<script type="importmap">tagAt runtime, the importmap looks like:
Browser fetches the bundle directly from
ga.jspm.io. The webjs server never sees the request. Same as Rails today.Why jspm.io (not esm.sh)
What this PR is NOT
.webjs/vendor/directory, nothing committed to source control)webjs vendor pin/unpin/list/warmCLI (no cache to manage)What this PR fixes vs main
/__webjs/vendor/<pkg>.jsURL never changed)route.tswsfrom type-only imports(?!type\s)lookaheadclsx/tailwind-mergefrom JSDoc commentsstripComments()pre-passCommits (3)
fcf2692Scanner tightening (route.ts/middleware.ts exclusion, test/, import type, comments). Cherry-picked from PR feat(server)!: version-in-URL for vendor + scanner tightening + lint rule + optional warm #88.83e77a9no-non-erasable-typescriptlint rule. Cherry-picked from PR feat(server)!: version-in-URL for vendor + scanner tightening + lint rule + optional warm #88.88e36cbThe main rewrite: vendor.js + dev.js + index.js + tests. ReplacesbundlePackage/serveVendorBundle/in-memory bundle cache withjspmGenerate/vendorImportMapEntries. Removes the/__webjs/vendor/*URL handler. Test suite updated; 1160/1160 pass.Breaking changes
@webjsdev/serverno longer exportsbundlePackageorserveVendorBundle(these functions are deleted; their work is done by jspm.io now)/__webjs/vendor/*URL paths no longer handled by the server (browser fetches direct fromga.jspm.io)vendorImportMapEntriesis now async (takesappDirparameter and callsjspmGenerateinternally)api.jspm.ioreachability at server boot to populate the vendor importmap. If unreachable, the server still boots and serves user routes; only vendor-importing pages report "unresolved bare specifier" errors in the browser until the API is reachable again.What "no build" means under this PR
Strict no-build for user-facing aspects:
Caveat: esbuild still in
@webjsdev/serverdeps as TS-stripping fallback for non-erasable syntax (rare, with the newno-non-erasable-typescriptlint rule catching most cases at commit time). Removing esbuild entirely is a separate decision tracked as a follow-up.Relationship to other PRs
PR #88 and this PR fix the same main-branch bugs. They differ on the bundler-locus axis. The user should choose one to merge and close the other.
Test plan
npm test: 1160/1160 passgetPackageVersionverified against installed picocolors + null fallbackjspmGenerateverified against real api.jspm.io for picocolors (network-gated)vendorImportMapEntriesintegration verified end-to-endFollow-ups
@webjsdev/serverdependencies entirely? Currently kept for TS-stripping fallback. Withno-non-erasable-typescriptlint enabled, the fallback rarely fires. Removing it would shed ~56 packages (esbuild + 52 platform binaries + wrappers). Decision deferred; the lint rule needs proven-in-the-wild reliability first.--download). Would eliminate the runtime dep onapi.jspm.iofor known package sets. Not in scope here.