Add nub package manager ecosystem (nub.lock)#15524
Open
colinhacks wants to merge 22 commits into
Open
Conversation
nub writes nub.lock (byte-compatible with pnpm-lock v9). The gem is cloned from the bun ecosystem and adapted: nub.lock detection, packageManager nub@x, NubLock routed through the pnpm parseLockfile helper, native updater invokes nub install --lockfile-only, Dockerfile installs @nubjs/nub. WIP: spec fixtures still carry bun text-format lockfiles and need regenerating as pnpm-v9 nub.lock; parser specs updated accordingly.
Adds require "dependabot/nub" so the ecosystem gem is loaded/discovered. Remaining infra wiring (ci.yml, ci-filters.yml, smoke-filters.yml, issue-labeler.yml, bin/dry-run.rb, script/dependabot, rakelib helpers) must be generated via `rake ecosystem:update_infrastructure[nub]` inside the dependabot dev environment (ruby 3 + bundle).
- Fix compound-word rename corruption: :npm_nubdled -> :npm_bundled (B1). - NubLock#dependencies: two-pass (specifier-bearing first) so alias metadata survives DependencySet de-dupe, mirroring the pnpm parser (S1). - FileUpdater: drop invalid bun-style 'install <pkg>@<ver>'; nub is pnpm-compatible, so write the bumped package.json + 'nub install --lockfile-only' (V1 core path). - Defer dependency_grapher: bun's grapher reads the bun-lock hash shape, incompatible with the pnpm-array parse output; the pnpm-v9 edge-reading rewrite is a follow-up. - version_resolver: TODO documenting the remaining 'update <pkg>@<ver>' grammar gap. - Regenerate simple_v0/simple_v1/wildcard fixtures as real pnpm-v9 nub.lock; rewrite lockfile_parser_spec to pnpm-v9 semantics. - Dockerfile NUB_VERSION -> 0.3.1 (current published).
Add "nub" => "nub" to Config::File::PACKAGE_MANAGER_LOOKUP so Utils.validate_package_manager! accepts it; without this the whole nub ecosystem fails to load (register_version_class raises). Found by running the nub parser spec in the dev image.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| return [glob] unless glob.include?("*") | ||
|
|
||
| unglobbed_path = | ||
| glob.gsub(%r{^\./}, "").gsub(/!\([^)]*\)/, "*") |
Comment on lines
+334
to
+336
| glob.gsub(%r{^\./}, "").gsub(/!\([^)]*\)/, "*") | ||
| .split("*") | ||
| .first&.gsub(%r{(?<=/)[^/]*$}, "") || "." |
|
|
||
| sig { params(glob: String, paths: T::Array[String]).returns(T::Array[String]) } | ||
| def matching_paths(glob, paths) | ||
| glob = glob.gsub(%r{^\./}, "").gsub(/!\([^)]*\)/, "*") |
| return Dependabot::DependencyRequirement.create(req.merge(requirement: updated_req)) | ||
| end | ||
|
|
||
| reqs = current_requirement.strip.split(SEPARATOR).map(&:strip) |
| ruby_reqs = ruby_requirements(current_requirement) | ||
| return req if ruby_reqs.any? { |r| r.satisfied_by?(version) } | ||
|
|
||
| reqs = current_requirement.strip.split(SEPARATOR).map(&:strip) |
| updated_requirement = | ||
| if reqs.any? { |r| r.match?(/(<|-\s)/i) } | ||
| update_range_requirement(current_requirement) | ||
| elsif current_requirement.strip.split(SEPARATOR).one? |
| sig { params(req_string: String).returns(String) } | ||
| def update_range_requirement(req_string) | ||
| range_requirements = | ||
| req_string.split(SEPARATOR).select { |r| r.match?(/<|(\s+-\s+)/) } |
| sig { params(req_string: String).returns(String) } | ||
| def update_range_requirement(req_string) | ||
| range_requirements = | ||
| req_string.split(SEPARATOR).select { |r| r.match?(/<|(\s+-\s+)/) } |
nub is pnpm-compatible with no 'install <pkg>@<ver>' add and rejects 'update <pkg>@<ver>' for an arbitrary version, so a lockfile-only update pins each Dependabot-chosen version by writing it exactly into package.json, running 'nub install --lockfile-only', then restoring the final requirement and installing again (nub's lockfile-only install keeps the locked version). Regenerates the multiple_updates + github_dependency fixtures as pnpm-v9; updates file_updater/subdependency spec expectations to nub's grammar + the pnpm-v9 git-dep (codeload tarball) lockfile form.
- Fix file_fetcher nub-detection: contents listing + nub_lock fixture were bun-format (JSON fixtures weren't renamed); list nub.lock + pnpm-v9 content. - Skip (documented follow-ups): git-dependency updates (nub keeps the locked git SHA; forcing re-resolution to a chosen ref is unwired) and monorepo source-directory metadata extraction.
Ran `rake ecosystem:update_infrastructure[nub]`: adds nub to the CI matrix (ci.yml, images-branch/latest), ci/smoke filters + smoke-matrix, the issue labeler, bin/dry-run LOAD_PATH, script/dependabot volume mount, updater setup app_dirs_pattern, and the rakelib gemspec list.
Replace the invalid `nub update <pkg>@<ver>` (nub rejects a pinned version on `update`) with the same pin-then-install approach the FileUpdater uses: pin the candidate version into the manifests and re-resolve `nub install --lockfile-only`, so a peer-dependency conflict surfaces as a classifiable subprocess failure.
…ess) Addresses an adversarial-review nit: NubLock references NubPackageManager::NAME but relied on load order for the constant. Make the dependency explicit.
nub.lock is byte-compatible with pnpm-lock v9, so nub can depend on dependabot-npm_and_yarn and alias its byte-identical classes rather than maintaining clones (uv/pre_commit precedent). Aliases PackageName, VersionSelector, and FileUpdater::PackageJsonPreparer.
- Remove the unreferenced yarn-derived error-pattern block from nub.rb (VALIDATION_GROUP_PATTERNS/Utils/regexes were never consumed). - Reuse NpmAndYarn::PnpmErrorHandler in the lockfile updater's error path (nub emits pnpm-format engine errors), layered before nub's own patterns. - Delete the Yarn PnP vendor/artifact updater and the git-lfs .yarn pull (berry-only, dead for nub.lock). - Delete the dead npm6 JS helper tree (no npm6: call site).
…me-branch case Grounding nub-dev shows nub re-resolves a git dependency when the manifest ref changes (dependabot#2.0.0 -> dependabot#2.1.0): PackageJsonUpdater rewrites the ref and nub's lockfile-only install picks up the new tag's commit. Add a fixture + FileUpdater example proving this end to end. The same-branch SHA advance (manifest ref unchanged, e.g. master -> master) stays skipped with a precise reason: nub's conservative lockfile-only install keeps the locked SHA, and nub exposes no targeted force-re-resolution. Refine the metadata monorepo-directory skip reason to name the missing spec stub.
nub now depends on dependabot-npm_and_yarn (reused pnpm-format classes). The updater-core base only stubs sibling ecosystems, so without copying the full npm_and_yarn source the nub image would load its stub and the aliases would fail at require time. Mirror pre_commit/Dockerfile, which copies its gem deps.
…transitive/security updates nub 0.4.1 (dependabot#355) accepts `update <pkg>@<version>` for registry deps INCLUDING transitive/sub-dependencies not in any manifest. Replace the pin-restore double-install with a single `nub update <deps>@<vers> --lockfile-only --no-save` (mirrors npm_and_yarn's pnpm updater) + a settling install. This wires Dependabot's security-update path for deps-of-deps (previously a NoChangeError). Adds a transitive-update fixture + spec (bumps lodash, a transitive of fetch-factory, in the lockfile). Suite: 338 examples, 0 failures, 3 pending (same-branch git-SHA-advance ×2 + monorepo-dir metadata ×1 — documented).
colinhacks
changed the title
Sorbet (srb tc): - version_resolver: bind `changed` as T.let(false, T::Boolean) so the in-block reassignment (srb 7001) and the later `if changed` (7006, "unreachable") type-check; extract the pinnable-groups array literal to a constant (also fixes Performance/CollectionLiteralInLoop). - nub_lockfile_updater#git_dependency?: read the source via the typed DependencyRequirement#source reader and cast the value, so the strong-typed file no longer branches/calls on T.untyped (7018 x3). Rubocop: - nub_lock#dependencies: extract dependency_args to drop AbcSize below the threshold and remove the now-redundant Metrics/MethodLength disable. - spec skip() calls reformatted for FirstMethodArgumentLineBreak. - nub/.rubocop.yml: exclude the inherited-from-bun heterogeneous-JSON files (Sorbet/ForbidTUntyped) and stubbing/constant idioms (RSpec/StubbedMock, RSpec/AnyInstance, Lint/UselessConstantScoping), matching how bun and npm_and_yarn grandfather the identical code in the root .rubocop_todo.yml. These live in the per-gem config because the ForbidTUntyped burndown ratchet forbids adding new files to the root todo list.
setup.rb already requires "dependabot/nub", but updater/Gemfile never declared the gem, so every locally-built updater image failed to load with `cannot load such file -- dependabot/nub (LoadError)` — the root cause of the integration failure and the e2e smoke failures across all ecosystems. Add the path gem alphabetically and regenerate the lock (additive: one PATH block plus its DEPENDENCIES/CHECKSUMS entries).
The standalone form-data@2.3.3 (pulled by request) is flagged by Dependency Review (GHSA-fjxv-7rqg-78g4, GHSA-hmw2-7cc7-3qxx). Add an npm override to the patched 2.5.6 and regenerate the lock. The copy bundled inside npm@6.14.18 is inBundle (no resolved URL) so it is not separately fetched or flagged. lockfileVersion stays 2; `npm ci` verified under npm 8 and 10.
package_json_preparer, package_name, and version_selector are thin constant aliases to npm_and_yarn classes — no methods, no T.untyped — so they satisfy strong. The Sorbet job's `spoom srb bump --from strict --to strong --dry` ratchet requires any strict file that can be strong to be bumped.
Dockerfile.updater-core's ECOSYSTEM_LIST drives a RUN loop that stubs <dir>/lib/dependabot/<name>.rb for every ecosystem, so setup.rb's unconditional `require "dependabot/nub"` resolves inside every ecosystem image (each carries only its own real source). nub was missing, so all non-nub images raised `LoadError: cannot load such file -- dependabot/nub` at runtime — the cause of the integration + e2e failures. Add nub alphabetically. Also fill the other enumeration lists that missed nub: bin/dry-run.rb (require + valid_package_managers), bin/docker-dev-shell (source mounts), and .github/labeler.yml (javascript label globs).
Drop 10 npm_responses fixtures and 7 project directories that no spec
under nub/spec references. These were carried over wholesale when the
gem's fixture set was cloned; the nub spec suite is a trimmed subset and
never loads them.
The remaining fixtures include names referenced only via interpolation
(e.g. fixture("npm_responses", "#{escaped_dependency_name}.json")), which
were verified live and kept. Full suite unchanged: 338 examples, 0
failures, 3 pending.
The nub refactor dropped the npm6 parser; nothing under nub/helpers requires the `npm` package (lockfile parsing uses @npmcli/arborist). The only remaining form-data@2.3.3 flagged by Dependency Review (GHSA-fjxv-7rqg-78g4, GHSA-hmw2-7cc7-3qxx) was the copy bundled inside npm@6.14.18's tarball, which npm overrides cannot reach. Removing the dependency drops that copy and its bundled subtree (~7.5k lock lines) plus the now-orphaned npm++pacote patch (it patched npm's bundled pacote). The form-data override is kept for transitive copies. Verified: no form-data@2.3.3 in the lock (only 2.5.6); npm ci green; helper build + jest 14/14 pass without npm; nub rspec 338/0/3.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #15523.
What are you trying to accomplish?
First-party support for the nub package manager, which writes
nub.lock— byte-compatible with the pnpm lockfile v9 format. nub is distributed on npm as@nubjs/nub; dependencies resolve from the npm registry.Because
nub.lockis a pnpm-lock v9 document, the gem depends ondependabot-npm_and_yarnand reuses its pnpm-format classes rather than duplicating them — theuv→dependabot-python/pre_commit→dependabot-npm_and_yarnpattern. The nub-specific surface isnub.lockdetection,packageManager: "nub@x", and invoking thenubCLI.Supported update types:
nub update <name>@<version> --lockfile-only.#2.0.0→#2.1.0).Known limitation (documented; the two tests are skipped): advancing a git dependency pinned to a branch to that branch's latest commit. nub's lockfile-only install conservatively keeps the locked SHA and exposes no force-re-resolution for this case; tracked for a follow-up nub release. Git ref changes (above) work.
Diff composition
Of ~110.8k added lines: ~8k gem code (
nub/lib), ~6k specs, ~61.5k spec fixtures, and ~35k native helpers (nub/helpers), plus ~200 of Docker/gemspec/config glue. The fixtures and helpers follow the Bun ecosystem's standalone layout — Dependabot's most recent comparable ecosystem, cloned to the same shape. Spec fixtures the nub suite never loads have been pruned (~53k lines removed) so the tree carries only what the tests reference.Happy to split this into sequential PRs — package manager → parser → fetcher → updater, the shape the Bun review took (#11267–#11300) — if the maintainers prefer.
Anything you want to highlight for special attention from reviewers?
The gem reuses
dependabot-npm_and_yarn's pnpm classes (the lockfile-parser helper,PnpmErrorHandler) via a gemspec dependency plus aliases;nub/Dockerfilecopies the fullnpm_and_yarnlib so the aliases load in the production image (thepre_commitprecedent).How will you know you've accomplished your goal?
The
nubrspec suite is green: 338 examples, 0 failures, 3 pending (the git-branch-SHA-advance limitation above, and one monorepo source-directory metadata case). Abin/dry-run.rb nubproduces update PRs against a real nub-lockfile repository.Checklist