Skip to content

fix(release): tolerate missing baseline package versions#126

Merged
linfangw merged 1 commit into
mainfrom
linfangw-release-baseline-fallback-2026-04-20
Apr 20, 2026
Merged

fix(release): tolerate missing baseline package versions#126
linfangw merged 1 commit into
mainfrom
linfangw-release-baseline-fallback-2026-04-20

Conversation

@linfangw

Copy link
Copy Markdown
Member

Summary

  • Problem: ./scripts/release-npm.sh could fail in the final install-smoke stage when scripts/test-install-sh-docker.sh tried to pack a hard-coded baseline version that was never published for the QVerisBot fork.
  • Why it matters: release validation for the fork should not hard-fail just because @qverisai/qverisbot@2026.4.10 does not exist on npm yet.
  • What changed: update-smoke baseline resolution now looks up published versions for the current package, falls back to the most recent usable published baseline when the requested one is unavailable, and skips only the update lanes when no published baseline exists at all.
  • What did NOT change (scope boundary): fresh/install smoke behavior, the packaged artifact under test, and the actual publish/tag flow are unchanged.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

  • Closes #N/A
  • Related #N/A
  • This PR fixes a bug or regression

Root Cause (if applicable)

  • Root cause: scripts/test-install-sh-docker.sh assumed the configured baseline version already existed on npm and unconditionally ran npm pack "${PACKAGE_NAME}@${UPDATE_BASELINE_VERSION}", which is false for the fork's unpublished historical versions.
  • Missing detection / guardrail: the script had no fallback path for unpublished baseline versions and no test coverage for fork-specific package histories with sparse or empty published version sets.
  • Contributing context (if known): the upstream flow assumes a long-lived published package line, but the QVerisBot fork has a shorter npm publication history.

Regression Test Plan (if applicable)

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file: test/scripts/test-install-sh-docker.test.ts
  • Scenario the test should lock in: the Docker install smoke script must fall back to another published baseline version when the requested one is unavailable, and it must skip only the update lanes when no published baseline exists.
  • Why this is the smallest reliable guardrail: the failure is in shell-script control flow and package-version selection, so a focused script test is enough to lock in the intended behavior without needing a full Docker publish simulation.
  • Existing test that already covers this (if any): the existing script tests already covered update package specs and pack budget checks for this same script.
  • If no new test is added, why not: N/A

User-visible / Behavior Changes

None for end users.

Release behavior changes:

  • install-smoke no longer hard-fails just because the requested baseline package version is missing from npm.
  • when possible, it falls back to a real published baseline version and logs the fallback.
  • when no baseline exists at all, only the update lanes are skipped.

Diagram (if applicable)

Before:
[release smoke] -> [pack hard-coded baseline version] -> [npm ETARGET] -> [release fails]

After:
[release smoke] -> [resolve published baseline]
                 -> [fallback to published version] -> [run update smoke]
                 -> or [none published] -> [skip only update lanes]

Security Impact (required)

  • New permissions/capabilities? (No)
  • Secrets/tokens handling changed? (No)
  • New/changed network calls? (No)
  • Command/tool execution surface changed? (No)
  • Data access scope changed? (No)
  • If any Yes, explain risk + mitigation: N/A

Repro + Verification

Environment

  • OS: macOS
  • Runtime/container: local checkout, Node 24.2.0, pnpm, Docker-based install smoke script
  • Model/provider: N/A
  • Integration/channel (if any): npm release/install smoke for @qverisai/qverisbot
  • Relevant config (redacted): requested update baseline defaults to 2026.4.10, but that version is not published for the fork package

Steps

  1. Run ./scripts/release-npm.sh on the fork.
  2. Let it reach the install-smoke update baseline packing step.
  3. Observe the old script try to pack @qverisai/qverisbot@2026.4.10 and fail with npm ERR! code ETARGET.

Expected

  • release validation should continue when the exact requested baseline version is missing, using a published fallback or skipping only update lanes when no baseline exists.

Actual

  • before this PR, the release flow aborted at baseline tarball packing.
  • after this PR, the script resolves a usable published baseline or degrades cleanly.

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios: bash -n scripts/test-install-sh-docker.sh and node scripts/run-vitest.mjs run test/scripts/test-install-sh-docker.test.ts
  • Edge cases checked: requested baseline missing, fallback baseline logging, no published baseline available, and avoiding empty baseline URL output.
  • What you did not verify: a full end-to-end rerun of ./scripts/release-npm.sh through Docker publish smoke after this patch.

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

  • Backward compatible? (Yes)
  • Config/env changes? (No)
  • Migration needed? (No)
  • If yes, exact upgrade steps: N/A

Risks and Mitigations

  • Risk: fallback could choose a baseline version that is older than the originally intended default.
    • Mitigation: the script logs the requested and resolved baseline versions explicitly, and only falls back within the package's actual published versions.
  • Risk: skipping update lanes when no published baseline exists reduces release validation breadth for brand-new package lines.
    • Mitigation: fresh/install smoke still runs, and skipping only update lanes is safer than failing the entire release for an unavailable historical artifact.

@linfangw
linfangw merged commit 6f647bc into main Apr 20, 2026
2 of 9 checks passed

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a fallback mechanism for the update baseline version in the install smoke tests, allowing the script to select an alternative published version if the requested one is unavailable or skip the update lane gracefully if no baseline exists. Feedback focuses on improving the robustness of the version resolution by using stdin for Node.js processing to avoid command-line length limits and refining the fallback selection logic to ensure it strictly tests upgrades rather than potential downgrades.


resolve_update_baseline_version() {
local versions_json
versions_json="$(quiet_npm view "$PACKAGE_NAME" versions --json 2>/dev/null || true)"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Passing the output of npm view directly as a command-line argument to node -e can be risky if the package has a very large number of versions, as it might exceed the maximum command-line length limit (ARG_MAX). A safer approach would be to pipe the JSON string to stdin and read it within the Node.js script.

process.stdout.write(requested);
process.exit(0);
}
const fallback = [...versions].reverse().find((value) => value !== expected) ?? "";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The fallback logic currently picks the most recently published version that is not the expected version. While this is generally correct, it doesn't guarantee that the fallback is older than the current version. If a newer version was published from another branch, the update smoke test would effectively be testing a downgrade. Consider filtering for versions that are semantically older than the expected version if the intent is strictly an upgrade test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant