fix(release,#339): bundle deps before erasing the RUNPATH that finds them - #340
Merged
Conversation
…them
The v0.8.27 HIP tarball never packaged. The build succeeded;
check-bundled-deps.py refused the staged directory:
crispasr needs libomp.so
crispasr-quantize needs libomp.so
bundle-linux-runtime.sh did its two jobs in the wrong order. It rewrote
RUNPATH to $ORIGIN first, and only then asked ldd what the binaries
needed — but ldd resolves through the binary's own RUNPATH, so erasing it
first turns exactly those dependencies into `=> not found`, and the copy
loop's `grep '^/'` discarded them along with the blank lines. ROCm's
clang links OpenMP against LLVM's libomp.so in /opt/rocm/lib/llvm/lib,
reachable only via that RUNPATH. gcc's libgomp.so.1 sits in the default
loader path, which is why every non-HIP leg was unaffected and this
stayed hidden.
Resolve first, rewrite second, and scan a copied library at its original
path rather than at the staged copy — a $ORIGIN-relative RUNPATH means
something different once the file has moved.
An unresolved dependency is now fatal in the bundler instead of silent.
It previously printed "rpaths normalised, 0 librar(ies) bundled" — a
green line over a dropped dependency — and left it to a later gate to
notice. The fatal check consults the same exclusion list the copy loop
uses, so a host-provided runtime that is legitimately absent from a CI
runner (libcuda.so.1 on the CUDA legs) still does not fail the release.
tests/test-bundle-linux-runtime.sh reproduces all of this with cc and a
private directory — no ROCm, no GPU, no release. Registered as a unit
test because these packaging scripts previously ran only inside a
release job, so a defect in them could not be observed without
publishing one, and two shipped that way.
Red-verified on Linux against the shipped bundler:
control (v0.8.27 script): FAIL — "rpaths normalised, 0 librar(ies)
bundled", dependency dropped
fixed: PASS — bundled, and the relocated binary runs
with its build-time libdir deleted
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test-bundle-linux-runtime SKIPs without it, and a skip reads exactly like a pass in the ctest summary — which is the same blindness that let the packaging scripts ship broken twice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The tag input documents "LEAVE EMPTY for a dry run", and then this job compared VERSION against the branch name and failed — so every dry run was red before the job you dispatched it for had finished, exactly when you are trying to read one job's result. Nothing to verify when nothing is published. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every Windows, Android and libcrispasr artifact already carried both. No CLI tarball did — not the seven Linux legs, not macOS — while the Linux ones bundle third-party shared libraries (libgomp today, libomp on HIP once #339's bundler fix lands) whose terms require the notice to travel with the binary. Added after the dependency gate and before the tar, so it cannot perturb the checks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
bundle-c2pa.sh drops libc2pa_c.so into the directory before the bundler runs, and crispasr-quantize used to carry no $ORIGIN in its RUNPATH at all — so `ldd` reports a library that is sitting right there as `not found`. Making an unresolved dependency fatal without this exemption would have failed every Linux leg on the next release. Fourth arm added: the file is staged but unreachable, the bundler must not fail, and the binary must run once step 2 grants it $ORIGIN. All four arms green on Linux; the control still red at arm 1. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…eps too Same defect as #339, two more artifact classes, neither reported. libcrispasr-linux-x86_64-hip as published in v0.8.27: libcrispasr.so.0.8.27 needs libomp.so libggml-base.so needs libomp.so libggml-cpu.so needs libomp.so libggml-hip.so needs libhipblas.so.2 crispasr-python-linux-x86_64: libwhisper.so needs libgomp.so.1, libblas.so.3 libggml-base.so needs libgomp.so.1 Both verified against the downloaded release assets, not inferred. WHY NOTHING CAUGHT IT. The libs bundles never ran check-bundled-deps.py; verify-lib-bundle.sh gates INTRA-bundle resolvability, which is a different question. Worse, its dlopen DID fail on libomp.so — and libomp is on its EXTERNAL tolerance list, so it printed "dlopen deferred: external driver absent in CI" and exited 0, skipping the rest of the check as well. That tolerance is sound for gcc's versioned libgomp.so.1 on the default loader path; ROCm clang's unversioned libomp.so lives only under /opt/rocm/lib/llvm/lib. Comment corrected in place rather than left to mislead the next reader. The python legs did the RUNPATH half inline and had no closure step and no gate at all, so `import crispasr` died in the loader on any host without OpenBLAS and gcc's OpenMP. - package-lib-bundle.sh copies the closure before rewriting rpaths, operating on the flattened lib/ so bundle-provided sonames are skipped by name and the SOVERSION symlinks are not overwritten. Linux branch only; macOS is untouched. - The five Linux libs legs and both python legs now run check-bundled-deps.py, with the same per-leg --allow contracts the CLI legs use. - tests/test-package-lib-bundle.sh builds a bundle shaped like a real one — SOVERSION symlinks, src/ + ggml/src/, one dependency reachable only via RUNPATH — packages it, deletes the toolchain dir, and dlopens the relocated result. Red-verified against the v0.8.27 script. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Linux archives bundle libgomp.so.1 and, on HIP, libomp.so, and the notices file mentioned neither. It also still claimed OpenBLAS was "not bundled … users must have libopenblas installed", which #296 changed two releases ago. CrispASR's own source stays MIT. The GCC Runtime Library Exception is what makes an MIT binary linking libgomp unproblematic, but shipping the libgomp.so.1 FILE is separately a conveyance of a GPLv3 work and carries GPLv3 §6's corresponding-source obligation — so there is now a source pointer and a written offer. LLVM's libomp is Apache-2.0 WITH LLVM-exception: permissive, notice only. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the remaining half of #339:
crispasr-linux-x86_64-hip.tar.gzstill failed to package in v0.8.27, for a reason the earlier fix could not have caught.The build succeeds;
check-bundled-deps.pythen refuses the staged directory withcrispasr needs libomp.so.bundle-linux-runtime.shrewroteRUNPATHto$ORIGINbefore askinglddwhat the binaries needed.lddresolves through the binary's ownRUNPATH, so erasing it first turns exactly those dependencies into=> not found, and the copy loop'sgrep '^/'dropped them with the blank lines. ROCm's clang links OpenMP against LLVM'slibomp.sounder/opt/rocm/lib/llvm/lib— reachable only that way. gcc'slibgomp.so.1is in the default loader path, which is why six legs were unaffected.$ORIGIN-relative RUNPATH means something else once the file has moved.rpaths normalised, 0 librar(ies) bundled— a success line with a count, over a dropped dependency. The check consults the same exclusion list the copy loop uses, solibcuda.so.1(legitimately absent from a driverless runner) still does not fail the CUDA legs.tests/test-bundle-linux-runtime.shreproduces all of it withccand-Wl,-rpath— no ROCm, no GPU, no release — in theunittier. These packaging scripts previously ran only inside a release job, which is why two defects in them shipped.patchelfadded to the CI unit job: without it the test SKIPs, and a skip reads like a pass.validate-versionno longer fails on dry runs (emptytag), the mode that input's own documentation recommends.Red-verified on Linux against the script as shipped in v0.8.27:
A dry run of the HIP leg alone is in flight; this is not merged on the unit test alone.
🤖 Generated with Claude Code