fix(relay-deploy): arm64 published binary cannot load, and the deploy runs twice

[bobleer]

Follow-up to #1801, which fixed the CRLF and Docker-config half. With those in place a real arm64 host got further and exposed two more defects: the published binary still could not start, and the deploy visibly ran twice.

Verified end to end on Ubuntu 24.04 arm64 hardware, not just reasoned about.

1. The arm64 published binary could never load

The release matrix builds x86_64 on ubuntu-22.04 but arm64 on ubuntu-24.04-arm. Measured against the real published 0.2.14 artifacts:

	requires	debian:bookworm-slim provides
x86_64 relay	GLIBC_2.35	2.36	works
arm64 relay	GLIBC_2.38	2.36	cannot load

Reproduced on the reporter's own machine, against the image their failed run had left behind:

$ docker run --rm --entrypoint /app/bitfun-relay-server bitfun-relay:release-v0.2.14
/app/bitfun-relay-server: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found
exit=1

The container exited instantly, so the deploy surfaced only as a failed health check followed by a ~20 minute source rebuild. x86_64 was unaffected, which is why it went unnoticed.

Fixed on both sides, because they have different blast radii:

• Deploy — runtime base is now debian:trixie-slim (glibc 2.41). This is the only fix that helps clients already in the wild.
• CI — arm64 now builds on ubuntu-22.04-arm, giving both arches one 2.35 floor. This is the only fix that helps anyone following the README's "Manual Run (without Docker)" path, who never touches Docker and is broken today.

debian:trixie-slim must stay even though CI now emits 2.35 binaries: Desktop pins BITFUN_RELEASE_TAG to its own version, so a v0.2.14 client installs the 2.38 archive forever. The base has to cover the highest floor still in the wild, not the floor CI produces from now on. Both files say so, to stop it being "optimised" back.

Two new guards, since nothing previously declared this contract:

• scripts/ci/check-glibc-floor.sh reads .gnu.version_r via readelf -V and fails the build above the declared floor. Verified against the real 0.2.14 archives: x86_64 passes at 2.35, arm64 fails at 2.35 reporting 2.38, arm64 passes at 2.38, and a mixed run reports each binary before failing.
• The runtime image greps ldd output and fails to build on an unloadable binary. Note ldd exits 0 while reporting an unsatisfied symbol version, so its output is the gate, not its status — my first attempt used RUN ldd and would never have fired. The relay binary is also useless as a probe: it has no --version and just starts serving.

2. The failure was invisible

docker logs --tail 40 bitfun-relay 2>/dev/null discarded the container's stderr — where both the relay's tracing output and the loader error above go. That is why the report showed a failed health check with no explanation at all. Both streams are kept now, plus container running/exit/OOM state so a crash is distinguishable from a container that is up but not answering.

3. The deploy ran twice

bitfun_run_deploy_sh is reached only after bitfun_try_release_deploy has failed, but it invoked deploy.sh with no arguments — and deploy.sh's own first step is that same release-binary path. It re-downloaded, rebuilt and restarted the binary that had just failed. It now passes --build-from-source, the flag that already existed for exactly this.

4. The CR strip no longer depends on a raw CR byte

#1801 shipped sed 's/<CR>$//' with a literal CR in the command string — which then travelled through to_unix_script on its way to the host, the CR remover passing through the CR remover. It worked only because the CR happened to be followed by $ rather than a newline. tr -d '\r' expands the escape itself, so the command is pure ASCII and cannot be damaged in transit. The rewrite also cleans up its scratch file when it fails.

Verified on real arm64 hardware

Generated scripts uploaded deliberately CRLF-corrupted (harsher than the original report: it dies at line 2 on set -euo pipefail\r, not line 37):

check	result
host-side CR strip	3879 CRs → 0, mode 700, no scratch file, both scripts parse
binary deploy	succeeds in ~15s, no source-build fallback
runtime image	glibc 2.41, ldd gate passes
container	Up (healthy) — docker's own HEALTHCHECK passes
/health	{"status":"healthy","account_features":true,...}
relay-admin	No accounts found.
compose labels / volumes / restart	preserved
re-deploy over a running relay	container replaced, backup renamed and cleaned up, new one healthy
--build-from-source	reaches deploy.sh in direct and sg modes

Forcing the fixed Dockerfile back to bookworm-slim fails the build with the exact loader error, confirming the gate fires.

An honest correction

#1801 also quoted arguments for sg docker -c. I implied that was related to the unknown shorthand flag: 't' in -t failure. Tested against the real sg binary, it was not — the old unquoted form handles that particular command fine. It does genuinely break on multi-word arguments:

old: docker inspect -f {{.Name}} {{.State.Status}} ...  -> error: no such object: {{.State.Status}}
new: docker inspect -f '{{.Name}} {{.State.Status}}' ... -> /bitfun-relay running

So it is worth having — the new container-state diagnostic in this PR uses exactly such a template and would break in sg mode without it — but it was not the cause of the reported bug. The root cause there was the root-owned config.json, already fixed in #1801 and confirmed by the reporter's follow-up run.

Reviewer note

ubuntu-22.04-arm is documented as available for public repositories, but the matrix change has not been executed — this workflow only runs via workflow_call from a release. If the label were unavailable the job would fail loudly rather than silently, and the fallback is to keep ubuntu-24.04-arm and build inside a debian:bookworm container instead.