fix: propagate failures in bench-ci and bench-sync#8
Merged
Conversation
1.tee swallowed go test exit code → set pipefail 2.; chain hid bench-sync errors → set -e + trap cleanup
There was a problem hiding this comment.
Pull request overview
This PR fixes two failure-propagation bugs in the repository’s benchmark automation so benchmark-related Make targets correctly return non-zero when the underlying benchmark run or sync step fails. That aligns the Makefile behavior with the rest of the project’s developer workflow and prevents benchmark CI from silently reporting success.
Changes:
- Set
SHELL := /bin/bashso the benchmark recipes can rely onpipefailandtrap. - Updated
bench-cito useset -o pipefail, ensuringgo testfailures are not masked bytee. - Updated
bench-syncto useset -eplus anEXITtrap so failures propagate while temporary files are still cleaned up.
1.> /dev/null hid 60s of bench output → unsilence 2.No phase markers → echo headers around the steps
1.Hardcoded /bin/bash broke on Nix → use PATH lookup
1.Global SHELL forced bash on every recipe → target-scoped 2.POSIX recipes now usable in /bin/sh-only environments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
`make bench-ci` and `make bench-sync` could exit 0 even when the underlying
`go test` or `benchsync` command failed, because:
itself fails). A failed bench would still report green.
any earlier failure exit from `bench-ci` or `go run ./cmd/benchsync`.
Both bugs were caught by GitHub Copilot during a review of three downstream
PRs in another repo (wspulse) that had ported this Makefile pattern. After
the wspulse fixes landed, CI immediately surfaced a real goroutine leak that
had been silently passing under the broken target. Fixing here prevents
future ports from inheriting the same trap.
Related issues
(none)
Changes
in recipes. All existing recipes are POSIX-compatible so this is safe.
failures.
file is always cleaned up but a real failure in `bench-ci` or
`go run ./cmd/benchsync` exits non-zero.
Checklist
Required