Skip to content

Fix benchmarks#993

Closed
jordanrule wants to merge 1 commit into
rue-language:trunkfrom
jordanrule:trunk
Closed

Fix benchmarks#993
jordanrule wants to merge 1 commit into
rue-language:trunkfrom
jordanrule:trunk

Conversation

@jordanrule

Copy link
Copy Markdown

Fix benchmark workflow trigger model + collector reliability. Fixes: #991

This PR fixes two benchmark CI issues:

  1. Benchmarks no longer run on a 15-minute schedule

    • Removed schedule trigger from .github/workflows/benchmarks.yml
    • Benchmarks now run per-commit (push to trunk) and via workflow_dispatch
    • Keeps concurrency.cancel-in-progress: true so newer commits supersede stale runs
  2. Benchmark collection path is no longer brittle

    • Checkout now pins to ${{ github.sha }} (instead of floating trunk) in benchmark and collector jobs
    • Added explicit failure when expected benchmark artifacts are missing
    • Fixed platform extraction from artifact names:
      • benchmark-results-<sha>-x86-64-linux now maps to x86-64-linux (previous logic collapsed to linux)
    • Fixed shell syntax in "Commit and push to perf branch" (missing fi)
    • Removed schedule-specific decision branches/messages that no longer apply

Local verification performed

  • YAML validation:
    • ruby -e 'require "yaml"; YAML.load_file(".github/workflows/benchmarks.yml"); puts "YAML parse OK"'
    • Result: YAML parse OK
  • Platform parsing sanity check:
    • Verified expected extraction for x86-64-linux, aarch64-linux, aarch64-macos
  • Attempted end-to-end benchmark run:
    • ./bench.sh --iterations 1 --output /tmp/rue-bench-results.json --no-history
    • In this local environment, the run stalled in buck2 build before producing results JSON

Why this should fix the CI failures

  • The collector now matches artifact names deterministically per commit SHA.
  • Platform keys now match website/perf history file naming conventions.
  • Broken shell block no longer causes collector step failure.
  • Trigger behavior is aligned with per-commit benchmarking (no redundant schedule path).

@jordanrule jordanrule closed this by deleting the head repository Jun 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Benchmarks are busted

1 participant