Skip to content

runner: resolve benchmarks at runtime, drop the allowlist #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
adriangb merged 1 commit into from
Jun 9, 2026
Merged

runner: resolve benchmarks at runtime, drop the allowlist #15

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,11 @@ show benchmark queue

## Supported Benchmarks

There is no allowlist: any name you pass is scheduled and resolved on the
runner. Names that match a Criterion `[[bench]]` target run via Criterion;
everything else runs through `bench.sh`. A name that matches neither fails on
the runner. The tables below list the benchmarks known to work today.

### DataFusion (standard)

| Benchmark | Description |
Expand Down Expand Up @@ -190,7 +195,7 @@ controller/ Rust controller crate
github_poller.rs Comment polling loop
job_manager.rs K8s Job lifecycle reconciler
db.rs SQLite queries (jobs, seen comments, scan state)
benchmarks.rs Trigger parsing, benchmark allowlists
benchmarks.rs Trigger parsing (no allowlist — any name is accepted)
migrations/ SQLite schema
runner/ Benchmark runner container (builds project, runs benchmarks, posts results)
queries/ SQL query files for ClickBench
Expand Down Expand Up @@ -293,7 +298,7 @@ One-time setup for GitHub Actions → GCP authentication:
| `github_poller` | Poll GitHub API, detect triggers, insert jobs |
| `job_manager` | Create K8s Jobs, monitor status, post results |
| `db` | SQLite persistence (jobs, seen comments, scan state) |
| `benchmarks` | Trigger parsing, per-repo allowlists, classification |
| `benchmarks` | Trigger parsing (no allowlist; benchmarks resolved on the runner) |
| `github` | GitHub REST API client (comments, reactions) |
| `config` | Environment-based configuration |

Expand Down
Loading
Loading