fix(runner-availability): scope detection to runner groups, fail loud

[kennethsinder]

What broke

runner-availability.yml read a single unpaginated page of /orgs/{org}/actions/runners.

Managed-runner vendors (Blacksmith, Namespace) register one ephemeral runner per job into their own runner groups and never deregister them. That list had grown to 1015 registrations for burin-labs, 1002 of them Blacksmith ephemerals, 997 of those offline. The org's own runners sort to index 1002+, on the last page.

So the probe saw 100 vendor ephemerals, matched none, and reported:

runner-availability linux=false (0 idle Linux harn-ci-stable runner(s))
runner-availability macos=false (0 idle macOS harn-ci runner(s))

Ground truth at that same moment, across all pages: 9 idle Linux, 4 idle macOS.

Every caller then routed onto paid GitHub-hosted runners. Nothing warned, because "I could not see the fleet" and "the fleet is busy" produced identical output.

Cost

burin-code net Actions spend: $515.56 (May) → $1,080.82 (Jun) → $2,379.15 (Jul 1-18), with a Jul 14-17 run rate of $160.50/day, a ~$4,815/mo pace. macOS alone was $1,053 of July while a dedicated idle CI Mac sat unused.

The fix

Query the declared runner groups rather than the flat org list. Request count is now proportional to the number of groups, not to vendor fleet size. Paginating the flat list would only have deferred this: the ephemeral fleet grew by roughly 170 stale registrations per day, so a paginated probe would need ~150 calls per run within three months.

Groups come from a new runner_groups input (default Default,burin-labs-mac-mbp), so callers declare their own capacity rather than inferring it.

Second defect fixed independently: every fall-through is now loud. An unresolvable group name, an API error, and a group set containing no runners are warnings rather than silent downgrades to the expensive path. A probe that cannot see the fleet must never be indistinguishable from a busy fleet.

Verification

Executed the exact script body against the live burin-labs org, 8 cases:

Case	Result
Happy path	13 runners observed, linux=true macos=true
Missing/renamed group	loud warning, lists known groups
No token	hosted fallback, notice
401 bad credentials	loud warning with stderr excerpt
Vendor group only	340 observed, correctly 0 matches
Whitespace in input	trimmed, linux=true macos=true
Empty/trailing commas	tolerated
Single group	macos=true only, correct

actionlint clean, shellcheck clean, YAML valid.

Note: the whitespace case initially failed. jq's sub replaces only the first match, so "^\\s+|\\s+$" left trailing whitespace and the group name did not resolve. Fixed to gsub and re-verified.

Rollout

Callers pin this by SHA, so merging changes nothing until burin-code and harn-cloud re-pin. Those follow separately so the routing change can be observed on its own.

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.}

[kennethsinder]

Correction to this PR's justification

I overstated one claim and want it on the record before it propagates.

I wrote that the flat-list approach "would only have deferred the same failure, since the ephemeral fleet grew by roughly 170 stale registrations per day." That growth framing is wrong.

GitHub auto-removes ephemeral self-hosted runners that have not connected for more than 1 day (docs). So Blacksmith's registrations self-reap on a 24h cycle, and the fleet is bounded at roughly one day's job churn, not accumulating without bound. Measured over 45 minutes this afternoon it moved 1002 → 1017, which is live churn under load, not stale accumulation.

What this changes: gh api --paginate on the flat list would in fact have held up long-term, at a stable ~11-15 requests per probe rather than degrading toward 150.

What it does not change: group scoping is still the better mechanism, on narrower grounds than I claimed:

• 2 requests instead of ~11-15, per probe, on a job that runs ~400x/day
• correctness does not depend on a vendor's reaping cadence or on how many jobs they happen to be running when we probe
• it states the intent directly (these groups hold our capacity) rather than filtering a list that mixes ours with theirs

The fix stands. The reasoning behind it was sloppier than the fix.

Also worth recording, from a sweep of prior art: there is no maintained off-the-shelf action to adopt here, and no native GitHub feature. The one live candidate (mikehardy/runner-fallback-action) does a single unpaginated GET with a default page size of 30 and has no runner-groups support, so it fails on exactly this defect. GitHub declined native self-hosted-to-hosted fallback on the record in community discussion #20019 and has shipped nothing since that changes it.