Skip to content

feat(audit): PS-169 — GitHub-hosted runners are forbidden, no exceptions - #344

Closed
ywatanabe1989 wants to merge 1 commit into
developfrom
ci/ps169-forbid-hosted-runners
Closed

feat(audit): PS-169 — GitHub-hosted runners are forbidden, no exceptions#344
ywatanabe1989 wants to merge 1 commit into
developfrom
ci/ps169-forbid-hosted-runners

Conversation

@ywatanabe1989

Copy link
Copy Markdown
Collaborator

Operator mandate (2026-07-14): 「PR用のテストとgithub側のランナーというのは本当にもう一切使わないでください」「もし使っているパッケージがあれば…リンター、フックでエラーにしてください。強制です、例外なしです

This rule is the only enforcement that exists

Blocking GitHub-hosted runners at the org level is an Enterprise Cloud policy, and scitex-ai is on the Free plan (/orgs/scitex-ai/actions/permissions controls which actions may run, never which runner). There is no backstop behind this check — which is why it ships at severity E, not warn-first.

Why this is not a grep ubuntu-latest

The runner is routinely indirect. Only the first of these is greppable on the runs-on: line:

runs-on: ubuntu-latest          # direct
runs-on: [ubuntu-22.04]         # list form
runs-on: ${{ matrix.os }}       # hides in strategy.matrix.os
runs-on: ${{ inputs.runner }}   # hides in a workflow_call input default

PS-169 resolves the effective runner, following matrix.* and inputs.* back to their declared values before judging. A literal line-scan would happily pass a workflow that runs every job on ubuntu-latest via a matrix — which is precisely how the violation hides. An unresolvable runner is also a violation: it cannot be proven Spartan-only, and absence of evidence is not evidence of compliance.

Consuming the shared workflow is not, by itself, compliance

A repo can be a thin caller of scitex-ai/.github and still run hosted if the callee is dirty. Closed structurally: the SSOT declares its runs-on inline, so this same rule — run against scitex-ai/.github — fires on a dirty shared workflow at source. Callers of reusable workflows from owners outside scitex-ai are flagged, since a third party’s runner cannot be vouched for.

Proof it fires (executed, not declared)

target PS-169 violations
scitex-ai/.github @ main (the real, dirty SSOT) 3
scitex-ai/.github @ fixed branch (.github#4) 0
scitex-dev 4
figrecipe 6
newb 7
scitex-agent-container 10

The 3 found in the dirty SSOT match an independent manual audit exactly (auto-merge-to-develop.yml ×1, cla.yml ×2).

12/12 unit tests green, covering every runner form: direct, list, matrix indirection, workflow_call input default, unresolvable expression, unparseable YAML (a broken file must not smuggle a runner past the gate), plus the compliant Spartan and trusted-caller cases that must stay silent.

Notes

  • Registered in _extra_rules.py (the sanctioned sidecar) — _registry.py is already over the 512-line cap, and a 1200-line split would collide with in-flight work in this repo.
  • Severity E means it fails scitex-dev ecosystem audit-all, i.e. the quality-audit workflow. That is the CI enforcement path; the pre-commit path is the natural follow-up.
  • Ecosystem impact, stated plainly: 67 of 68 repos currently violate this rule (809 hosted-runner references). Merging PS-169 turns their quality-audit red until the CI migration lands. Sequence it with the rollout — that is a deliberate ordering decision for the operator, not an accident.

Operator mandate (2026-07-14): 「PR用のテストとgithub側のランナーというのは
本当にもう一切使わないでください」「強制です、例外なしです」— never use a
GitHub-hosted runner, including for PR tests; any package still using one must
be a hard ERROR.

This rule is the ONLY enforcement that exists. Blocking hosted runners at the
org level is a GitHub Enterprise Cloud policy and `scitex-ai` is on the Free
plan, so there is no backstop behind this check. Hence severity E.

Why it is not a grep for `ubuntu-latest`
----------------------------------------
The runner is routinely INDIRECT, and only the first of these is greppable on
the `runs-on:` line:

    runs-on: ubuntu-latest            # direct
    runs-on: [ubuntu-22.04]           # list form
    runs-on: ${{ matrix.os }}         # hides in strategy.matrix.os
    runs-on: ${{ inputs.runner }}     # hides in a workflow_call input default

PS-169 resolves the EFFECTIVE runner — it follows `matrix.*` and `inputs.*`
back to their declared values before judging. A literal line-scan would pass a
workflow that runs every job on ubuntu-latest via a matrix, which is exactly
how the violation hides. An UNRESOLVABLE runner is also a violation: it cannot
be proven Spartan-only, and absence of evidence is not evidence of compliance.

Consuming the shared workflow is not, by itself, compliance: a repo can be a
thin caller of scitex-ai/.github and still run hosted if the CALLEE is dirty.
That is closed structurally — the SSOT's own `runs-on` is audited by this same
rule in its own repo, so a dirty shared workflow fires at source. Callers of
reusable workflows from owners outside `scitex-ai` are flagged, since a third
party's runner cannot be vouched for.

Proof it fires (not merely declared):
  - scitex-ai/.github @ main         -> 3 violations (the real dirty SSOT)
  - scitex-ai/.github @ fixed branch -> 0
  - scitex-dev 4 | figrecipe 6 | newb 7 | scitex-agent-container 10
  - 12/12 unit tests green, covering every runner form incl. matrix
    indirection, workflow_call input defaults, unparseable YAML, and the
    compliant Spartan + trusted-caller cases (which must stay silent).

Registered in _extra_rules.py (the sanctioned sidecar) rather than _registry.py,
which is already over the 512-line cap.
@ywatanabe1989

Copy link
Copy Markdown
Collaborator Author

Closing under operator's 3-day PR-expiry rule (all repos, 2026-07-18). Its intent + reland priority are captured on card scitex-dev-closed-stale-pr-intents-rebuild-fresh-20260718 — rebuild fresh off current develop rather than rebase this stale branch. Branch is retained.

@github-actions github-actions Bot locked and limited conversation to collaborators Jul 18, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant