Skip to content

feat(runner): add native built-in gate policy - #34

Draft
jesse-merhi wants to merge 1 commit into
mainfrom
jesse/openclaw-native-gate-policy
Draft

feat(runner): add native built-in gate policy#34
jesse-merhi wants to merge 1 commit into
mainfrom
jesse/openclaw-native-gate-policy

Conversation

@jesse-merhi

Copy link
Copy Markdown
Member

New behavior

Trusted profiles can now opt built-in SkillSpector and clawscan-static into native gate policy without redefining either scanner as a command. ClawScan derives pass, warn, or block from their unchanged raw JSON and does not require a judge.

  • SkillSpector recommendation DO_NOT_INSTALL or any CRITICAL finding blocks.
  • A SkillSpector HIGH finding warns.
  • Any clawscan-static finding warns and never blocks.
  • Native and exit-code rules compose on the same scanner; the strongest action wins.
  • The embedded clawhub profiles now enable the shipped native policy.

Profile and artifact proof

A built-in scanner reference can carry policy while remaining backed by the registered adapter:

scanners:
  - id: skillspector
    gate:
      native: true
  - id: clawscan-static
    gate:
      native: true

The observable verdict matrix is:

Scanner evidence Gate
SkillSpector SAFE or CAUTION without HIGH/CRITICAL findings pass
SkillSpector HIGH finding warn
SkillSpector DO_NOT_INSTALL recommendation block
SkillSpector CRITICAL finding, including after the 25-item prompt display cap block
Any clawscan-static finding, including high severity warn

Fired native rules identify the matched recommendation or finding through value, findingCode, findingTitle, and findingSeverity. Existing exit-code rules retain exitCode. Raw scanner evidence is byte-identical with native policy on and off.

How to verify

  1. Resolve a trusted profile containing the YAML above and confirm both scanner IDs use the built-in registry adapters.
  2. Run fixture-backed SkillSpector reports for SAFE, CAUTION, HIGH, CRITICAL, and DO_NOT_INSTALL; inspect top-level gate and gateRules.
  3. Run clean and finding-bearing static reports and confirm findings produce only warn.
  4. Return valid SkillSpector JSON with a gate-eligible nonzero exit and confirm native plus exit-code rules both fire while the strongest action is recorded.

Checks

  • go test -count=1 ./internal/profiles — passed.
  • Focused native gate, raw-evidence, exit-code composition, normalization, and CLI summary tests — passed.
  • env TMPDIR=/private/tmp go test -count=1 ./... — passed.
  • go vet ./... — passed.
  • make docs-site — passed; no tracked dist/ changes.

Implementation notes

This deliberately does not add generic JSONPath/BYOS finding mappings. Command-defined scanners continue to participate through blockOnExitCode and warnOnExitCode. Gate actions remain record-only and do not alter scan ordering, judge execution, or process exit status.

@jesse-merhi
jesse-merhi force-pushed the jesse/openclaw-native-gate-policy branch from 422c7a3 to b9605f9 Compare July 27, 2026 15:30
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. labels Jul 27, 2026
@clawsweeper

clawsweeper Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed July 27, 2026, 11:33 AM ET / 15:33 UTC.

ClawSweeper review

What this changes

This PR adds opt-in native gate evaluation for the registered SkillSpector and clawscan-static scanners, records the matched recommendation or finding in artifacts and CLI summaries, and enables the policy in embedded ClawHub profiles.

Merge readiness

⚠️ Ready for maintainer review - 4 items remain

Keep this draft PR open for normal member review. The proposed native policy is a new public profile/configuration surface and changes the embedded ClawHub profile defaults, so it needs an explicit compatibility and product-direction check rather than cleanup automation. Likely related people: jesse-merhi is the available low-confidence routing candidate from the proposed runner/profile work.

Priority: P3
Reviewed head: b9605f91a09b7d158d58dda4c36feb3497edaee1
Owner decision: Required. See Decision needed.

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) The patch is focused and test-backed in the supplied context, but its shipped-profile compatibility and default-policy intent still need maintainer confirmation.
Proof confidence 🌊 off-meta tidepool Not applicable: This is a member-authored draft PR, so the external-contributor real-behavior-proof gate does not apply; the PR nevertheless reports focused tests and full-suite checks.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Not applicable Not applicable: This is a member-authored draft PR, so the external-contributor real-behavior-proof gate does not apply; the PR nevertheless reports focused tests and full-suite checks.
Evidence reviewed 5 items Runner and artifact implementation: The proposed patch adds Native policy support and extends fired gate-rule records so native recommendation/finding matches can be recorded without changing the raw scanner payload.
Profile resolution change: The proposed patch accepts object-form built-in scanner references with gate.native: true and maps them back to registered built-in adapters rather than requiring a command-backed scanner.
Embedded profile behavior: The proposed patch changes the shipped ClawHub and ClawHub-AIG profiles from plain scanner IDs to native-gate-enabled object references.
Findings None None.
Security None None.

How this fits together

ClawScan profiles resolve scanner selections and gate rules into registered scanner adapters. Scanner runs preserve raw JSON evidence, then gate evaluation records the strongest pass, warn, or block outcome in the run artifact and CLI summary.

flowchart LR
  A[Trusted profile] --> B[Profile resolver]
  B --> C[Registered scanner adapters]
  C --> D[Raw scanner JSON]
  D --> E[Native and exit-code gate rules]
  E --> F[Run artifact]
  F --> G[CLI gate summary]
Loading

Decision needed

Question Recommendation
Should the embedded clawhub and clawhub-aig profiles begin emitting native warn/block gate outcomes by default, or should native policy remain opt-in only for custom trusted profiles? Enable in shipped profiles: Land the proposed embedded-profile defaults after documenting the new artifact verdict behavior and showing representative upgrade evidence.

Why: The implementation is technically bounded, but changing shipped profile defaults can alter artifact verdicts consumed by existing operator workflows; that default belongs to maintainer product policy.

Before merge

  • Resolve merge risk (P1) - Enabling native rules in the embedded ClawHub profiles changes the recorded gate verdict for existing profile users; any downstream automation consuming gate or gateRules needs upgrade evidence that the new warnings and blocks are intentional.
  • Resolve merge risk (P1) - FiredGateRule.exitCode changes from an always-present integer to an optional field so native rules can coexist with exit-code rules; artifact consumers should be checked for assumptions that every fired rule includes an integer exit code.
  • Complete next step (P2) - A maintainer must decide whether the embedded ClawHub profiles should change their default recorded gate behavior; this member-authored draft is not a cleanup or automated repair candidate.
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Patch surface 9 files affected; 521 additions, 36 deletions The feature spans public profile configuration, runner artifacts, CLI summaries, embedded profiles, documentation, and focused tests.
Built-in policy scope 2 registered scanners Only SkillSpector and clawscan-static receive native policies, which keeps the new behavior bounded to known raw-result formats.

Merge-risk options

Maintainer options:

  1. Prove embedded-profile compatibility (recommended)
    Before merge, capture redacted artifacts for representative existing ClawHub profile runs and document which gate verdicts newly become warn or block.
  2. Keep profile defaults unchanged
    Retain native policy support but remove the embedded-profile enablement if maintainers do not want existing profile artifacts to change by default.

Technical review

Best possible solution:

Keep native policies narrowly limited to built-in adapters with stable raw-result contracts, document the changed ClawHub profile verdict behavior, and show fresh-profile plus upgrade-profile artifacts before merge.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this PR introduces a new native gate-policy mode rather than fixing a reported failure. The branch includes focused fixture coverage claims for the intended scanner-result matrix.

Is this the best way to solve the issue?

Unclear: the runner design is narrowly aligned with the repository's registered-adapter and raw-evidence rules, but enabling it by default in embedded profiles needs maintainer confirmation of the upgrade behavior.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against b3bab6b88d91.

Labels

Label changes:

  • add P3: This is a draft optional gate-policy feature with limited scope rather than a current user-facing regression.
  • add merge-risk: 🚨 compatibility: The patch changes shipped ClawHub profile defaults and the serialized fired-rule shape that downstream artifact consumers may rely on.
  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: This is a member-authored draft PR, so the external-contributor real-behavior-proof gate does not apply; the PR nevertheless reports focused tests and full-suite checks.

Label justifications:

  • P3: This is a draft optional gate-policy feature with limited scope rather than a current user-facing regression.
  • merge-risk: 🚨 compatibility: The patch changes shipped ClawHub profile defaults and the serialized fired-rule shape that downstream artifact consumers may rely on.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: This is a member-authored draft PR, so the external-contributor real-behavior-proof gate does not apply; the PR nevertheless reports focused tests and full-suite checks.

Evidence

What I checked:

  • Runner and artifact implementation: The proposed patch adds Native policy support and extends fired gate-rule records so native recommendation/finding matches can be recorded without changing the raw scanner payload. (internal/runner/runner.go:66, b9605f91a09b)
  • Profile resolution change: The proposed patch accepts object-form built-in scanner references with gate.native: true and maps them back to registered built-in adapters rather than requiring a command-backed scanner. (internal/profiles/resolver.go:50, b9605f91a09b)
  • Embedded profile behavior: The proposed patch changes the shipped ClawHub and ClawHub-AIG profiles from plain scanner IDs to native-gate-enabled object references. (internal/profiles/clawhub/clawscan.yml:3, b9605f91a09b)
  • Focused coverage in the proposed branch: The PR adds resolver, runner, and CLI-summary coverage for built-in native policies, delayed critical findings, raw-evidence preservation, and exit-code composition. (internal/runner/runner_test.go:1618, b9605f91a09b)
  • Repository policy: The repository guidance requires registered scanner adapters, raw evidence preservation, fixture-backed scanner tests, and no ClawHub-specific flags in the public CLI; the visible patch follows those boundaries. (AGENTS.md:1, b3bab6b88d91)

Likely related people:

  • jesse-merhi: Authored the active runner, profile-resolution, artifact, test, and documentation changes for this feature; no stronger current-main ownership trail was available in the supplied review context. (role: recent area contributor; confidence: low; commits: b9605f91a09b; files: internal/runner/runner.go, internal/profiles/resolver.go, internal/profiles/clawhub/clawscan.yml)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Add a redacted artifact or terminal transcript comparing existing and native-enabled embedded-profile gate output.
  • Document the expected upgrade effect for consumers of gate and gateRules before marking the draft ready.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant