Skip to content

fix(security): 조직 보안 실패 수집기를 fail-closed로 전환#316

Open
seonghobae wants to merge 11 commits into
developfrom
agent/security-collector-fail-closed
Open

fix(security): 조직 보안 실패 수집기를 fail-closed로 전환#316
seonghobae wants to merge 11 commits into
developfrom
agent/security-collector-fail-closed

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

변경 사항

  • 전용 수집기 GitHub App 설정이 없을 때 조직 공용 Noema App 설정을 사용합니다.
  • 두 App 설정 모두 불완전하면 예약 수집을 성공으로 건너뛰지 않고 이유가 보이는 오류로 종료합니다.
  • 모듈 실행 방식으로 collector import 경로를 고정합니다.
  • 미설정 fail-closed 계약 회귀 테스트를 추가합니다.

보안 근거

현재 조직 Noema App 설치는 Actions/Checks read 및 Issues write 권한을 보유합니다. 이 변경이 병합되면 Strix 등 보안 워크플로 실패를 AppGuardrail issue로 수집하는 예약 제어가 무음 no-op으로 남지 않습니다.

검증

  • python3 -m pytest -q tests/test_org_security_failure_collector.py tests/test_issueops_core.py: 14 passed
  • actionlint -shellcheck= -pyflakes= .github/workflows/org-security-failure-collector.yml: passed
  • git diff --check HEAD^ HEAD: passed

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 6e797590973cbfc7e683d6b09a51437661d2e956.

  • Head SHA: 6e797590973cbfc7e683d6b09a51437661d2e956

  • Workflow run: 29463702361

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: org-security-failure-collector.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: org-security-failure-collector.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Test: test_org_security_failure_collector.py"]
  S2 --> I2["regression suite"]
  I2 --> R2["Review risk: Test: test_org_security_failure_collector.py"]
  R2 --> V2["targeted test run"]
Loading

@opencode-agent

opencode-agent Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

OpenCode Review Overview

  • Head SHA: cca330b26189780e0d833e407d52b3675fdfd8af
  • Workflow run: 29471730434
  • Workflow attempt: 1
  • Gate result: REQUEST_CHANGES (approval step)

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head cca330b26189780e0d833e407d52b3675fdfd8af.

  • Head SHA: cca330b26189780e0d833e407d52b3675fdfd8af

  • Workflow run: 29471730434

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: org-security-failure-collector.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: org-security-failure-collector.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file (2 files)"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file (2 files)"]
  R2 --> V2["required checks"]
  Evidence --> S3["CI script: collect_org_security_failures.py"]
  S3 --> I3["review and security gate shell path"]
  I3 --> R3["Review risk: CI script: collect_org_security_failures.py"]
  R3 --> V3["bash -n plus Strix self-test"]
  Evidence --> S4["Test: test_org_security_failure_collector.py"]
  S4 --> I4["regression suite"]
  I4 --> R4["Review risk: Test: test_org_security_failure_collector.py"]
  R4 --> V4["targeted test run"]
Loading

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 43bd71aaf7f1254f2c4ccb16a70f20d67a61e4ba.

  • Head SHA: 43bd71aaf7f1254f2c4ccb16a70f20d67a61e4ba

  • Workflow run: 29464724725

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: org-security-failure-collector.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: org-security-failure-collector.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file: Dockerfile"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file: Dockerfile"]
  R2 --> V2["required checks"]
  Evidence --> S3["Test: test_org_security_failure_collector.py"]
  S3 --> I3["regression suite"]
  I3 --> R3["Review risk: Test: test_org_security_failure_collector.py"]
  R3 --> V3["targeted test run"]
Loading

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 60adcf19fa04bdfd666bf423c54236d2fb3c4859.

  • Head SHA: 60adcf19fa04bdfd666bf423c54236d2fb3c4859

  • Workflow run: 29465272804

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: org-security-failure-collector.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: org-security-failure-collector.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file: Dockerfile"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file: Dockerfile"]
  R2 --> V2["required checks"]
  Evidence --> S3["Test: test_org_security_failure_collector.py"]
  S3 --> I3["regression suite"]
  I3 --> R3["Review risk: Test: test_org_security_failure_collector.py"]
  R3 --> V3["targeted test run"]
Loading

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 268239eb89bae3e90964ed0585a329c1e54a56d5.

  • Head SHA: 268239eb89bae3e90964ed0585a329c1e54a56d5

  • Workflow run: 29465774600

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: org-security-failure-collector.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: org-security-failure-collector.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file: Dockerfile"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file: Dockerfile"]
  R2 --> V2["required checks"]
  Evidence --> S3["CI script: collect_org_security_failures.py"]
  S3 --> I3["review and security gate shell path"]
  I3 --> R3["Review risk: CI script: collect_org_security_failures.py"]
  R3 --> V3["bash -n plus Strix self-test"]
  Evidence --> S4["Test: test_org_security_failure_collector.py"]
  S4 --> I4["regression suite"]
  I4 --> R4["Review risk: Test: test_org_security_failure_collector.py"]
  R4 --> V4["targeted test run"]
Loading

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 443ccd407a07dd3752df4799645968f5794ca0ae.

  • Head SHA: 443ccd407a07dd3752df4799645968f5794ca0ae

  • Workflow run: 29466418435

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: org-security-failure-collector.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: org-security-failure-collector.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file: Dockerfile"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file: Dockerfile"]
  R2 --> V2["required checks"]
  Evidence --> S3["CI script: collect_org_security_failures.py"]
  S3 --> I3["review and security gate shell path"]
  I3 --> R3["Review risk: CI script: collect_org_security_failures.py"]
  R3 --> V3["bash -n plus Strix self-test"]
  Evidence --> S4["Test: test_org_security_failure_collector.py"]
  S4 --> I4["regression suite"]
  I4 --> R4["Review risk: Test: test_org_security_failure_collector.py"]
  R4 --> V4["targeted test run"]
Loading

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 14d84c966788465a20e341571c3708f37ae8ba3a.

  • Head SHA: 14d84c966788465a20e341571c3708f37ae8ba3a

  • Workflow run: 29467510728

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: org-security-failure-collector.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: org-security-failure-collector.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file (2 files)"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file (2 files)"]
  R2 --> V2["required checks"]
  Evidence --> S3["CI script: collect_org_security_failures.py"]
  S3 --> I3["review and security gate shell path"]
  I3 --> R3["Review risk: CI script: collect_org_security_failures.py"]
  R3 --> V3["bash -n plus Strix self-test"]
  Evidence --> S4["Test: test_org_security_failure_collector.py"]
  S4 --> I4["regression suite"]
  I4 --> R4["Review risk: Test: test_org_security_failure_collector.py"]
  R4 --> V4["targeted test run"]
Loading

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 8e558ffc8a1e3690e4dad5e7bf6540d4ccfffaa8.

  • Head SHA: 8e558ffc8a1e3690e4dad5e7bf6540d4ccfffaa8

  • Workflow run: 29468445899

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: org-security-failure-collector.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: org-security-failure-collector.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file (2 files)"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file (2 files)"]
  R2 --> V2["required checks"]
  Evidence --> S3["CI script: collect_org_security_failures.py"]
  S3 --> I3["review and security gate shell path"]
  I3 --> R3["Review risk: CI script: collect_org_security_failures.py"]
  R3 --> V3["bash -n plus Strix self-test"]
  Evidence --> S4["Test: test_org_security_failure_collector.py"]
  S4 --> I4["regression suite"]
  I4 --> R4["Review risk: Test: test_org_security_failure_collector.py"]
  R4 --> V4["targeted test run"]
Loading

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 41da8955b40aef6ace43b42d56ba14776e129563.

  • Head SHA: 41da8955b40aef6ace43b42d56ba14776e129563

  • Workflow run: 29468922166

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: org-security-failure-collector.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: org-security-failure-collector.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file (2 files)"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file (2 files)"]
  R2 --> V2["required checks"]
  Evidence --> S3["CI script: collect_org_security_failures.py"]
  S3 --> I3["review and security gate shell path"]
  I3 --> R3["Review risk: CI script: collect_org_security_failures.py"]
  R3 --> V3["bash -n plus Strix self-test"]
  Evidence --> S4["Test: test_org_security_failure_collector.py"]
  S4 --> I4["regression suite"]
  I4 --> R4["Review risk: Test: test_org_security_failure_collector.py"]
  R4 --> V4["targeted test run"]
Loading

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 24ced6dd18e60ce86c802a76df8437d87a948830.

  • Head SHA: 24ced6dd18e60ce86c802a76df8437d87a948830

  • Workflow run: 29469556872

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: org-security-failure-collector.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: org-security-failure-collector.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file (2 files)"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file (2 files)"]
  R2 --> V2["required checks"]
  Evidence --> S3["CI script: collect_org_security_failures.py"]
  S3 --> I3["review and security gate shell path"]
  I3 --> R3["Review risk: CI script: collect_org_security_failures.py"]
  R3 --> V3["bash -n plus Strix self-test"]
  Evidence --> S4["Test: test_org_security_failure_collector.py"]
  S4 --> I4["regression suite"]
  I4 --> R4["Review risk: Test: test_org_security_failure_collector.py"]
  R4 --> V4["targeted test run"]
Loading

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 8b157507e23466a5d8c80bd6badea706c0533d69.

  • Head SHA: 8b157507e23466a5d8c80bd6badea706c0533d69

  • Workflow run: 29470855188

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: org-security-failure-collector.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: org-security-failure-collector.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file (2 files)"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file (2 files)"]
  R2 --> V2["required checks"]
  Evidence --> S3["CI script: collect_org_security_failures.py"]
  S3 --> I3["review and security gate shell path"]
  I3 --> R3["Review risk: CI script: collect_org_security_failures.py"]
  R3 --> V3["bash -n plus Strix self-test"]
  Evidence --> S4["Test: test_org_security_failure_collector.py"]
  S4 --> I4["regression suite"]
  I4 --> R4["Review risk: Test: test_org_security_failure_collector.py"]
  R4 --> V4["targeted test run"]
Loading

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

  • Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.

  • Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.

  • Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.

  • Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.

  • Result: REQUEST_CHANGES

  • Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head cca330b26189780e0d833e407d52b3675fdfd8af.

  • Head SHA: cca330b26189780e0d833e407d52b3675fdfd8af

  • Workflow run: 29471730434

  • Workflow attempt: 1

Coverage evidence

Coverage Decision

  • Result: FAIL
  • Test evidence: not proven passing
  • Docstring evidence: not proven passing when configured
  • Failure count: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: org-security-failure-collector.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: org-security-failure-collector.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Changed file (2 files)"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Changed file (2 files)"]
  R2 --> V2["required checks"]
  Evidence --> S3["CI script: collect_org_security_failures.py"]
  S3 --> I3["review and security gate shell path"]
  I3 --> R3["Review risk: CI script: collect_org_security_failures.py"]
  R3 --> V3["bash -n plus Strix self-test"]
  Evidence --> S4["Test: test_org_security_failure_collector.py"]
  S4 --> I4["regression suite"]
  I4 --> R4["Review risk: Test: test_org_security_failure_collector.py"]
  R4 --> V4["targeted test run"]
Loading

@cwl-noema-review cwl-noema-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noema PydanticAI review

This PR attempts a security hardening refactor for the org security failure collector container, script, and workflow, primarily by switching it to a fail-closed, least-privilege configuration. However, based on the bounded evidence:

  • The required coverage-evidence job failed (failure), so there is no workflow, test, or docstring evidence proving passing repository checks, as required for approval.
  • Workflow job log confirms missing or failed evidence; both OpenCode and Noema can only approve when coverage-evidence and all required test gates are green.
  • There are no unresolved security or dependency scanner findings, so no package bump is required, but the coverage gate remains the blocker.
  • Review test files show that fail-closed is under test, and tests are targeting the critical risk boundaries, but the required evidence run did not succeed.

Action: The PR cannot be approved until a clean coverage-evidence run proves the changes are covered by passing tests and gate evidence. This is a process gate, not a behavioral or correctness bug in the PR itself.

Findings

  • [high] .github/workflows/opencode-review.yml:1: Install or configure the repository test/docstring evidence, rerun the current-head coverage-evidence job, and only approve when it reports success. (The required coverage-evidence job result was failure, so required test/docstring evidence was not proven for current head cca330b26189780e0d833e407d52b3675fdfd8af.)

  • [high] .github/checks/coverage-evidence: Fix the logged root cause and rerun the check on the current head. (Current-head check concluded failure; see bounded workflow_logs.)

  • Result: REQUEST_CHANGES

  • Verdict: request_changes

  • Confidence: high

  • Head SHA: cca330b26189780e0d833e407d52b3675fdfd8af

  • Reviewer credential: noema-github-app

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant