diff --git a/evals/triage-security/evals.json b/evals/triage-security/evals.json index 7b33b0460..ecc0359d9 100644 --- a/evals/triage-security/evals.json +++ b/evals/triage-security/evals.json @@ -231,6 +231,19 @@ "The embargo warning gate does NOT trigger for Low or Moderate severity CVEs (CVSS < 7.0) — it is skipped silently when severity is below threshold (§1.70)" ] }, + { + "id": 18, + "prompt": "Triage Vulnerability issue TC-8001. The issue details are in vuln-issue-standard-already-triaged.md, the security matrix is in security-matrix-mock.md, and the project CLAUDE.md is in claude-md-security-config.md. The issue has ALREADY been triaged in a prior run: it has the ai-cve-triaged label, Status is In Progress, two remediation tasks (TC-8100, TC-8101) are linked via Depend, a description digest comment exists, and a post-triage summary comment exists. This is a RE-RUN of triage on the same issue. Do NOT actually call Jira MCP, git show, or any external tools. Instead, write your triage analysis to the workspace outputs/ directory: write outputs/data-extraction.md with the parsed CVE data table from Step 1, write outputs/idempotency-check.md with the analysis of all pre-existing triage artifacts detected and skipped, and write outputs/triage-outcome.md explaining why the second run produces no new mutations.", + "expected_output": "A triage analysis where the second run detects all pre-existing triage artifacts — ai-cve-triaged label, In Progress status, existing Depend links to remediation tasks TC-8100 and TC-8101, existing description digest comment, and existing post-triage summary comment — and skips all mutations. No new remediation tasks are created, no labels are added, no status transition occurs, no duplicate comments are posted, and no duplicate links are created. The output documents each idempotent skip with an explanation.", + "files": ["files/vuln-issue-standard-already-triaged.md", "files/security-matrix-mock.md", "files/claude-md-security-config.md"], + "assertions": [ + "Step 8 detects existing remediation tasks TC-8100 and TC-8101 via the Depend links on the issue and skips remediation task creation — does NOT create duplicate tasks (idempotency)", + "The ai-cve-triaged label is already present on the issue — the triage does NOT attempt to add it again (label idempotency)", + "The issue is already In Progress — the triage does NOT attempt to transition the status again or warns that the issue is already in a post-triage state (status idempotency)", + "The existing description digest comment is detected — no duplicate digest comment is posted (comment idempotency)", + "The existing post-triage summary comment is detected — no duplicate summary comment is posted (comment idempotency)" + ] + }, { "id": 19, "prompt": "Triage Vulnerability issue TC-8001. The issue details are in vuln-issue-standard.md, the security matrix is in security-matrix-stale-mock.md (which has a Last-Updated timestamp of 2026-05-01T10:00:00Z — more than 14 days old), and the project CLAUDE.md is in claude-md-security-config.md. Do NOT actually call Jira MCP, git show, or any external tools. Instead, write your triage analysis to the workspace outputs/ directory: write outputs/staleness-check.md with the Step 0.3 staleness detection output including the warning message and user options.", diff --git a/evals/triage-security/files/vuln-issue-standard-already-triaged.md b/evals/triage-security/files/vuln-issue-standard-already-triaged.md new file mode 100644 index 000000000..b2d5127dd --- /dev/null +++ b/evals/triage-security/files/vuln-issue-standard-already-triaged.md @@ -0,0 +1,81 @@ + + +# Mock Jira Vulnerability Issue + +**Key**: TC-8001 +**Summary**: CVE-2026-31812 quinn-proto - Panic on large stream counts [rhtpa-2.2] +**Issue Type**: Vulnerability +**Status**: In Progress +**Labels**: CVE-2026-31812, pscomponent:org/rhtpa-server, ai-cve-triaged +**Affects Versions**: RHTPA 2.2.0, RHTPA 2.2.1 +**Due Date**: 2026-07-15 +**Assignee**: engineer-a@example.com + +## Custom Fields + +- **customfield_10632** (Upstream Affected Component): quinn-proto +- **customfield_10669** (PS Component): pscomponent:org/rhtpa-server +- **customfield_10832** (Stream): rhtpa-2.2 + +## Issue Links + +The following links already exist on this issue: + +- **Depend**: TC-8100 (remediation Task — upstream backport) + - **Summary**: Backport quinn-proto fix to >= 0.11.14 on release/0.4.z [rhtpa-2.2] + - **Status**: In Progress + - **Labels**: ai-generated-jira, Security, CVE-2026-31812 +- **Depend**: TC-8101 (remediation Task — downstream propagation) + - **Summary**: Propagate quinn-proto bump to rhtpa-server release branch [rhtpa-2.2] + - **Status**: Open + - **Labels**: ai-generated-jira, Security, CVE-2026-31812 + - **Blocks**: TC-8100 + +## Remote Links + +- [GHSA-2026-qp73-x4mq](https://github.com/advisories/GHSA-2026-qp73-x4mq) — GitHub Advisory +- [CVE-2026-31812](https://www.cve.org/CVERecord?id=CVE-2026-31812) — CVE Record +- [quinn-rs/quinn#2048](https://github.com/quinn-rs/quinn/pull/2048) — Upstream fix PR + +## Comments + +1. **[sdlc-workflow] Description digest: sha256-md:a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2** + - Posted by: sdlc-workflow/triage-security + - Created: 2026-07-01T10:00:00Z + +2. **Post-triage summary** + + Triage complete for CVE-2026-31812 (quinn-proto). + + **Version impact**: RHTPA 2.2.0 and 2.2.1 ship quinn-proto < 0.11.14 (affected). + RHTPA 2.2.2 and later ship quinn-proto 0.11.14 (not affected). + + **Actions taken**: + - Affects Versions corrected to RHTPA 2.2.0, RHTPA 2.2.1 + - Label `ai-cve-triaged` added + - Remediation tasks created: TC-8100 (upstream backport), TC-8101 (downstream propagation) + - Transitioned to In Progress + + --- + This comment was AI-generated by [sdlc-workflow/triage-security](https://github.com/mrizzi/sdlc-plugins) v0.11.1. + + - Posted by: sdlc-workflow/triage-security + - Created: 2026-07-01T10:01:00Z + +--- + +## Description + +A vulnerability was found in quinn-proto. The quinn-proto crate before version 0.11.14 allows a remote attacker to cause a panic by sending a QUIC transport frame that creates an excessive number of streams. This vulnerability is classified as a denial of service (DoS). + +**Affected package**: quinn-proto +**Affected versions**: versions before 0.11.14 +**Fixed version**: 0.11.14 +**CVSS**: 7.5 (High) + +The vulnerability exists because quinn-proto does not properly validate the number of streams requested in a STREAMS frame. An attacker can send a specially crafted frame that causes the server to allocate an unbounded number of stream state objects, leading to a panic when the allocation exceeds internal limits. + +### References + +- https://github.com/advisories/GHSA-2026-qp73-x4mq +- https://rustsec.org/advisories/RUSTSEC-2026-0042.html