diff --git a/evals/triage-security/evals.json b/evals/triage-security/evals.json index 7b33b0460..94b253e44 100644 --- a/evals/triage-security/evals.json +++ b/evals/triage-security/evals.json @@ -303,6 +303,18 @@ "The triage proceeds normally through all steps without errors from the missing Deployment Context column", "Backward compatibility is maintained — existing eval behavior is unaffected by the absence of the Deployment Context column" ] + }, + { + "id": 25, + "prompt": "Triage Vulnerability issue TC-8040. The issue details are in vuln-issue-standard.md, the security matrix is in security-matrix-mock.md, and the project CLAUDE.md is in claude-md-security-config.md. However, assume the Ecosystem detection in Step 1 resolves to 'Go modules' — an ecosystem that is NOT listed in the Ecosystem Mappings table. Do NOT actually call Jira MCP, git show, or any external tools. Instead, write your triage analysis to the workspace outputs/ directory: write outputs/data-extraction.md with the parsed CVE data including the ecosystem detection result, and write outputs/unsupported-ecosystem.md with the unsupported ecosystem notification you would present to the user.", + "expected_output": "A triage analysis where Step 1 Ecosystem detection resolves to Go modules, which is not in the Ecosystem Mappings table. The skill presents the unsupported ecosystem message using the detected ecosystem name ('Go modules') in the notification, and stops automated triage for that ecosystem.", + "files": ["files/vuln-issue-standard.md", "files/security-matrix-mock.md", "files/claude-md-security-config.md"], + "assertions": [ + "The unsupported ecosystem notification uses the detected ecosystem name ('Go modules') — not a hard-coded ecosystem name (Step 5.3 unsupported ecosystem handling)", + "The notification follows the template pattern: the message contains the actual ecosystem name substituted into the placeholder, matching the angle-bracket placeholder convention used by other user-facing messages in the skill", + "Automated triage stops after the unsupported ecosystem notification — no version impact analysis or remediation task creation is performed for the unsupported ecosystem", + "The notification includes both the ecosystem name and the instruction that manual assessment is required" + ] } ] } diff --git a/plugins/sdlc-workflow/skills/triage-security/SKILL.md b/plugins/sdlc-workflow/skills/triage-security/SKILL.md index 4e041ef16..2c251492a 100644 --- a/plugins/sdlc-workflow/skills/triage-security/SKILL.md +++ b/plugins/sdlc-workflow/skills/triage-security/SKILL.md @@ -371,13 +371,19 @@ assuming a fixed set. Common ecosystems include: - **npm** — JavaScript/TypeScript packages (e.g., axios, webpack, react) - **RPM** — System packages in container images (e.g., openssl, curl, glibc) +If the detected ecosystem is not listed in the stream's Ecosystem Mappings table (e.g., +Go modules), inform the user and stop automated triage for that ecosystem: + +> "**Unsupported ecosystem**: is not yet supported for automated triage. +> Manual assessment is required." + The ecosystem determines which lock file to inspect and which parsing command to use. Both the lock file path and check command are configured per ecosystem in each stream's `security-matrix.md` Ecosystem Mappings table. ```mermaid flowchart TD - A["Identify ecosystem\nfrom library + component"] --> B{"Source dependency?\n(Cargo, npm, Go, ...)"} + A["Identify ecosystem\nfrom library + component"] --> B{"Source dependency?\n(Cargo, npm, ...)"} B -->|Yes| C["Lock file inspection\n(git show commit:lock-file)"] B -->|No| D{"RPM / system\npackage?"} D -->|Yes| E{"RPM lock file\nconfigured?"} @@ -557,7 +563,7 @@ flowchart TD C -->|Yes| E["Case B: Post cross-stream\nimpact comment"] C -->|No| F["Case A: Create\nremediation tasks"] E --> F - F --> G{"Source dependency?\n(Cargo, npm, Go)"} + F --> G{"Source dependency?\n(Cargo, npm)"} G -->|Yes| H["2 tasks: upstream\nbackport + downstream\npropagation"] G -->|No| I["1 task: Konflux\nrepo fix"] D --> J{"VEX Justification\nconfigured?"} @@ -688,7 +694,7 @@ Read `remediation-templates.md` for the full task description templates, Jira issue creation API calls, digest comment procedures, and linkage procedures. The key distinction: -- **Source dependency ecosystems** (Cargo, npm, Go modules): create **two** tasks — +- **Source dependency ecosystems** (Cargo, npm): create **two** tasks — an upstream backport task (fix in the source repo) and a downstream propagation subtask (update the reference in the Konflux release repo). The downstream subtask is blocked by the upstream task. @@ -749,7 +755,7 @@ MUST include the Comment Footnote (see above). This skill only creates remediation Tasks. Cross-stream impact is reported via comment on the current issue. 8. **One remediation Task per affected stream, plus a downstream propagation - subtask** when the ecosystem is a source dependency (Cargo, npm, Go modules). + subtask** when the ecosystem is a source dependency (Cargo, npm). The upstream task covers the source repo fix; the downstream subtask covers the Konflux release repo update and is blocked by the upstream task. System package ecosystems produce a single task. A single Task spanning multiple diff --git a/plugins/sdlc-workflow/skills/triage-security/remediation-templates.md b/plugins/sdlc-workflow/skills/triage-security/remediation-templates.md index ceb351b8e..2534d8fc5 100644 --- a/plugins/sdlc-workflow/skills/triage-security/remediation-templates.md +++ b/plugins/sdlc-workflow/skills/triage-security/remediation-templates.md @@ -6,7 +6,7 @@ These templates define the Jira Task descriptions created during CVE triage The number of tasks depends on the ecosystem type: -- **Source dependency ecosystems** (Cargo, npm, Go modules): create **two** tasks — +- **Source dependency ecosystems** (Cargo, npm): create **two** tasks — an upstream backport task (fix in the source repo) and a downstream propagation subtask (update the reference in the Konflux release repo). The downstream subtask is blocked by the upstream task. diff --git a/plugins/sdlc-workflow/skills/triage-security/version-impact-analysis.md b/plugins/sdlc-workflow/skills/triage-security/version-impact-analysis.md index 10aa16366..b9a936c3e 100644 --- a/plugins/sdlc-workflow/skills/triage-security/version-impact-analysis.md +++ b/plugins/sdlc-workflow/skills/triage-security/version-impact-analysis.md @@ -174,7 +174,7 @@ to give the engineer remediation context. The investigation method depends on the ecosystem: -#### Source-level dependencies (Cargo, npm, Go) +#### Source-level dependencies (Cargo, npm) Inspect the lock file and manifest files to determine: