Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion evals/triage-security/evals.json
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@
},
{
"id": 14,
"prompt": "Triage Vulnerability issue TC-8005. The issue details are in vuln-issue-rpm.md, the security matrix is in security-matrix-mock.md, and the project CLAUDE.md is in claude-md-security-config.md. The issue has stream suffix [rhtpa-2.2] so triage is scoped to the 2.2.x stream. Assume cosign IS available (which cosign returns /usr/bin/cosign). Use the following mock SBOM comparison results for the affected versions: for versions 2.2.0 through 2.2.2, openssl-libs appears in BOTH the final image SBOM and the base image SBOM (confirming base image origin); the rpms.lock.yaml for these versions also lists openssl-libs (explicit install). This means the SBOM classification DISAGREES with the rpms.lock.yaml classification for these versions. Do NOT actually call Jira MCP, git show, cosign, or any external tools. Instead, write your triage analysis to the workspace outputs/ directory: write outputs/data-extraction.md with the parsed CVE data table from Step 1, write outputs/version-impact.md with the version impact table from Step 2, and write outputs/sbom-verification.md with the SBOM verification results from Step 2.3.5 showing both rpms.lock.yaml and SBOM signals side by side.",
"prompt": "Triage Vulnerability issue TC-8005. The issue details are in vuln-issue-rpm.md, the security matrix is in security-matrix-mock.md, and the project CLAUDE.md is in claude-md-security-config.md. The issue has stream suffix [rhtpa-2.2] so triage is scoped to the 2.2.x stream. Assume cosign IS available (which cosign returns /usr/bin/cosign). Use the following mock SBOM comparison results for the affected versions: for versions 2.2.0 through 2.2.2, openssl-libs appears in BOTH the final image SBOM and the base image SBOM (confirming base image origin); the rpms.lock.yaml for these versions also lists openssl-libs (explicit install). This means the SBOM classification DISAGREES with the rpms.lock.yaml classification for these versions. Do NOT actually call Jira MCP, git show, cosign, or any external tools. Instead, write your triage analysis to the workspace outputs/ directory: write outputs/data-extraction.md with the parsed CVE data table from Step 1, and write outputs/version-impact.md with the version impact table from Step 2, including the SBOM verification results from Step 2.3.5 inline in the dependency chain output (showing both rpms.lock.yaml and SBOM signals side by side — do not create a separate file for SBOM results).",
"expected_output": "A triage analysis for an RPM system package (openssl-libs) where cosign SBOM verification is available. The dependency chain output presents both rpms.lock.yaml classification (explicit install) and SBOM comparison result (base image) side by side. Because the two signals disagree, the discrepancy is flagged to the engineer for manual investigation.",
"files": ["files/vuln-issue-rpm.md", "files/security-matrix-mock.md", "files/claude-md-security-config.md"],
"assertions": [
Expand Down Expand Up @@ -358,6 +358,19 @@
"The VEX justification prompt includes the library name (rustls) and the feature flag name (tls-rustls) so the user has context for the decision",
"If the user chooses to skip remediation, the version is closed as not affected with the VEX justification — no remediation tasks are created for that version"
]
},
{
"id": 28,
"prompt": "Triage Vulnerability issue TC-8060. The issue details are in vuln-issue-transitive.md, the security matrix is in security-matrix-mock.md, and the project CLAUDE.md is in claude-md-security-config.md. The issue has stream suffix [rhtpa-2.2] so triage is scoped to the 2.2.x stream. The vulnerable library (h2) is a transitive dependency — the mock dependency chain data in the fixture shows h2 is pulled in through reqwest → hyper → h2 (3 levels deep). Do NOT actually call Jira MCP, git show, or any external tools. Instead, write your triage analysis to the workspace outputs/ directory: write outputs/data-extraction.md with the parsed CVE data table from Step 1, write outputs/version-impact.md with the version impact table from Step 2 including the dependency chain context from Step 2.3.5, and write outputs/remediation.md with the remediation task descriptions showing the full dependency chain and two-tier remediation approach.",
"expected_output": "A triage analysis where Step 2.3.5 identifies h2 as a transitive dependency (3 levels deep: reqwest → hyper → h2). The remediation task descriptions include the full dependency chain in the Implementation Notes and use the two-tier remediation approach: preferred approach is to bump reqwest (the direct dependency) to a version whose transitive closure includes h2 >= 0.4.5; fallback approach is to pin h2 directly via cargo add h2@0.4.5.",
"files": ["files/vuln-issue-transitive.md", "files/security-matrix-mock.md", "files/claude-md-security-config.md"],
"assertions": [
"Step 2.3.5 dependency chain context identifies h2 as a transitive dependency with the chain: backend → reqwest → hyper → h2 (3 levels deep)",
"The dependency chain output labels h2 as transitive (not direct) and includes the full chain path in the version impact output",
"Remediation task Implementation Notes include the dependency type as transitive with the dependency chain",
"Remediation task Implementation Notes describe the two-tier remediation approach: preferred is bumping reqwest (the direct dependency), fallback is pinning h2 directly",
"The fallback approach specifies the ecosystem-specific pinning mechanism: cargo add h2@0.4.5 for Cargo"
]
}
]
}
94 changes: 94 additions & 0 deletions evals/triage-security/files/vuln-issue-transitive.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
<!-- SYNTHETIC TEST DATA — Vulnerability issue for a transitive dependency (h2) for triage-security eval testing -->

# Mock Jira Vulnerability Issue

**Key**: TC-8060
**Summary**: CVE-2026-99010 h2 - Memory exhaustion via CONTINUATION frames [rhtpa-2.2]
**Issue Type**: Vulnerability
**Status**: New
**Labels**: CVE-2026-99010, pscomponent:org/rhtpa-server
**Affects Versions**: RHTPA 2.2.0
**Due Date**: 2026-08-15
**Assignee**: Unassigned
**Reporter**: psirt-analyst (account ID: 557058:psirt-analyst-mock-id)

## Remote Links

- [CVE-2026-99010](https://www.cve.org/CVERecord?id=CVE-2026-99010) — CVE Record
- [hyperium/h2#800](https://github.com/hyperium/h2/pull/800) — Upstream fix PR

## Comments

_(no comments)_

---

## Description

A vulnerability was found in h2. The h2 crate before version 0.4.5 allows a remote attacker to cause memory exhaustion by sending a large number of CONTINUATION frames. This vulnerability is classified as a denial of service (DoS).

**Affected package**: h2
**Affected versions**: versions before 0.4.5
**Fixed version**: 0.4.5
**CVSS**: 7.5 (High)

The vulnerability exists because h2 does not properly limit the number of CONTINUATION frames that can be received for a single HEADERS frame, allowing an attacker to send an unbounded sequence that consumes server memory.

### References

- https://www.cve.org/CVERecord?id=CVE-2026-99010

---

## Mock Dependency Chain Data

The following data simulates what Step 2.3.5 would discover when inspecting
the Cargo.toml manifest files and Cargo.lock. In a real triage, the skill
reads manifests via `git show`; in this eval, use this data as the simulated
output.

### h2 dependency chain for backend

```
Dependency chain for h2:
backend (workspace) → reqwest → hyper → h2
Type: transitive (3 levels deep)
Profile: production (reqwest is a runtime dependency)

First appeared: 2.1.0 (initial project setup — reqwest has always depended on hyper/h2)
Present in all versions
```

**Manifest evidence:**
```toml
# backend/Cargo.toml (all versions)
[dependencies]
reqwest = { version = "0.12", features = ["json"] }
# h2 is NOT a direct dependency — it comes through reqwest → hyper → h2
```

**Lock file evidence (affected versions):**
```
# Cargo.lock (versions 2.2.0 through 2.2.2)
[[package]]
name = "h2"
version = "0.4.4"

[[package]]
name = "hyper"
version = "1.4.1"
dependencies = ["h2"]

[[package]]
name = "reqwest"
version = "0.12.5"
dependencies = ["hyper"]
```

**Lock file evidence (fixed versions):**
```
# Cargo.lock (versions 2.2.3+)
[[package]]
name = "h2"
version = "0.4.5"
```
23 changes: 19 additions & 4 deletions plugins/sdlc-workflow/skills/triage-security/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -599,10 +599,12 @@ flowchart TD
A["Version impact table\n(after Steps 4-6)"] --> Z{"Step 7:\nConcurrent triage\non same component?"}
Z -->|No or user proceeds| B{"Any supported versions\naffected?"}
Z -->|User waits/skips| STOP["Stop or skip\ntask creation"]
B -->|Yes| C{"Other streams\nalso affected?"}
B -->|Yes| SCOPE{"Issue scoped to\na single stream?"}
B -->|No| D["Case C: Close as\nNot a Bug"]
SCOPE -->|"Yes (scoped)"| C{"Other streams\nalso affected?"}
SCOPE -->|"No (unscoped)"| F["Case A: Create\nremediation tasks\nfor affected streams"]
C -->|Yes| E["Case B: Post cross-stream\nimpact comment"]
C -->|No| F["Case A: Create\nremediation tasks"]
C -->|No| F
E --> F
F --> G{"Source dependency?\n(Cargo, npm)"}
G -->|Yes| H["2 tasks: upstream\nbackport + downstream\npropagation"]
Expand Down Expand Up @@ -644,8 +646,15 @@ are affected:

### Case B: Cross-stream impact — proactive remediation

If the version impact analysis reveals that **other streams** (outside this
issue's scope) are also affected:
**Guard — scoped issues only.** Case B applies exclusively to stream-scoped
issues (those whose summary contains a stream suffix like `[myproduct-2.2]`).
Unscoped issues cover all streams by definition — there are no "other streams
outside this issue's scope," so the cross-stream impact check is not applicable.
For unscoped issues, skip Case B entirely and proceed directly to Case A task
creation for all affected streams.

If the issue is **scoped** and the version impact analysis reveals that **other
streams** (outside this issue's scope) are also affected:

1. **Post the cross-stream impact comment** (existing behavior):
```
Expand Down Expand Up @@ -730,6 +739,12 @@ After creating each remediation task, post a description digest comment per
`shared/description-digest-protocol.md`. This ensures `/implement-task` can
verify description integrity in its Step 1.5.

For transitive dependencies, the remediation task must document the full
dependency chain (from Step 2.3.5) and specify the two-tier remediation
approach: bump the direct dependency first, fall back to pinning the
transitive dep. This may require coordination with intermediate package
maintainers if they pin the vulnerable version.
Comment thread
sourcery-ai[bot] marked this conversation as resolved.

Read `remediation-templates.md` for the full task description templates, Jira
issue creation API calls, digest comment procedures, and linkage procedures.
The key distinction:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,16 +38,42 @@ Advisory: [advisory URL from remote links]

## Implementation Notes

- Update [library] dependency to >= [fixed-version] in [lock-file-path]
- Target branch: [upstream-branch from Ecosystem Mappings]
- Check for pinned versions or transitive dependency constraints
that might prevent the bump
- If a direct bump introduces breaking changes, assess whether a
code-level workaround is viable (see upstream changelog)
- **Dependency type**: [direct | transitive (chain: [dependency-chain])]
- If the vulnerable dependency is dev-only or build-only (identified
in Step 2.3.5), the remediation priority is Normal regardless of CVE
severity. Add `dev-dependency` label to the task.

### Remediation approach (direct dependency)

When the vulnerable package is a **direct** dependency of a workspace member:

- Update [library] dependency to >= [fixed-version] in [lock-file-path]
- If a direct bump introduces breaking changes, assess whether a
code-level workaround is viable (see upstream changelog)

### Remediation approach (transitive dependency)

When the vulnerable package is a **transitive** dependency (pulled in
through intermediate packages), use a two-tier approach:

**Preferred: bump the direct dependency**
- Identify the direct dependency that pulls in [library] (see dependency
chain above)
- Bump the direct dependency to a version whose transitive closure
includes [library] >= [fixed-version]
- Verify the bump does not introduce breaking API changes to the
direct dependency

**Fallback: pin the transitive dependency directly**
If bumping the direct dependency is not viable (breaking API changes,
no release available with the fix):
- Cargo: `cargo add [library]@[fixed-version]` to add as a direct
dependency, overriding the transitive resolution
- npm: add `"[library]": ">=[fixed-version]"` to `overrides` (npm)
or `resolutions` (yarn/pnpm) in `package.json`
- Document why the direct dep bump was not viable in the PR description

## Acceptance Criteria

- [ ] [library] dependency is >= [fixed-version]
Expand Down Expand Up @@ -90,7 +116,11 @@ Konflux release repo so the next build ships the fix.
## Implementation Notes

- Source pinning method: [from Source Pinning Method in security-matrix.md]
- **Dependency type**: [direct | transitive] — carried forward from upstream task
- Update the [source-repo] reference to the merged commit or new release tag
- If the upstream fix pinned a transitive dependency directly (fallback
approach), verify the pinning is reflected in the downstream build's
lock file after the source reference update
- Verify the Konflux build pipeline triggers successfully

## Acceptance Criteria
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,10 @@ within the affected range), trace the dependency chain to give the engineer cont
about how the vulnerable package entered the tree. This information helps assess
remediation complexity — a direct dependency is a simple bump, while a deep
transitive dependency may require coordinating updates across intermediate packages.
The dependency chain also determines the remediation approach: direct dependencies
use a straightforward version bump, while transitive dependencies require a two-tier
strategy (bump the direct dep first; pin the transitive dep as a fallback) — see
`remediation-templates.md` for the full remediation approach templates.

For each affected version, trace how the vulnerable dependency entered the build
to give the engineer remediation context.
Expand All @@ -179,9 +183,16 @@ The investigation method depends on the ecosystem:
Inspect the lock file and manifest files to determine:

1. **Direct vs transitive** — is the vulnerable package a direct dependency of a
workspace member, or pulled in transitively?
workspace member, or pulled in transitively? This distinction drives the
remediation approach in `remediation-templates.md`:
- **Direct**: bump the dependency version in the manifest/lock file
- **Transitive**: prefer bumping the direct dependency that pulls it in;
fall back to pinning the transitive dep directly if the direct dep bump
is not viable (breaking API changes, no compatible release available)
2. **Dependency path** — the chain from a workspace root to the vulnerable package
(e.g., `workspace-root → reqwest → hyper → h2 → vulnerable-lib`)
(e.g., `workspace-root → reqwest → hyper → h2 → vulnerable-lib`). Record the
full chain — remediation tasks for transitive dependencies must include it so
the engineer knows which direct dependency to bump first.
3. **Profile/scope** — whether the dependency is included in all build profiles or
only specific ones:
- **Cargo**: `[dev-dependencies]` (test/bench only, not shipped),
Expand All @@ -195,14 +206,28 @@ Inspect the lock file and manifest files to determine:
another, note when it was introduced (helps identify which upgrade or feature
addition brought it in)

Example output:
Example output (direct dependency):
```
Dependency chain for quinn-proto:
backend (workspace) → reqwest [features: http3] → h3 → quinn → quinn-proto
Dependency chain for serde:
backend (workspace) → serde
Type: direct dependency
Profile: production (serde is a runtime dependency)

Remediation: bump serde to >= [fixed-version] in Cargo.toml
```

Example output (transitive dependency):
```
Dependency chain for vulnerable-lib:
backend (workspace) → reqwest → hyper → h2 → vulnerable-lib
Type: transitive (4 levels deep)
Profile: production (reqwest is a runtime dependency)

First appeared: 2.2.0 (commit 05a3af91 added reqwest http3 feature)
Not present in: 2.1.x (reqwest used without http3 feature)

Remediation: bump reqwest if a version with fixed vulnerable-lib is
available; otherwise pin vulnerable-lib directly via cargo add
```

#### Dependency scope decision tree
Expand Down Expand Up @@ -323,6 +348,8 @@ for the stream.
result (sub-step 6) or the skip notice above. This line is mandatory even
though the cosign verification itself is optional; omitting it leaves the
engineer without visibility into whether SBOM cross-validation was attempted.
Present all SBOM verification results inline within the dependency chain
output — do not create a separate output file or section for SBOM results.

4. **Base image reference** (when origin is base image) — extract the `FROM`
reference from the Dockerfile to identify the update path:
Expand Down
Loading