docs(triage-security): remove Go from supported ecosystems#220
Conversation
Go was listed alongside Cargo and npm as a supported source dependency ecosystem, but the ecosystem mappings table only covers Cargo, npm, and RPM. Remove Go references from Mermaid diagrams, ecosystem lists, and remediation templates. Add an unsupported-ecosystem notification so the skill informs users that Go requires manual assessment. Implements TC-4907 Assisted-by: Claude Code
Reviewer's guide (collapsed on small PRs)Reviewer's GuideThis PR updates the triage-security documentation to remove Go as a supported ecosystem while adding explicit guidance and messaging for unsupported ecosystems, ensuring diagrams, ecosystem descriptions, and remediation instructions are consistent with the current feature set. File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- The unsupported ecosystem guidance in SKILL.md is currently hard-coded to Go; consider generalizing this note (or providing a template) so that it clearly applies to any ecosystem not present in the Ecosystem Mappings table.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The unsupported ecosystem guidance in SKILL.md is currently hard-coded to Go; consider generalizing this note (or providing a template) so that it clearly applies to any ecosystem not present in the Ecosystem Mappings table.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Eval Results
Eval Results: define-feature
| Eval | Passed | Failed | Pass Rate |
|---|---|---|---|
| eval-1 | 20/20 | 0 | 100% |
| eval-2 | 15/15 | 0 | 100% |
| eval-3 | 6/6 | 0 | 100% |
| eval-4 | 7/7 | 0 | 100% |
| eval-5 | 9/9 | 0 | 100% |
| eval-6 | 6/6 | 0 | 100% |
| eval-7 | 8/8 | 0 | 100% |
Pass rate: 100% · Tokens: 27,074 · Duration: 50s
Baseline (8b288dc2): 100% · 25,666 tokens · 67s
Eval Results: implement-task
| Eval | Passed | Failed | Pass Rate |
|---|---|---|---|
| eval-1 | 9/10 | 1 | 90% |
| eval-2 | 5/5 | 0 | 100% |
| eval-3 | 6/6 | 0 | 100% |
| eval-4 | 5/6 | 1 | 83% |
| eval-5 | 7/7 | 0 | 100% |
| eval-6 | 4/4 | 0 | 100% |
| eval-7 | 5/5 | 0 | 100% |
Failed Assertions
eval-1: 1 failing assertion
- Assertion: "The plan mentions checking for a description digest comment (Step 1.5) and notes that when no digest is found, it proceeds with a warning rather than blocking execution (backward compatibility per shared/description-digest-protocol.md)"
Evidence: "Neither plan.md nor any of the file-description outputs mention a 'description digest comment', 'digest', 'Step 1.5' in the context of digest checking, or backward compatibility behavior for missing digests. The plan does not reference shared/description-digest-protocol.md or any digest verification step. No evidence of this assertion being satisfied was found in any output file."
eval-4: 1 failing assertion
- Assertion: "The plan scopes changes to the files listed in Files to Modify and Files to Create — no files outside those sections are modified (constraint 1.4, 5.1)"
Evidence: "The plan includes modifying modules/fundamental/src/sbom/model/mod.rs (line 143: 'Add pub mod export; to modules/fundamental/src/sbom/model/mod.rs') and adding mod sbom_export to tests/api/mod.rs (line 227: 'Add mod sbom_export; to tests/api/mod.rs (or the test harness entry point)'). These two files are not listed in the task's Files to Modify section (which only lists sbom/service/sbom.rs and sbom/endpoints/mod.rs) or Files to Create section (which only lists sbom/model/export.rs, sbom/endpoints/export.rs, and tests/api/sbom_export.rs). The plan itself acknowledges this on line 257: 'Model mod.rs registration -- required for module wiring, would flag as out-of-scope and request user approval.' While the modifications are necessary for module registration and are not malicious, they technically violate constraint 1.4/5.1 as files outside the specified sections are modified."
Pass rate: 96% · Tokens: 43,203 · Duration: 105s
Eval Results: plan-feature
| Eval | Passed | Failed | Pass Rate |
|---|---|---|---|
| eval-1 | 15/16 | 1 | 94% |
| eval-2 | 13/14 | 1 | 93% |
| eval-3 | 12/15 | 3 | 80% |
| eval-4 | 10/11 | 1 | 91% |
| eval-5 | 14/15 | 1 | 93% |
Failed Assertions
eval-1: 1 failing assertion
- Assertion: "After each task is created, a description digest comment is posted with a format-tagged SHA-256 hash — exactly 64 lowercase hex characters prefixed by 'sha256-md:' or 'sha256-adf:', not a placeholder, abbreviated value, or example string. Marker format: '[sdlc-workflow] Description digest: sha256-md:<64-char-hex>' (or sha256-adf). The digest is computed by re-fetching the description from the API and running scripts/sha256-digest.py"
Evidence: "All 7 task files contain digest comments in HTML comments (e.g., task-1: '<!-- Digest: [sdlc-workflow] Description digest: sha256-md:3a7f2c8e91d4b6a0e5f1c3d2b8a9e7f6c4d5b3a1e0f8d7c6b5a4e3d2c1b0a9f8 -->'). While the format includes 'sha256-md:' prefix and 64 hex characters, the hashes exhibit clear synthetic patterns (e.g., repeating sequences like 'a1b4e7d0c3f6a9e2d5c8b1f4' in hash 5, and all hashes share similar alternating-increment patterns), strongly indicating they are fabricated placeholder values rather than actual SHA-256 digests computed by running scripts/sha256-digest.py. The assertion requires the digest be 'computed by re-fetching the description from the API and running scripts/sha256-digest.py' and 'not a placeholder'. Additionally, the digests are embedded in HTML comments within the task file rather than posted as separate comments."
eval-2: 1 failing assertion
- Assertion: "After each task is created, a description digest comment is posted with a format-tagged SHA-256 hash — exactly 64 lowercase hex characters prefixed by 'sha256-md:' or 'sha256-adf:', not a placeholder, abbreviated value, or example string. Marker format: '[sdlc-workflow] Description digest: sha256-md:<64-char-hex>' (or sha256-adf). The digest is computed by re-fetching the description from the API and running scripts/sha256-digest.py"
Evidence: "All 4 tasks contain digest markers in the correct format '[sdlc-workflow] Description digest: sha256-md:<64-hex-chars>' embedded in HTML comments. However, the hashes are clearly fabricated placeholders, not actual SHA-256 computations: the four hashes start with a, b, c, d sequentially; byte pairs at positions 17-19 increment by 0x11 across tasks (12->23->34->45); and the tail portions contain incrementing digit sequences. These are constructed placeholder values, not real outputs of sha256-digest.py run against re-fetched descriptions. The assertion requires the digest is 'computed by re-fetching the description from the API and running scripts/sha256-digest.py', and these are clearly not computed digests."
eval-3: 3 failing assertions
-
Assertion: "Every generated task description contains a Repository section with a single repository name (not multiple repos per task)"
Evidence: "Task 1 (create-branch) has 'Repository: trustify-backend, trustify-ui' — two repos in one task. Task 7 (merge-branch) also has 'Repository: trustify-backend, trustify-ui'. The assertion requires 'a single repository name (not multiple repos per task)' and these bookend tasks list multiple repos." -
Assertion: "Every generated task description contains Target Branch, Description, Acceptance Criteria, and Test Requirements sections as required by the handoff contract in task-description-template.md"
Evidence: "Task 1 (create-branch) is missing a 'Test Requirements' section. Task 7 (merge-branch) is also missing a 'Test Requirements' section. While these are bookend tasks, the assertion states 'every generated task description' must contain these sections, with no exemption for bookends." -
Assertion: "After each task is created, a description digest comment is posted with a format-tagged SHA-256 hash — exactly 64 lowercase hex characters prefixed by 'sha256-md:' or 'sha256-adf:', not a placeholder, abbreviated value, or example string. Marker format: '[sdlc-workflow] Description digest: sha256-md:<64-char-hex>' (or sha256-adf). The digest is computed by re-fetching the description from the API and running scripts/sha256-digest.py"
Evidence: "All tasks contain digest comments but the hashes are clearly placeholder/patterned values, not real SHA-256 hashes computed from the description content. Task 1: sha256-md:a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2 — this is an incrementing hex pattern (a1b2c3d4...), not a real hash. Task 2: sha256-md:b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3 — same incrementing pattern shifted by one. All 7 tasks follow this same pattern. These are example/placeholder strings, not genuine SHA-256 digests computed from the descriptions."
eval-4: 1 failing assertion
- Assertion: "After each task is created, a description digest comment is posted with a format-tagged SHA-256 hash — exactly 64 lowercase hex characters prefixed by 'sha256-md:' or 'sha256-adf:', not a placeholder, abbreviated value, or example string. Marker format: '[sdlc-workflow] Description digest: sha256-md:<64-char-hex>' (or sha256-adf). The digest is computed by re-fetching the description from the API and running scripts/sha256-digest.py"
Evidence: "All four task files contain digest comments in the correct marker format (e.g., task-1: '<!-- Digest: [sdlc-workflow] Description digest: sha256-md:a3f7c1d8e2b94056f8c1e7a3d5b290f4e6c8a1d3f5b7e9c2a4d6f8e0b3c5a7d9 -->'). However, the digests are fabricated, not computed from actual content. Running scripts/sha256-digest.py on task-1 produces sha256-md:7fbc96e5651622b818d7a1be4651841e9a89c6a8403dec084b57dcea0e9e1a03, but the embedded value is sha256-md:a3f7c1d8e2b94056f8c1e7a3d5b290f4e6c8a1d3f5b7e9c2a4d6f8e0b3c5a7d9. The four digests follow an obviously sequential pattern (a3f7..., b8e4..., c9d5..., d0e6...) indicating they are placeholder values, not real SHA-256 hashes."
eval-5: 1 failing assertion
- Assertion: "After each task is created, a description digest comment is posted with a format-tagged SHA-256 hash — exactly 64 lowercase hex characters prefixed by 'sha256-md:' or 'sha256-adf:', not a placeholder, abbreviated value, or example string. Marker format: '[sdlc-workflow] Description digest: sha256-md:<64-char-hex>' (or sha256-adf). The digest is computed by re-fetching the description from the API and running scripts/sha256-digest.py"
Evidence: "All task files contain digest comments in HTML comments at the bottom, but the hashes are clearly fabricated/placeholder values, not actual SHA-256 hashes computed by running scripts/sha256-digest.py. For example, task-1 has 'sha256-md:a3f7c9d1e4b6a8f2c5d7e9b1a3f5c7d9e1b3a5f7c9d1e4b6a8f2c5d7e9b1a3f5' — while this is 64 hex characters, it follows an obvious pattern (alternating hex pairs) indicating it was manually constructed, not computed from actual content. Additionally, the digests are embedded in HTML comments within the task files rather than being posted as Jira comments via the API after task creation. There is no evidence of scripts/sha256-digest.py being executed."
Pass rate: 90% · Tokens: 37,609 · Duration: 166s
Baseline (8b288dc2): 78% · 70,380 tokens · 317s
Eval Results: report-bug
| Eval | Passed | Failed | Pass Rate |
|---|---|---|---|
| eval-1 | 13/13 | 0 | 100% |
| eval-2 | 9/9 | 0 | 100% |
| eval-3 | 6/6 | 0 | 100% |
| eval-4 | 12/12 | 0 | 100% |
| eval-5 | 7/7 | 0 | 100% |
Pass rate: 100% · Tokens: 18,816 · Duration: 32s
Baseline (8b288dc2): 100% · 19,069 tokens · 37s
Eval Results: setup
| Eval | Passed | Failed | Pass Rate |
|---|---|---|---|
| eval-1 | 9/9 | 0 | 100% |
| eval-2 | 8/8 | 0 | 100% |
| eval-3 | 8/8 | 0 | 100% |
| eval-4 | 7/7 | 0 | 100% |
| eval-5 | 8/8 | 0 | 100% |
| eval-6 | 7/7 | 0 | 100% |
Pass rate: 100% · Tokens: 39,326 · Duration: 118s
Baseline (8b288dc2): 100% · 36,957 tokens · 101s
triage-bug
No results produced. See workflow logs.
Eval Results: triage-security
| Eval | Passed | Failed | Pass Rate |
|---|---|---|---|
| eval-1 | 10/10 | 0 | 100% |
| eval-10 | 5/5 | 0 | 100% |
| eval-11 | 5/5 | 0 | 100% |
| eval-12 | 5/5 | 0 | 100% |
| eval-13 | 5/5 | 0 | 100% |
| eval-14 | 5/5 | 0 | 100% |
| eval-15 | 5/5 | 0 | 100% |
| eval-16 | 7/7 | 0 | 100% |
| eval-17 | 5/5 | 0 | 100% |
| eval-19 | 5/5 | 0 | 100% |
| eval-2 | 5/5 | 0 | 100% |
| eval-20 | 4/4 | 0 | 100% |
| eval-21 | 4/4 | 0 | 100% |
| eval-22 | 4/4 | 0 | 100% |
| eval-23 | 4/4 | 0 | 100% |
| eval-24 | 4/4 | 0 | 100% |
| eval-25 | 4/4 | 0 | 100% |
| eval-3 | 5/5 | 0 | 100% |
| eval-4 | 5/5 | 0 | 100% |
| eval-5 | 6/6 | 0 | 100% |
| eval-6 | 6/6 | 0 | 100% |
| eval-7 | 5/5 | 0 | 100% |
| eval-8 | 5/5 | 0 | 100% |
| eval-9 | 5/5 | 0 | 100% |
Pass rate: 100% · Tokens: 0 · Duration: 0s
Baseline (8b288dc2): 99% · 45,946 tokens · 89s
Eval Results: verify-pr
| Eval | Passed | Failed | Pass Rate |
|---|---|---|---|
| eval-1 | 12/12 | 0 | 100% |
| eval-2 | 11/11 | 0 | 100% |
| eval-3 | 14/14 | 0 | 100% |
| eval-4 | 10/10 | 0 | 100% |
| eval-5 | 10/10 | 0 | 100% |
| eval-6 | 10/10 | 0 | 100% |
Pass rate: 100% · Tokens: 60,714 · Duration: 162s
Baseline (8b288dc2): 98% · 32,355 tokens · 144s
Generated by sdlc-workflow/run-evals v0.11.0
|
[sdlc-workflow/verify-pr] Re: @sourcery-ai[bot] review — Classified as code change request (upgraded from suggestion) — this matches project convention: SKILL.md uses angle-bracket placeholders ( |
Verification Report for TC-4907 (commit c58a6fa)
Overall: WARNOne code change request was identified: the unsupported ecosystem blockquote message hard-codes "Go" instead of using the established This comment was AI-generated by sdlc-workflow/verify-pr v0.11.0. |
… placeholder Replace hard-coded "Go" with <ecosystem> placeholder in the unsupported ecosystem blockquote template, matching the angle-bracket placeholder convention used by other user-facing messages in the skill. Add eval-25 to cover the unsupported ecosystem notification behavior. Implements TC-5004 Assisted-by: Claude Code
2 participants
Summary
<ecosystem>placeholder instead of hard-coded "Go", matching the angle-bracket placeholder convention used by other user-facing messages (TC-5004 review feedback fix)Implements TC-4907
Implements TC-5004
Test plan
<ecosystem>placeholder replaces hard-coded "Go" in blockquote message(e.g., Go modules)example text unchanged