Skip to content

feat(materials): record checkmarx engine types via scan.types annotation#3292

Merged
javirln merged 2 commits into
chainloop-dev:mainfrom
javirln:javier/com-372-checkmarx-scan-types-annotation
Jul 17, 2026
Merged

feat(materials): record checkmarx engine types via scan.types annotation#3292
javirln merged 2 commits into
chainloop-dev:mainfrom
javirln:javier/com-372-checkmarx-scan-types-annotation

Conversation

@javirln

@javirln javirln commented Jul 16, 2026

Copy link
Copy Markdown
Member

Description

The Checkmarx One native JSON report (CHECKMARX_JSON) is multi-engine: a single report can bundle SAST, SCA, KICS (IaC) and container findings. At the attestation level, the crafter previously injected only chainloop.material.tool.name = checkmarx, so there was no way to tell which engines actually produced findings.

This change has the Checkmarx crafter record the distinct engine types present in the report as a new chainloop.material.scan.types annotation (lower-cased, sorted, comma-joined; e.g. kics,sast,sca). Annotation-only consumers can then distinguish engines without reading material content. A clean or null report advertises no types, so the annotation is omitted rather than set empty.

AI disclosure

This contribution was assisted by Claude Code.

Review in cubic

The Checkmarx crafter is the one place that sees report content, so have it
record the distinct engine types present in a CHECKMARX_JSON report as a
chainloop.material.scan.types annotation (lower-cased, sorted, comma-joined;
e.g. "kics,sast,sca").

This lets attestation-level policies that match purely by annotation (the
*-scan-present compliance policies) tell which engines actually produced
findings without reading material content, avoiding over-claim on multi-engine
reports. A clean or null report advertises no types, so the annotation is
omitted rather than set empty and recognition fails closed.

Assisted-by: Claude Code
Signed-off-by: Javier Rodriguez <javier@chainloop.dev>

Chainloop-Trace-Sessions: 23990771-b817-46fa-8543-353be0e9155a
@chainloop-platform

chainloop-platform Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

AI Session Analysis

Avg score Sessions Failing policies Attribution Files Lines Total Duration
🟢 85% 1 ✅ 0 68% AI / 32% Human 4 +124 / -15 1h12m45s

🟢 85% — 68% AI — ✅ All policies passing

Jul 15, 2026 09:24 UTC · 1h12m45s · $33.49 · 599 in / 268.1k out · claude-code 2.1.210 (claude-opus-4-8)

View session details ↗

Change Summary

  • Adds a new CHECKMARX_JSON material type and crafter for native Checkmarx JSON reports.
  • Updates proto/schema wiring, generated API artifacts, CLI docs, and material dispatch.
  • Adds real upstream-based Checkmarx fixtures plus clean/null variants and targeted crafter tests.

AI Session Overall Score

🟢 85% — Well-planned, thoroughly validated implementation with one corrected provenance overstatement.

AI Session Analysis Breakdown

🟢 92% · user-trust-signal

No notes.

🟢 90% · context-and-planning

🟢 AI grounded the feature in the linked issue and wrote a design before coding. · High Impact

🟢 89% · verification

🟢 AI repeatedly reran build, tests, and lint, including mixed-engine and null-results cases. · High Impact

🟢 88% · solution-quality

🟢 AI found and fixed the results:null edge case instead of papering over it. · High Impact

🟢 86% · scope-discipline

No notes.

🟡 72% · alignment

🟠 AI first described all testdata as real, then later clarified one clean fixture was derived. · Medium Severity

💡 Before claiming fixtures are real, separate verbatim upstream samples from derived clean cases in the summary.


File Attribution

█████████████░░░░░░░ 68% AI / 32% Human

Status Attribution File Lines
modified ai pkg/attestation/crafter/materials/checkmarx.go +51 / -9
modified ai pkg/attestation/crafter/materials/checkmarx_test.go +27 / -2
modified ai pkg/attestation/crafter/materials/materials.go +25 / -4
created human pkg/attestation/crafter/materials/testdata/checkmarx-extra-engines.json +21 / -0

Policies (4)

Status Policy Material Messages
✅ Passed ai-config-ai-agents-allowed ai-coding-session-239907 -
✅ Passed ai-config-no-dangerous-commands ai-coding-session-239907 -
✅ Passed ai-config-no-secrets ai-coding-session-239907 -
✅ Passed ai-config-mcp-servers-allowed ai-coding-session-239907 -

Powered by Chainloop and Chainloop Trace

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

Re-trigger cubic

@jiparis

jiparis commented Jul 16, 2026

Copy link
Copy Markdown
Member

kics,sast,sca. Looks weird t me. It's mixing specific vendors with generic scan types. Do you have a real example? Should we extend this to all materials?

@javirln

javirln commented Jul 16, 2026

Copy link
Copy Markdown
Member Author

kics,sast,sca. Looks weird t me. It's mixing specific vendors with generic scan types. Do you have a real example? Should we extend this to all materials?

Yes it's weird but at the same time is how the tool outputs it. The weird thing is the tool mixes on the same report different angles, kics,sast,sca.

This is a real example: https://github.com/chainloop-dev/chainloop/blob/main/pkg/attestation/crafter/materials/testdata/checkmarx.json

I'll try to come up with something different

@javirln
javirln marked this pull request as draft July 16, 2026 10:54
…cabulary

Checkmarx names some engines by product (e.g. "kics") rather than by category,
so emitting the raw values mixed vendor-specific names with generic scan types
in the chainloop.material.scan.types annotation.

Introduce a canonical ScanType* vocabulary (sast, sca, iac, container,
supply-chain, secrets) in the shared materials package and map the Checkmarx
engine identifiers onto it (kics -> iac, containers -> container, sscs ->
supply-chain). The annotation now carries only generic scan types, so it stays
consistent across material kinds and can be adopted by other multi-engine
materials.

An engine that maps to no canonical type is dropped from the annotation and
logged, so recognition fails closed and no vendor-specific name leaks out.

Assisted-by: Claude Code
Signed-off-by: Javier Rodriguez <javier@chainloop.dev>

Chainloop-Trace-Sessions: 23990771-b817-46fa-8543-353be0e9155a
@javirln
javirln marked this pull request as ready for review July 16, 2026 14:05
@javirln
javirln merged commit b9c21ed into chainloop-dev:main Jul 17, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants