Skip to content

Check for TEST_OUTPUT in RPM build tasks #1709

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
jorris wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Check for TEST_OUTPUT in RPM build tasks #1709

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions policy/release/test/test.rego
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,30 @@
result := metadata.result_helper(rego.metadata.chain(), [])
}

# METADATA
# title: RPM Test data found in task results
# description: >-
# This is a temporary duplicate of test_data_found. The RPM build pipeline
# will also require TEST_OUTPUT, but initially use a warning until we are certain
# everything works without breaking the build pipeline.
# custom:
# short_name: rpm_test_data_found
# failure_msg: No RPM test data found
# solution: >-
# Confirm at least one task in the RPM build pipeline contains a result named TEST_OUTPUT.
# collections:
# - redhat_rpms
# depends_on:
# - attestation_type.known_attestation_type
#
warn contains result if {

Check failure on line 106 in policy/release/test/test.rego

View workflow job for this annotation

GitHub Actions / all-tests-and-checks 

Messy incremental rule. To learn more, see: https://www.openpolicyagent.org/projects/regal/rules/style/messy-rule
count(lib.pipelinerun_attestations) > 0 # make sure we're looking at a PipelineRun attestation
results := lib.results_from_tests
count(results) == 0 # there are none at all

Comment on lines +106 to +110

@coderabbitai coderabbitai Bot Apr 6, 2026 •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Scope this warning to RPM attestations only

Line 108 uses lib.results_from_tests, which aggregates TEST_OUTPUT from all PipelineRun tasks, not RPM-only. That means this rule can miss missing RPM test output when non-RPM tasks contain TEST_OUTPUT, and can also warn on non-RPM-only inputs. Please filter attestations/results to RPM builds before evaluating count(results) == 0.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@policy/release/test/test.rego` around lines 106 - 110, The warning currently
uses lib.results_from_tests over lib.pipelinerun_attestations and thus examines
all TEST_OUTPUT entries; restrict the scope to RPM-only before counting: filter
lib.pipelinerun_attestations to RPM attestations (e.g., by an attestation field
that identifies RPM builds or task name) and then derive results from that
filtered set (or filter lib.results_from_tests output for entries with
TEST_OUTPUT that are tied to RPM attestations) so that the check in warn
contains result if uses only RPM-related results when evaluating count(results)
== 0.

@simonbaird simonbaird Apr 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine, because we're expecting one build pipeline attestation, or if there is multiple for some reason, all of them will be rpm builds.

@coderabbitai coderabbitai Bot Apr 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@simonbaird, that makes sense — since the policy is already scoped to the redhat_rpms collection and the pipeline will only ever have RPM build attestations in that context, the existing check is sufficient. I'll drop this concern.

📝 Learnings were identified and not saved (knowledge base disabled). Enable

result := metadata.result_helper(rego.metadata.chain(), [])
}

# METADATA
# title: Test data includes results key
# description: >-
Expand All @@ -103,7 +127,7 @@
# depends_on:
# - test.test_data_found
#
deny contains result if {

Check failure on line 130 in policy/release/test/test.rego

View workflow job for this annotation

GitHub Actions / all-tests-and-checks 

Messy incremental rule. To learn more, see: https://www.openpolicyagent.org/projects/regal/rules/style/messy-rule
with_results := [r.value.result | some r in lib.results_from_tests]
count(with_results) != count(lib.results_from_tests)
result := metadata.result_helper(rego.metadata.chain(), [])
Expand Down
18 changes: 18 additions & 0 deletions policy/release/test/test_test.rego
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,24 @@ test_needs_non_empty_data if {
}}) with input.attestations as attestations
}

# Because TEST_OUTPUT isn't in the task results, the lib.results_from_tests will be empty
test_rpm_needs_non_empty_data if {
_task_base := tekton_test.slsav1_task("task2")
slsav1_task = tekton_test.with_results(
_task_base,
[{"name": "NOT_TEST_OUTPUT", "type": "string", "value": {}}],
)

attestations := [
lib_test.att_mock_helper_ref("NOT_TEST_OUTPUT", {}, "task1", _bundle),
tekton_test.slsav1_attestation([slsav1_task]),
]
assertions.assert_equal_results(test.warn, {{
"code": "test.rpm_test_data_found",
"msg": "No RPM test data found",
}}) with input.attestations as attestations
}

# There is a test result, but the data inside it doesn't include the "result" key
test_needs_tests_with_results if {
_task_base := tekton_test.slsav1_task("task2")
Expand Down
Loading