Note: This issue was generated with AI assistance (GitHub Copilot) based on automated triage experience.
Filed by @canonical/solutions-qa
Priority
Low. There is a straightforward workaround (see below), so this is a usability/ergonomics improvement rather than a blocker.
Summary
Allow issue-to-test-result attachment rules to be scoped to a specific artefact (charm/snap). Today the attachment-rule create schema (IssueTestResultAttachmentRulePostRequest) accepts families, environment_names, test_case_names, template_ids, test_result_statuses, and execution_metadata — but no artefact dimension.
Problem
When a single charm fails with a variable failure signature, there is no way to write one clean rule:
- Keying on
execution_metadata (a specific endpoint/message) matches only one variant — you'd need N rules for N signatures.
- Keying on
test_case_names (e.g. test_build_bundle) matches that test case for every charm, which massively over-attaches.
- Net result: the issue must be attached by explicit result-ID lists, and future recurrences are not auto-attached.
Workaround (why this is low priority)
The situation is easy to work around today:
- Attach the issue to the affected results directly via an explicit
test_result_ids list (precise, no over-matching), or
- Create one narrow rule per stable
execution_metadata signature.
The only real cost is manual re-attachment when new failures of the same variable-signature bug appear.
Evidence
self-signed-certificates bundle-build fails across ~30 different neighbor-endpoint signatures (charm-integration-testing#747). With no artefact-scoped rule available, the issue was manually attached three times (revisions v650 ×2, v652), ~236 explicit result IDs total, with no auto-attach going forward. By contrast, kubeapi-load-balancer and nova-cloud-controller had a single stable signature, so a normal execution_metadata rule worked cleanly.
Proposed change
- API: add optional artefact filters to
IssueTestResultAttachmentRulePostRequest and the rule-matching logic — at minimum artefacts (names); ideally also artefact_tracks, artefact_versions, artefact_stages to mirror the dimensions already returned by GET .../attachment-rule-filters under artefact_filters.
- MCP server (
canonical/test-observer-mcp): expose the new filter(s) in the create_attachment_rule tool (add artefacts to createAttachmentRuleInput) and forward them.
Example enabled after the change
create_attachment_rule(
issue_id=<id>,
artefacts=["self-signed-certificates"],
test_case_names=["test_build_bundle"],
test_result_statuses=["FAILED"],
)
Matches every SSC bundle-build failure regardless of which neighbor endpoint is unfulfilled, without touching any other charm.
Acceptance criteria
- A rule with
artefacts=[X] matches only results whose artefact name is X.
- Combinable with existing filters (test_case, families, status, execution_metadata).
matching_test_results_count reflects the artefact scope.- MCP
create_attachment_rule accepts and forwards artefacts (+ optional track/version/stage).
Caveat
Artefact-scoped rules should still generally be paired with a failure-signature dimension (test_case / template / execution_metadata) so a rule doesn't attach an issue to all of a charm's failures across unrelated bugs.
Note: This issue was generated with AI assistance (GitHub Copilot) based on automated triage experience.
Filed by @canonical/solutions-qa
Priority
Low. There is a straightforward workaround (see below), so this is a usability/ergonomics improvement rather than a blocker.
Summary
Allow issue-to-test-result attachment rules to be scoped to a specific artefact (charm/snap). Today the attachment-rule create schema (
IssueTestResultAttachmentRulePostRequest) acceptsfamilies,environment_names,test_case_names,template_ids,test_result_statuses, andexecution_metadata— but no artefact dimension.Problem
When a single charm fails with a variable failure signature, there is no way to write one clean rule:
execution_metadata(a specific endpoint/message) matches only one variant — you'd need N rules for N signatures.test_case_names(e.g.test_build_bundle) matches that test case for every charm, which massively over-attaches.Workaround (why this is low priority)
The situation is easy to work around today:
test_result_idslist (precise, no over-matching), orexecution_metadatasignature.The only real cost is manual re-attachment when new failures of the same variable-signature bug appear.
Evidence
self-signed-certificatesbundle-build fails across ~30 different neighbor-endpoint signatures (charm-integration-testing#747). With no artefact-scoped rule available, the issue was manually attached three times (revisions v650 ×2, v652), ~236 explicit result IDs total, with no auto-attach going forward. By contrast,kubeapi-load-balancerandnova-cloud-controllerhad a single stable signature, so a normalexecution_metadatarule worked cleanly.Proposed change
IssueTestResultAttachmentRulePostRequestand the rule-matching logic — at minimumartefacts(names); ideally alsoartefact_tracks,artefact_versions,artefact_stagesto mirror the dimensions already returned byGET .../attachment-rule-filtersunderartefact_filters.canonical/test-observer-mcp): expose the new filter(s) in thecreate_attachment_ruletool (addartefactstocreateAttachmentRuleInput) and forward them.Example enabled after the change
Matches every SSC bundle-build failure regardless of which neighbor endpoint is unfulfilled, without touching any other charm.
Acceptance criteria
artefacts=[X]matches only results whose artefact name is X.matching_test_results_countreflects the artefact scope.create_attachment_ruleaccepts and forwardsartefacts(+ optional track/version/stage).Caveat
Artefact-scoped rules should still generally be paired with a failure-signature dimension (test_case / template / execution_metadata) so a rule doesn't attach an issue to all of a charm's failures across unrelated bugs.