Skip to content

fix(metrics): scope failed_updates to last day and add group label - #1568

Draft
Harsh63870 wants to merge 1 commit into
flatcar:mainfrom
Harsh63870:fix/failed-updates-metrics-window-group
Draft

fix(metrics): scope failed_updates to last day and add group label#1568
Harsh63870 wants to merge 1 commit into
flatcar:mainfrom
Harsh63870:fix/failed-updates-metrics-window-group

Conversation

@Harsh63870

Copy link
Copy Markdown

Summary

Fixes #1567.

nebraska_failed_updates previously counted every failed update event ever recorded and only exposed an application label. That made it hard to use for current rollout health.

This PR:

  • Limits counts to failed update-complete events from the last day (same window as other reporting queries)
  • Adds a group label (via instance_applicationgroups; missing group → unknown)
  • Resets the gauge vector before each refresh to avoid stale series

Test plan

  • cd backend && go test ./pkg/api/ -run TestGetFailedUpdatesMetrics
  • Scrape /metrics and confirm series look like nebraska_failed_updates{application="...",group="..."}
  • Confirm old failures outside the 1-day window are not counted

Notes

Adding the group label changes the metric’s label set. Existing Grafana queries that only use application can keep working with sum by (application) (nebraska_failed_updates).

Filter nebraska_failed_updates to the last day, break it down by group, and reset the gauge before refresh so operators can alert on recent per-group failure rates. Fixes flatcar#1567.

Signed-off-by: Harsh63870 <harshvardhanpandey372@gmail.com>
Copilot AI lite review requested due to automatic review settings August 11, 2026 20:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

metrics: nebraska_failed_updates is lifetime-only and has no group dimension

2 participants