feat(agent): cross-cycle volatility gauge for incident dedup + change awareness#96
Merged
Merged
Conversation
… awareness Closes #77. The coordinator is stateless between cycles and never read its own history, so a persistent anomaly (latency stays bad for N cycles) was re-diagnosed and re-escalated every cycle (alert spam), and every sighting was treated as the first. This adds a lightweight in-memory cross-cycle memory on the coordinator instance (which lives for the whole run loop). New VolatilityGauge (agent/volatility.py): - fingerprints each anomaly as anomaly_type:component (idempotent dedup key), - classifies a detection as NEW / ONGOING / WORSENING against history, and marks other previously-active incidents RESOLVED, - WORSENING when observed_value rose by >= agent_incident_worsen_pct since last cycle; ONGOING recurrence within agent_incident_ongoing_gap cycles, else NEW, - decides should_report: NEW/WORSENING always; ONGOING only until reported once (agent_incident_dedup), then suppressed, - prior_context() feeds a "since last cycle" summary into the next detection so an ongoing anomaly is recognized as such (change awareness). run_cycle now stamps a cycle number, observes each detection, injects prior context into detection, suppresses a reported unchanged ongoing incident before the expensive diagnose/report, marks reported after escalation, and logs resolved incidents on all-clear. Thresholds externalized to config, validated at startup. Tests: 11 gauge unit tests (fingerprint, new/ongoing/worsening/gap, suppression, not-yet-reported still reports, resolved, prior_context, all-clear, dedup-off), a run_cycle two-cycle integration test proving a persistent incident reports once and is suppressed the second cycle, and config defaults/validation. Full suite 273 passed; ruff + mypy clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #77.
Problem
The coordinator is stateless between cycles and never read its own history, so a persistent anomaly (latency stays bad for N cycles) was re-diagnosed and re-escalated every cycle (alert spam), and every sighting was treated as the first.
Fix
A lightweight in-memory cross-cycle memory on the coordinator instance (which lives for the whole run loop). New
VolatilityGauge(agent/volatility.py):anomaly_type:component(idempotent dedup key);observed_valuerose by >=agent_incident_worsen_pctsince last cycle; ONGOING recurrence withinagent_incident_ongoing_gapcycles, else NEW;agent_incident_dedup), then suppressed;prior_context()feeds a "since last cycle" summary into the next detection, so an ongoing anomaly is recognized as such (change awareness).run_cyclenow stamps a cycle number, observes each detection, injects prior context into detection, suppresses a reported unchanged ongoing incident before the expensive diagnose/report, marks reported after escalation, and logs resolved incidents on all-clear. Thresholds externalized to config, validated at startup.State is in-memory (resets on restart), an accepted tradeoff, durable history already lives in the audit log.
Tests
11 gauge unit tests (fingerprint, new/ongoing/worsening/gap, suppression, not-yet-reported-still-reports, resolved, prior_context, all-clear, dedup-off); a
run_cycletwo-cycle integration test proving a persistent incident reports once and is suppressed the second cycle; config defaults + validation.Full suite 273 passed; ruff + mypy clean.
Pairs with #67 (parallel diagnosis): together the coordinator now has cross-cycle and cross-hypothesis intelligence.
https://claude.ai/code/session_01R5VygSzbGTggW7mHd3PVwE