fix(sight): avoid false interruption signals#1393
Conversation
|
Thanks for splitting this out — the false-positive suppression goal makes sense. One concern on the recall side: After this change Concretely: Anthropic Messages streaming returns 200 then (Reproduced pre/post against the SSE parse path.) Flagging in case the 200-status-with-error-body case is collateral rather than intended. |
966b627 to
a04de87
Compare
Only provider error text and non-2xx response bodies feed runtime interruption classification. Successful assistant output may mention timeout, auth, or rate-limit troubleshooting. Those phrases should not create captured interruption events by themselves. Assisted-by: Codex Signed-off-by: wanghao <1562495626@qq.com>
Parse structured error envelopes from successful SSE streams before truncated-stream detection. Keep ordinary HTTP 200 assistant text out of keyword matching. Restore recall for provider errors emitted after the stream status line is committed. The detector only trusts explicit error-shaped events. Assisted-by: Codex Signed-off-by: wanghao <1562495626@qq.com>
a04de87 to
9d1e8e2
Compare
|
Thanks — this closes the recall regression I raised in round 1. I verified by building and running this time, not just reading the diff:
Also confirmed my main worry does not happen: a >1s 200-SSE error stream is not double-classified — rule 7 ( Two non-blocking notes for later (not for this PR):
LGTM on the recall fix. |
jfeng18
left a comment
There was a problem hiding this comment.
Recall regression from round 1 is closed and verified by build + tests + mutation (42 passed; neutralizing the fix flips the two new SSE tests). No blockers; the two notes above are non-blocking follow-ups. LGTM.
Description
Avoid false interruption records when a successful assistant response mentions troubleshooting words such as timeout, auth, or rate limit. The detector now only scans structured provider error text and non-2xx response bodies for runtime error classifications.
This PR is split out of #1349 so the interruption classifier fix can be reviewed independently. It also includes the current-toolchain mechanical AgentSight clippy fixes required for this split PR to pass
-D warningsindependently.Related Issue
Refs #1304
Type of Change
Scope
cosh(copilot-shell)cosh-ng(cosh-ng)sec-core(agent-sec-core)skill(os-skills)sight(agentsight)tokenless(tokenless)ckpt(ws-ckpt)memory(agent-memory)anolisa(anolisa-cli)skillfs(SkillFS)Checklist
cosh: Lint passes, type check passes, and tests passcosh-ng:cargo clippy --all-targets -- -D warningsandcargo fmt --checkpasssec-core(Rust):cargo clippy -- -D warningsandcargo fmt --checkpasssec-core(Python): Ruff format and pytest passskill: Skill directory structure is valid and shell scripts pass syntax checksight:cargo clippy -- -D warningsandcargo fmt --checkpasstokenless:cargo clippy -- -D warningsandcargo fmt --checkpassmemory(Linux only):cargo clippy --all-targets -- -D warnings,cargo fmt --check, andcargo testpassanolisa:cargo clippy --all-targets --locked -- -D warnings,cargo fmt --all --check, andcargo test --lockedpassskillfs:cargo fmt --all --check,cargo clippy --workspace --all-targets -- -D warnings, andcargo test --workspacepasspackage-lock.json/Cargo.lock)Testing
Ran on the Linux AgentSight test host from an archive of this split branch:
cargo fmt --all -- --checkcargo test -p agentsight interruption::detector --locked --quietcargo clippy -q -p agentsight --all-targets --locked -- -D warningsAdditional Notes
Split from #1349 following review feedback to keep capture fixes and the grader feature reviewable in smaller units.