What happened
On PR #2869, the review agent's first run (29854473950) found a LOW-severity documentation bug (missing closing backticks in user-guide/05-version-updates.md) and posted an inline comment at 17:55 UTC. The author fixed it in commit 7a1f855 at 18:21 UTC, touching only that one file. This triggered a full re-review (29856874945) that re-analyzed all 183 changed files from scratch, took 5m20s, cost $1.70, and found no new issues. The re-review correctly dismissed the stale finding but did full-scope analysis to reach that conclusion. Total review cost was $3.13, with the re-review accounting for 54%.
What could go better
When a push exclusively modifies files that were flagged in the prior review's findings, the re-review could be narrowed to: (1) verify the fix addresses the finding correctly, (2) check for regressions in the modified files, (3) confirm no new issues in the delta between the last reviewed commit and HEAD. This would cost an estimated $0.30–$0.50 instead of $1.70, saving ~$1.20 per finding-fix cycle. Confidence is moderate — the main risk is that a finding-fix commit could introduce a subtle regression in an unrelated part of a touched file, though this is unlikely for small single-purpose fix commits. Full re-reviews would still run when the push touches files beyond the finding locations or when the delta is large. Existing issues #1390 (reduce unnecessary re-reviews on large PRs), #1287 (skip rebase-only re-reviews), and #270 (targeted side-effect analysis for HIGH/CRITICAL fix commits) address adjacent concerns but none cover the specific pattern of scoping re-reviews when the push only fixes the prior review's own findings.
Proposed change
In scripts/pre-review.sh, add detection logic for finding-fix commits: (1) fetch the prior review's inline comments from the GitHub API to extract finding file paths, (2) compute git diff --name-only <last_reviewed_commit>..HEAD to identify files changed since the prior review, (3) if all changed files are a subset of the prior finding locations, set REVIEW_MODE=finding-verify in the environment. In agents/review.md (or the review harness config), add a conditional instruction block: when REVIEW_MODE=finding-verify, scope analysis to verifying that each prior finding was addressed, checking for regressions in the changed files only, and reviewing the commit delta — skip full-PR analysis. The review agent should still post a summary indicating it performed a targeted re-review and why.
Validation criteria
On the next 5 re-reviews triggered by single-file or few-file finding-fix commits across all repos: (1) the targeted review completes in under 2 minutes and costs under $0.60, (2) the targeted review correctly verifies whether each prior finding was addressed, (3) no cases where a targeted review misses a regression that a full review would have caught — verify by running a shadow full review on the first 3 instances and comparing findings. If the shadow comparison reveals missed findings, tighten the eligibility criteria (e.g., require the delta to be under 20 lines).
Generated by retro agent from redhat-developer/rhdh-plugin-export-overlays#2463
What happened
On PR #2869, the review agent's first run (29854473950) found a LOW-severity documentation bug (missing closing backticks in
user-guide/05-version-updates.md) and posted an inline comment at 17:55 UTC. The author fixed it in commit7a1f855at 18:21 UTC, touching only that one file. This triggered a full re-review (29856874945) that re-analyzed all 183 changed files from scratch, took 5m20s, cost $1.70, and found no new issues. The re-review correctly dismissed the stale finding but did full-scope analysis to reach that conclusion. Total review cost was $3.13, with the re-review accounting for 54%.What could go better
When a push exclusively modifies files that were flagged in the prior review's findings, the re-review could be narrowed to: (1) verify the fix addresses the finding correctly, (2) check for regressions in the modified files, (3) confirm no new issues in the delta between the last reviewed commit and HEAD. This would cost an estimated $0.30–$0.50 instead of $1.70, saving ~$1.20 per finding-fix cycle. Confidence is moderate — the main risk is that a finding-fix commit could introduce a subtle regression in an unrelated part of a touched file, though this is unlikely for small single-purpose fix commits. Full re-reviews would still run when the push touches files beyond the finding locations or when the delta is large. Existing issues #1390 (reduce unnecessary re-reviews on large PRs), #1287 (skip rebase-only re-reviews), and #270 (targeted side-effect analysis for HIGH/CRITICAL fix commits) address adjacent concerns but none cover the specific pattern of scoping re-reviews when the push only fixes the prior review's own findings.
Proposed change
In
scripts/pre-review.sh, add detection logic for finding-fix commits: (1) fetch the prior review's inline comments from the GitHub API to extract finding file paths, (2) computegit diff --name-only <last_reviewed_commit>..HEADto identify files changed since the prior review, (3) if all changed files are a subset of the prior finding locations, setREVIEW_MODE=finding-verifyin the environment. Inagents/review.md(or the review harness config), add a conditional instruction block: whenREVIEW_MODE=finding-verify, scope analysis to verifying that each prior finding was addressed, checking for regressions in the changed files only, and reviewing the commit delta — skip full-PR analysis. The review agent should still post a summary indicating it performed a targeted re-review and why.Validation criteria
On the next 5 re-reviews triggered by single-file or few-file finding-fix commits across all repos: (1) the targeted review completes in under 2 minutes and costs under $0.60, (2) the targeted review correctly verifies whether each prior finding was addressed, (3) no cases where a targeted review misses a regression that a full review would have caught — verify by running a shadow full review on the first 3 instances and comparing findings. If the shadow comparison reveals missed findings, tighten the eligibility criteria (e.g., require the delta to be under 20 lines).
Generated by retro agent from redhat-developer/rhdh-plugin-export-overlays#2463