Skip to content

[DEVEX-1523] Overhaul AI reviewer prompt to reduce noise#124

Merged
ronneseth merged 1 commit into
mainfrom
DEVEX-1523-improve-review-prompt
May 19, 2026
Merged

[DEVEX-1523] Overhaul AI reviewer prompt to reduce noise#124
ronneseth merged 1 commit into
mainfrom
DEVEX-1523-improve-review-prompt

Conversation

@ronneseth
Copy link
Copy Markdown
Contributor

Summary

  • Claude now reads existing review threads before posting and skips any issue already discussed
  • Three clear decision paths: silent approve, inline comments only (no summary), or defer on complex PRs
  • Skips very large diffs (>1500 lines) entirely rather than posting incomplete reviews
  • No more summary comments on approval — just a silent approve
  • Never re-raises issues a human already responded to

JIRA: DEVEX-1523

Made with Cursor

@ronneseth ronneseth requested a review from a team as a code owner May 19, 2026 22:12
@ronneseth ronneseth requested a review from aspencer May 19, 2026 22:12
@cursor
Copy link
Copy Markdown

cursor Bot commented May 19, 2026

PR Summary

Low Risk
Low risk workflow/prompt-only change, but it alters automated review behavior (can now defer on large/complex diffs and avoids re-commenting), which may affect CI review outcomes.

Overview
Updates the claude-review.yaml GitHub Action prompt to be more conservative and less noisy by reading existing PR/inline review threads first and only raising new issues.

Adds a configurable max_diff_lines threshold (default 1500) to defer to human review for large/complex PRs, and restructures the instructions into two explicit outcomes: silent approve (AI Review: Approved) or inline-comments + comment review (no approvals when concerns exist). Also expands allowed tooling to include gh api for fetching existing review comments.

Reviewed by Cursor Bugbot for commit 818d233. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread .github/workflows/claude-review.yaml
@ronneseth ronneseth force-pushed the DEVEX-1523-improve-review-prompt branch from 3953f4f to 2d5b60c Compare May 19, 2026 22:16
Comment thread .github/workflows/claude-review.yaml Outdated
@ronneseth ronneseth force-pushed the DEVEX-1523-improve-review-prompt branch from 2d5b60c to c469e1f Compare May 19, 2026 22:21
- Check existing review threads before posting; never re-raise issues
  a human already responded to
- Three clear options: silent approve, inline comments only, or defer
  on complex PRs
- No summary comments on approval — just approve silently
- Skip very large diffs (>1500 lines) entirely
- Never post duplicate comments

Co-authored-by: Cursor <cursoragent@cursor.com>
@ronneseth ronneseth force-pushed the DEVEX-1523-improve-review-prompt branch from c469e1f to 818d233 Compare May 19, 2026 22:23
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 818d233. Configure here.

2. If the diff exceeds ${{ inputs.max_diff_lines }} lines changed, or the PR is too complex to confidently review (many files, complex logic across multiple systems, architectural changes), defer to human review:
`gh pr review ${{ github.event.pull_request.number }} --comment --body "AI Review: Deferring to human review — this PR exceeds the automated review threshold."`
Then stop. Do NOT approve. Do NOT post inline comments.
3. Run `gh api repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/comments` to read ALL existing inline review comments and reply threads. Also run `gh pr view ${{ github.event.pull_request.number }} --comments` for top-level PR comments.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review comments fetch not paginated

Medium Severity

Step 1 tells the reviewer to load all existing inline comments via gh api, but the command has no --paginate. GitHub returns about 30 review comments per page, so busy PRs can hide older threads and the model may re-raise resolved issues or duplicate comments despite the new rules.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 818d233. Configure here.

@ronneseth ronneseth merged commit 06a8176 into main May 19, 2026
2 checks passed
@ronneseth ronneseth deleted the DEVEX-1523-improve-review-prompt branch May 19, 2026 23:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant