Skip irrelevant review dimensions for docs-only PRs to reduce token cost

[fullsend-ai-retro]

What happened

On PR #5203, a 16-line annotation to a single ADR file (docs/ADRs/0027-allowed-and-disallowed-tools-for-agents.md), the review agent dispatched its full fleet of opus-tier sub-agents across all review dimensions on every successful run. The PR had 8 successful review runs, each dispatching sub-agents for correctness, security, intent-coherence, style-conventions, docs-currency, and cross-repo-contracts, plus the challenger.

For a docs-only change with no code modifications:

• Correctness sub-agent had no code logic, nil checks, edge cases, or API contracts to analyze
• Security sub-agent had no auth flows, data handling, or injection surfaces to examine
• Cross-repo-contracts sub-agent had no API changes, schema modifications, or breaking changes to detect

Only intent-coherence (ADR amendment scope), docs-currency (accuracy of documentation), and style-conventions (formatting, language) produced meaningful findings. The review agent itself applied the component/docs label, correctly identifying this as a documentation change, but did not use that classification to optimize dimension dispatch.

The review agent runs against the fullsend-ai/agents repo at commit 937a16b672ac0b535660b358f89e2538afc0bf4b, using the pr-review skill for orchestration.

What could go better

Docs-only PRs (where all changed files are documentation — matching patterns like docs/**/*.md, **/*.md, README*, CHANGELOG*, LICENSE*) should skip sub-agent dispatch for dimensions that cannot produce meaningful findings on non-code changes. This would reduce per-review token cost by approximately 30-50% for docs-only PRs without sacrificing review quality.

This is distinct from existing dimension-scoping proposals:

• agents#354 scopes re-review dispatch to dimensions with prior findings — this is about re-reviews, not first reviews
• agents#343 scopes re-review when the push only addresses prior findings — also re-review-specific
• agents#108 short-circuits on rebase-only pushes — different trigger condition

None of these address the first-review case where the change type itself (docs-only) makes certain dimensions irrelevant.

Confidence: High that correctness and security dimensions add no value on docs-only PRs. The dimensions are defined around code constructs (nil checks, race conditions, injection, auth bypass) that do not exist in markdown files.

Uncertainty: The definition of "docs-only" needs care. A PR that modifies both docs/foo.md and internal/bar.go is NOT docs-only. The classification should be conservative — any non-documentation file in the changeset should trigger the full dimension set.

Proposed change

In skills/pr-review/SKILL.md, add a change-type classification step after Step 2b (where the diff is loaded) and before Step 3 (sub-agent dispatch):

1. Classify change type: Check if ALL changed files match documentation patterns (e.g., *.md, docs/**, website/**, *.txt, LICENSE, CHANGELOG). If yes, classify as docs-only.

2. Dimension filtering for docs-only PRs: When the change type is docs-only, dispatch sub-agents only for dimensions relevant to documentation:

   • Keep: intent-coherence, docs-currency, style-conventions
   • Skip: correctness, security, cross-repo-contracts
   • Always keep: challenger (operates on whatever findings exist)

3. Log the optimization: When dimensions are skipped, include a note in the review output: "Dimensions skipped (docs-only PR): correctness, security, cross-repo-contracts."

4. Conservative fallback: If classification is uncertain (e.g., mixed file types, generated code in docs), dispatch the full dimension set. Err on the side of thoroughness.

This change lives in the pr-review skill's orchestration logic, not in individual sub-agent definitions.

Validation criteria

On the next 10 docs-only PRs (labeled component/docs or where all changed files are *.md):

• The review agent should dispatch at most 4 sub-agents (intent-coherence, docs-currency, style-conventions, challenger) instead of 7
• No docs-relevant findings should be missed compared to the full-dimension baseline
• Per-review token usage (measurable from the workflow run's billing data or transcript token counts) should decrease by at least 25%
• The review output should include a note indicating which dimensions were skipped and why

---

Generated by retro agent from fullsend-ai/fullsend#5203