Skip to content

Remove --max-turns limits from issue_solver and pr_followup - #456

Merged
stbenjam merged 1 commit into
stbenjam:mainfrom
not-stbenjam:remove-max-turns-limits
Jul 26, 2026
Merged

Remove --max-turns limits from issue_solver and pr_followup#456
stbenjam merged 1 commit into
stbenjam:mainfrom
not-stbenjam:remove-max-turns-limits

Conversation

@not-stbenjam

@not-stbenjam not-stbenjam commented Jul 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Remove --max-turns 200 from the issue_solver workflow
  • Remove --max-turns 100 from the pr_followup workflow

These caps truncate agent work on complex tasks, losing progress mid-solve. Without them, Claude runs to completion naturally.

Evidence

Failed run where work was truncated by the turns limit:
https://github.com/stbenjam/skillsaw/actions/runs/30209909213/job/89814287177

Test plan

  • Changes are minimal — only the two --max-turns lines removed
  • Verify next issue_solver and pr_followup runs complete without truncation

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Chores
    • Updated automated issue-solving and pull-request follow-up workflows to remove fixed limits on processing turns.
    • Preserved existing model selection and enhanced-effort settings for applicable tasks.

The max-turns caps (200 for issue_solver, 100 for pr_followup) truncate
agent work on complex tasks, losing progress mid-solve. Removing these
limits lets Claude run to completion.

Evidence: https://github.com/stbenjam/skillsaw/actions/runs/30209909213/job/89814287177

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@not-stbenjam
not-stbenjam requested a review from stbenjam as a code owner July 26, 2026 17:07
@coderabbitai

coderabbitai Bot commented Jul 26, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 20e6430b-b20d-40f4-af21-bc9ec7622f69

📥 Commits

Reviewing files that changed from the base of the PR and between 5cdb537 and 4ce023f.

📒 Files selected for processing (2)
  • .github/workflows/skillsaw-issue-solver.yml
  • .github/workflows/skillsaw-pr-followup.yml
💤 Files with no reviewable changes (2)
  • .github/workflows/skillsaw-issue-solver.yml
  • .github/workflows/skillsaw-pr-followup.yml

📝 Walkthrough

Walkthrough

Removes conditional --max-turns limits from Claude action invocations in the issue solver and PR follow-up workflows. Existing model selection and ultracode effort arguments remain unchanged.

Changes

Claude workflow argument updates

Layer / File(s) Summary
Remove conditional turn caps
.github/workflows/skillsaw-issue-solver.yml, .github/workflows/skillsaw-pr-followup.yml
Removes the non-ultracode --max-turns 200 and --max-turns 100 arguments while retaining model and ultracode effort configuration.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Possibly related PRs

Suggested reviewers: stbenjam

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and accurately summarizes the main change: removing max-turns limits from both workflows.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Remove max-turns caps from Issue Solver and PR Follow-up workflows

⚙️ Configuration changes ✨ Enhancement 🕐 Less than 10 minutes

Grey Divider

AI Description

• Remove --max-turns caps that truncated long-running agent sessions mid-task.
• Let claude-code-action run to natural completion for complex issues and PR follow-ups.
Diagram

graph TD
  Trigger["Schedule / workflow_dispatch"] --> IssueWF["Issue Solver workflow"] --> ClaudeAction["claude-code-action"] --> GitHub["GitHub repo + API"]
  Trigger --> FollowWF["PR Follow-up workflow"] --> ClaudeAction
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Raise the caps instead of removing
  • ➕ Retains a hard-stop safety valve against runaway loops/costs
  • ➕ Still reduces truncation frequency for complex tasks
  • ➖ Still fails on the longest tasks; progress can still be cut off mid-solve
  • ➖ Requires ongoing tuning as task complexity varies
2. Make max-turns dynamic (label/env-driven)
  • ➕ Allows higher limits for explicitly opted-in runs (e.g., ultracode, priority issues)
  • ➕ Keeps a default guardrail for routine tasks
  • ➖ More workflow complexity and more knobs to maintain
  • ➖ Still subject to misconfiguration or incorrect labeling
3. Replace turn cap with job-level timeout/cost guardrails
  • ➕ Prevents indefinite runtime without truncating reasoning progress arbitrarily by turns
  • ➕ Aligns with platform-native controls (e.g., timeout-minutes)
  • ➖ Timeouts still cut off work, just on elapsed time instead of turns
  • ➖ Doesn’t directly bound token usage unless paired with additional controls

Recommendation: Removing --max-turns is the simplest way to stop turn-based truncation and matches the stated goal. If cost/runtime guardrails are still needed, prefer adding/adjusting job-level timeout-minutes (or other budgeting controls) rather than reintroducing a low turn cap that can cut off complex runs mid-progress.

Files changed (2) +0 / -2

Other (2) +0 / -2
skillsaw-issue-solver.ymlRemove conditional '--max-turns 200' from claude args +0/-1

Remove conditional '--max-turns 200' from claude args

• Drops the non-ultracode conditional '--max-turns 200' flag from the claude-code-action invocation. This prevents issue-solver runs from being terminated early due to turn limits.

.github/workflows/skillsaw-issue-solver.yml

skillsaw-pr-followup.ymlRemove conditional '--max-turns 100' from claude args +0/-1

Remove conditional '--max-turns 100' from claude args

• Drops the non-ultracode conditional '--max-turns 100' flag from the claude-code-action invocation. This prevents PR follow-up runs from being terminated early due to turn limits.

.github/workflows/skillsaw-pr-followup.yml

@qodo-code-review

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (1)

Context used
✅ Compliance rules (platform): 136 rules
✅ Skills: 5 invoked
  skillsaw-pr-review
  skillsaw-issue-solver
  skillsaw-pr-followup
  skillsaw-create-plugin
  skillsaw-review-panel

Grey Divider


Remediation recommended

1. Removed max-turns cap 🐞 Bug ☼ Reliability
Description
The Issue Solver and PR Follow-up workflows no longer pass explicit non-ultracode --max-turns caps
(previously 200 and 100 respectively), removing a turn-budget guardrail and allowing substantially
longer Claude runs than before. Because both are scheduled and share concurrency: skillsaw-agent
with cancel-in-progress: false, unusually long runs can serialize the queue and delay other agent
workflows from starting.
Code

.github/workflows/skillsaw-issue-solver.yml[R134-137]

          claude_args: |
            --dangerously-skip-permissions
-            ${{ !matrix.issue.ultracode && '--max-turns 200' || '' }}
            --model ${{ matrix.issue.fable && 'claude-fable-5' || 'claude-opus-5' }}
            ${{ matrix.issue.ultracode && '--effort ultracode' || '' }}
Relevance

⭐⭐ Medium

Workflow guardrails often accepted, but restoring max-turns contradicts PR goal; team may prefer
other mitigations.

PR-#370

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Both workflows run on a schedule and are serialized behind the same shared concurrency group
(skillsaw-agent) with cancel-in-progress: false, so any increase in runtime for a single Claude
invocation directly reduces throughput for other queued agent jobs (including future Issue Solver
runs, PR follow-ups, and other agent automation). The cited changes remove the explicit
non-ultracode --max-turns argument from the Claude invocation in these workflows, eliminating the
prior hard turn budget (200 for Issue Solver and 100 for PR Follow-up), while other Claude workflows
in the repository still use explicit --max-turns caps as a guardrail pattern.

.github/workflows/skillsaw-issue-solver.yml[3-11]
.github/workflows/skillsaw-issue-solver.yml[118-138]
.github/workflows/skillsaw-maintenance.yml[20-29]
.github/workflows/skillsaw-pr-followup.yml[3-11]
.github/workflows/skillsaw-pr-followup.yml[146-170]
.github/workflows/skillsaw-review-panel.yml[33-46]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The Issue Solver and PR Follow-up workflows removed the explicit `--max-turns` guardrail for non-ultracode runs, so the Claude steps can take much longer than they used to and potentially run unbounded.

## Issue Context
Both workflows are scheduled every 15 minutes and share a single concurrency group (`skillsaw-agent`) with `cancel-in-progress: false`, which means long-running executions serialize the pipeline and can delay subsequent scheduled agent work (including later Issue Solver runs and PR follow-ups).

## Fix Focus Areas
- .github/workflows/skillsaw-issue-solver.yml[3-11]
- .github/workflows/skillsaw-issue-solver.yml[100-138]
- .github/workflows/skillsaw-pr-followup.yml[3-11]
- .github/workflows/skillsaw-pr-followup.yml[128-170]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Informational

2. --max-turns removal untested 📜 Skill insight ▣ Testability
Description
This PR fixes truncated agent runs by removing --max-turns caps in two workflows, but adds no
regression test to prevent the truncation scenario from reappearing. Without a regression test, the
workflow behavior change can silently regress in future edits.
Code

.github/workflows/skillsaw-issue-solver.yml[136]

-            ${{ !matrix.issue.ultracode && '--max-turns 200' || '' }}
Relevance

⭐ Low

Team previously rejected adding regression tests for workflow behavior changes.

PR-#453

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The compliance checklist requires bug-fix PRs to include a regression test (PR Compliance ID
2315530). The changed workflows show the claude_args blocks where the --max-turns caps were
removed, but no corresponding regression test is included in this PR’s changes.

.github/workflows/skillsaw-issue-solver.yml[133-138]
.github/workflows/skillsaw-pr-followup.yml[164-170]
Skill: skillsaw-review-panel

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The PR removes workflow `--max-turns` caps to fix truncated agent runs, but there is no regression test ensuring the caps don’t get reintroduced.

## Issue Context
Per the compliance rule, bug-fix PRs should include a regression test that would fail without the fix and pass with it. For GitHub Actions workflows, a pragmatic regression test is to assert the relevant workflow YAMLs do not contain `--max-turns` in their `claude_args` blocks.

## Fix Focus Areas
- tests/test_workflows_no_max_turns.py[1-120]
- .github/workflows/skillsaw-issue-solver.yml[133-138]
- .github/workflows/skillsaw-pr-followup.yml[164-170]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

Comment on lines 134 to 137
claude_args: |
--dangerously-skip-permissions
${{ !matrix.issue.ultracode && '--max-turns 200' || '' }}
--model ${{ matrix.issue.fable && 'claude-fable-5' || 'claude-opus-5' }}
${{ matrix.issue.ultracode && '--effort ultracode' || '' }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

2. Removed max-turns cap 🐞 Bug ☼ Reliability

The Issue Solver and PR Follow-up workflows no longer pass explicit non-ultracode --max-turns caps
(previously 200 and 100 respectively), removing a turn-budget guardrail and allowing substantially
longer Claude runs than before. Because both are scheduled and share concurrency: skillsaw-agent
with cancel-in-progress: false, unusually long runs can serialize the queue and delay other agent
workflows from starting.
Agent Prompt
## Issue description
The Issue Solver and PR Follow-up workflows removed the explicit `--max-turns` guardrail for non-ultracode runs, so the Claude steps can take much longer than they used to and potentially run unbounded.

## Issue Context
Both workflows are scheduled every 15 minutes and share a single concurrency group (`skillsaw-agent`) with `cancel-in-progress: false`, which means long-running executions serialize the pipeline and can delay subsequent scheduled agent work (including later Issue Solver runs and PR follow-ups).

## Fix Focus Areas
- .github/workflows/skillsaw-issue-solver.yml[3-11]
- .github/workflows/skillsaw-issue-solver.yml[100-138]
- .github/workflows/skillsaw-pr-followup.yml[3-11]
- .github/workflows/skillsaw-pr-followup.yml[128-170]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

@stbenjam
stbenjam merged commit 4dbd92e into stbenjam:main Jul 26, 2026
16 checks passed
@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.54%. Comparing base (5cdb537) to head (4ce023f).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #456   +/-   ##
=======================================
  Coverage   93.54%   93.54%           
=======================================
  Files         145      145           
  Lines       11418    11418           
=======================================
  Hits        10681    10681           
  Misses        737      737           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

stbenjam pushed a commit that referenced this pull request Jul 26, 2026
The review panel was hitting the 50-turn cap and failing. This follows
the same change made in #456 for the issue_solver and pr_followup
workflows.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants