Skip to content

Feature/issue triage#8

Merged
norrietaylor merged 9 commits into
mainfrom
feature/issue-triage
Mar 28, 2026
Merged

Feature/issue triage#8
norrietaylor merged 9 commits into
mainfrom
feature/issue-triage

Conversation

@norrietaylor

@norrietaylor norrietaylor commented Mar 28, 2026

Copy link
Copy Markdown
Owner

Summary by CodeRabbit

  • New Features

    • Added automated workflows for bug fixes, feature implementation, and issue planning that can create PRs, tests, and sub-issues as needed.
  • Improvements

    • Workflows now respond to issue labels (bug/feature) and handle issue descriptions with reliable fallbacks.
    • Enhanced automation for PR creation, issue commenting, labeling, and validation steps.
  • Bug Fixes

    • Improved diagnostic output when agent responses are unstructured, providing clearer comments and fallback messaging.

norrietaylor and others added 6 commits March 27, 2026 19:43
Create .github/workflows/agentry-planning-pipeline.yml triggered on
issues:opened, running the planning-pipeline workflow with the
github-actions binder. Update planning-pipeline.yaml with source/fallback
inputs, issue:comment + issue:label capabilities, and an agent block.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ping, and pr:create

Replace top-level model: block with agent: block (runtime: claude-code,
max_iterations: 3). Add source: issue.body / fallback: issue.title to
issue-description input so GitHub issue bodies are auto-resolved. Add
pr:create and issue:comment to tools.capabilities. Expand
bug-fix-system-prompt.md to instruct the agent to commit the fix with the
originating issue number, open a PR with the agent-proposed label, and
post a comment on the original issue linking to the fix PR.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Delete .github/workflows/agentry-issue-triage.yml which has been superseded by
agentry-planning-pipeline.yml. Update the planning-pipeline workflow with a
comment explaining that it replaces the triage-only workflow.

Verification confirms no remaining references to the deleted workflow in .github/
or active workflow configurations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Create .github/workflows/agentry-bug-fix.yml that triggers on issues labeled with
'category:bug' and runs the bug-fix workflow. Follows the established pattern from
agentry-code-review.yml with proper permissions and secrets configuration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add feature-implement.yaml with identity v1.0.0, claude-code agent
(max_iterations: 10, timeout: 600s, elevated trust), issue.body source
mapping with issue.title fallback, all six required tool capabilities
(repository:read, shell:execute, pr:create, issue:comment, issue:label,
issue:create), and an output schema with action enum
(implemented|decomposed), pr_url, sub_issues, and reasoning fields.

Add feature-implement-system-prompt.md instructing the agent to assess
implementability via a <=5 files / <=500 lines heuristic, implement with
tests and open a PR with agent-proposed label when in scope, or create
scoped sub-issues with category:feature + agent-decomposed labels and
comment on the parent issue when too large.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Create .github/workflows/agentry-feature-implement.yml that triggers on issues
labeled with 'category:feature' and runs the feature-implement workflow. Follows
the established pattern from agentry-bug-fix.yml with proper permissions and
secrets configuration.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Mar 28, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

Adds three GitHub Actions workflows (planning, bug-fix, feature-implement), converts several workflows to use an agent runtime (claude-code) with expanded tool capabilities (PR creation, issue comment/label), updates workflow triggers/permissions, enhances prompts for autonomous implementation, and removes many legacy proof/test artifacts and recordings.

Changes

Cohort / File(s) Summary
GitHub Actions workflows
​.github/workflows/agentry-bug-fix.yml, ​.github/workflows/agentry-feature-implement.yml, ​.github/workflows/agentry-planning-pipeline.yml
Added two workflows and modified bug-fix workflow: changed triggers to issue labeling, added job-level label conditions, expanded permissions to write contents/issues/pull-requests, standardized job steps (checkout, Python 3.12, npm/pip installs, run agentry CLI).
Workflow definitions (agent/runtime changes)
workflows/bug-fix.yaml, workflows/planning-pipeline.yaml, workflows/feature-implement.yaml
Switched from top-level model to agent (runtime: claude-code, model: claude-sonnet-4-20250514), added/updated tools.capabilities (e.g., pr:create, issue:comment, issue:label), changed inputs.issue-description to source from issue.body with fallback to issue.title, and added/adjusted max_iterations.
Prompts / Autonomous behavior
workflows/prompts/bug-fix-system-prompt.md, workflows/prompts/feature-implement-system-prompt.md
Updated bug-fix prompt to require implement→validate→commit→open PR→comment flow and confidence gating (no PR if confidence < 0.4); added feature-implement prompt describing implement vs. decompose heuristics (≤5 files & ≤500 lines), test/commit/PR/issue interactions, and required JSON output schema.
Binder / output formatting
src/agentry/binders/github_actions.py
Improved handling of agent output parsing: added _extract_json_from_text static helper to parse JSON from raw text/fenced blocks/braced blocks, normalized non-dict outputs, included diagnostics when structured output is absent, and adjusted comment formatting accordingly.
Docs / proofs removed
docs/specs/04-spec-agentry-ci/..., docs/specs/07-spec-issue-triage/..., docs/specs/triage/...
Removed numerous proof/test artifact files and proof-summary docs related to prior triage-only design (test outputs, CLI proofs, workflow validation proofs, and related spec pages).

Sequence Diagram(s)

sequenceDiagram
    autonumber
    participant User as Issue Author
    participant GH as GitHub (Issues & Actions)
    participant Action as GitHub Actions Runner
    participant Agentry as agentry Agent (claude-code)
    participant Repo as Repository (git/PR)

    User->>GH: create issue / add label (category:bug/category:feature)
    GH->>Action: trigger workflow (planning / bug-fix / feature-implement)
    Action->>Agentry: run agentry CLI (workflows/*.yaml) with inputs
    Agentry->>Repo: read repository files (repository:read)
    alt implementable
      Agentry->>Repo: create commit + push
      Agentry->>Repo: create PR (pr:create)
      Agentry->>GH: post issue comment (issue:comment) linking PR
    else decompose
      Agentry->>GH: create sub-issues (issue:create) and label parent (issue:label)
      Agentry->>GH: comment on parent issue
    end
    Agentry->>Action: return JSON result (action/pr_url/sub_issues/reasoning)
    Action->>GH: surface results (comments/labels/PR)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Poem

🐰 I hopped through issues, labels in my paws,

I nudged the agent, set its careful laws.
Commit, then PR, I left the issue a note,
Tests ran a jig while CI kept afloat—
A tiny rabbit cheer for an automated coat.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Title check ❓ Inconclusive The title 'Feature/issue triage' is vague and generic, using non-descriptive terms that don't clearly convey the main changes to someone scanning the repository history. Provide a more specific, descriptive title that summarizes the primary change, such as 'Add automated issue triage and feature implementation workflows' or 'Implement multi-stage automation pipeline for issue management'.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/issue-triage

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

Copy link
Copy Markdown

Agentry Code Review

Looking at this git diff, I'll analyze the changes for quality, correctness, security, and performance issues. This appears to be a significant refactoring of GitHub Actions workflows and related configuration files for an AI-powered development automation system.

```json
{
  "findings": [
    {
      "file": ".github/workflows/agentry-bug-fix.yml",
      "line": 26,
      "severity": "warning",
      "category": "correctness",
      "description": "The conditional check `github.event.label.name == 'category:bug'` could fail if github.event.label is null or undefined, causing workflow errors when triggered by label removal or other edge cases.",
      "suggestion": "Add a null check: `github.event.label && github.event.label.name == 'category:bug'` or use `contains(github.event.label.name, 'category:bug')`"
    },
    {
      "file": ".github/workflows/agentry-feature-implement.yml",
      "line": 26,
      "severity": "warning",
      "category": "correctness",
      "description": "Same issue as bug-fix workflow - conditional check could fail if github.event.label is null or undefined.",
      "suggestion": "Add a null check: `github.event.label && github.event.label.name == 'category:feature'` or use `contains(github.event.label.name, 'category:feature')`"
    },
    {
      "file": ".github/workflows/agentry-bug-fix.yml",
      "line": 20,
      "severity": "warning",
      "category": "security",
      "description": "The workflow grants 'contents: write' and 'pull-requests: write' permissions, which allows the action to modify repository contents and create PRs. This increases the attack surface if the workflow is compromised.",
      "suggestion": "Consider using more granular permissions or implementing additional validation checks before granting write access. Document the security rationale for these elevated permissions."
    },
    {
      "file": ".github/workflows/agentry-bug-fix.yml",
      "line": 42,
      "severity": "info",
      "category": "performance",
      "description": "No timeout specified for the agentry command, which could lead to runaway processes consuming GitHub Actions minutes.",
      "suggestion": "Add a timeout to the step, similar to the 600s timeout used in the feature-implement workflow: `timeout-minutes: 10`"
    },
    {
      "file": "workflows/bug-fix.yaml",
      "line": 27,
      "severity": "info",
      "category": "style",
      "description": "The model name 'claude-sonnet-4-20250514' appears to be hardcoded with a specific date, which may become outdated and could cause issues if the model is deprecated.",
      "suggestion": "Consider using a more generic model identifier or making this configurable through environment variables or workflow inputs."
    },
    {
      "file": ".github/workflows/agentry-feature-implement.yml",
      "line": 17,
      "severity": "info",
      "category": "security",
      "description": "The workflow can be triggered by anyone who can apply labels to issues, po

Tokens: 25,427 in / 2,090 out

@github-actions

Copy link
Copy Markdown

Agentry Code Review

I'll analyze this git diff for code quality, security, and other issues. Let me examine the changes systematically.

```json
{
  "findings": [
    {
      "file": "docs/specs/",
      "line": 1,
      "severity": "info",
      "category": "style",
      "description": "Large number of documentation/proof files (40+ files) are being added to the repository, which may be test artifacts rather than production documentation.",
      "suggestion": "Consider whether these proof files should be kept in the repository long-term, or if they could be generated as part of a testing process and excluded from version control."
    },
    {
      "file": "workflows/bug-fix.yaml",
      "line": 27,
      "severity": "info",
      "category": "style",
      "description": "Hardcoded model name 'claude-sonnet-4-20250514' in agent configuration may become outdated over time.",
      "suggestion": "Consider making the model name configurable through environment variables or a central configuration file for easier updates."
    },
    {
      "file": ".github/workflows/agentry-feature-implement.yml",
      "line": 34,
      "severity": "info",
      "category": "style",
      "description": "Python version '3.12' is hardcoded in multiple workflow files.",
      "suggestion": "Consider defining the Python version in a central location or using a matrix strategy if multiple Python versions need to be supported."
    },
    {
      "file": "workflows/prompts/feature-implement-system-prompt.md",
      "line": 14,
      "severity": "info",
      "category": "correctness",
      "description": "The heuristics for determining implementability (5 files, 500 lines) are hardcoded and may not be appropriate for all codebases or feature types.",
      "suggestion": "Consider making these thresholds configurable or adding guidance on when to adjust them based on project context."
    }
  ],
  "summary": "The diff shows a well-structured refactoring that replaces a simple triage workflow with a more comprehensive planning pipeline and adds automated bug-fix and feature-implementation workflows. The code quality is generally good with proper security practices, appropriate permissions, and consistent patterns. The main considerations are around the large number of documentation files being added and some hardcoded configuration values.",
  "confidence": 0.85
}

---
*Tokens: 25,427 in / 2,906 out*

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🧹 Nitpick comments (5)
docs/specs/ci-workflows/04-proofs/T04-03-no-remaining-references.txt (1)

5-6: Make the proof command assertive instead of informational.

The current command does not fail on unexpected matches, so it’s weak as a verification artifact.

Suggested change
-Command: grep -r agentry-issue-triage .github/ || echo "No references found"
+Command: |
+  matches=$(grep -R "agentry-issue-triage" .github/ \
+    | grep -v '^.github/workflows/agentry-planning-pipeline.yml:# This workflow replaces the superseded agentry-issue-triage.yml workflow\.$' || true)
+  test -z "$matches" && echo "No unexpected references found" || { echo "$matches"; exit 1; }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/specs/ci-workflows/04-proofs/T04-03-no-remaining-references.txt` around
lines 5 - 6, Replace the informational grep command so it fails the CI on
unexpected matches: instead of "grep -r agentry-issue-triage .github/ || echo
'No references found'", run a grep that excludes planning-pipeline.yml (or
explicitly ignores its explanatory comment) and returns non‑zero on any match
(e.g., use grep -r --exclude=planning-pipeline.yml agentry-issue-triage .github/
and if it finds matches print an error and exit 1); update the command string in
the proof (the grep invocation referencing "agentry-issue-triage" and the
mention of planning-pipeline.yml) so CI fails when stray references are present.
.github/workflows/agentry-planning-pipeline.yml (1)

27-29: Pin the Claude Code npm package version to stabilize CI.

Installing @anthropic-ai/claude-code without a version can cause unexpected workflow breakages after upstream releases. The latest stable version is 2.1.86.

Suggested change
-      - name: Install Claude Code
-        run: npm install -g `@anthropic-ai/claude-code`
+      - name: Install Claude Code
+        run: npm install -g `@anthropic-ai/claude-code`@2.1.86
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/agentry-planning-pipeline.yml around lines 27 - 29, Pin
the global installation of the Claude Code package by updating the workflow step
named "Install Claude Code" to install a fixed version of
`@anthropic-ai/claude-code` (use 2.1.86); replace the current npm install -g
`@anthropic-ai/claude-code` invocation with an explicit version specifier `@2.1.86`
to stabilize CI.
.github/workflows/agentry-feature-implement.yml (1)

27-27: Add a job timeout to prevent stuck runner consumption.

At Line 27, consider adding timeout-minutes for the agentry job. This improves CI reliability and cost control when external calls hang.

Proposed fix
   agentry:
     if: github.event.label.name == 'category:feature'
     runs-on: ubuntu-latest
+    timeout-minutes: 30
     steps:
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/agentry-feature-implement.yml at line 27, Add a job-level
timeout to the agentry GitHub Actions job to prevent stuck runners: in the job
block with id "agentry" (the block containing the runs-on: ubuntu-latest line),
add a top-level key like "timeout-minutes: 30" (or an appropriate number for
your workflow) so the job is automatically canceled after that duration.
.github/workflows/agentry-bug-fix.yml (1)

25-27: Add a job-level timeout to prevent hung runs.

Line 25 through Line 27 define a long-running agent job with no timeout-minutes. Add an explicit limit for reliability and cost control.

Proposed workflow update
 jobs:
   agentry:
     if: github.event.label.name == 'category:bug'
     runs-on: ubuntu-latest
+    timeout-minutes: 15
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/agentry-bug-fix.yml around lines 25 - 27, The "agentry"
job currently has no timeout, which can lead to hung runs and extra cost; add a
job-level timeout by inserting a "timeout-minutes: 30" (or other agreed limit)
under the agentry job block (alongside "if:" and "runs-on:") so the workflow
will be automatically terminated after the specified number of minutes.
workflows/prompts/bug-fix-system-prompt.md (1)

25-26: Return PR metadata in output for machine traceability.

Line 25 and Line 26 define output keys that omit pr_url, even though the flow requires opening a PR. Adding PR metadata improves downstream automation and auditing.

Proposed output contract tweak
-Output format: JSON object with keys `diagnosis`, `root_cause`, `suggested_fix`, and `confidence`.
+Output format: JSON object with keys `diagnosis`, `root_cause`, `suggested_fix`, `confidence`, and `pr_url` (when a PR is opened).
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@workflows/prompts/bug-fix-system-prompt.md` around lines 25 - 26, Update the
output contract string "Output format: JSON object with keys `diagnosis`,
`root_cause`, `suggested_fix`, and `confidence`." to include a `pr_url` field,
e.g. add `pr_url` to the top-level keys so downstream automation can trace the
PR; keep the existing `suggested_fix` requirement (must contain `file`, `line`,
and `change`) and ensure the prompt text clearly mandates `pr_url` be a fully
qualified URL in the produced JSON.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/specs/ci-workflows/03-proofs/T03-03-workflow-structure.txt`:
- Around line 18-20: The current boolean checks (steps_checkout, steps_python,
steps_agentry) use substring matching on str(step) which yields false positives;
update the checks to iterate the agentry job's steps and explicitly inspect
step.get('uses') and step.get('run') fields: for steps_checkout check if
step.get('uses', '').startswith('actions/checkout@'), for steps_python check if
step.get('uses', '').startswith('actions/setup-python@'), and for steps_agentry
check for a precise match in step.get('uses', '') or a recognizable command in
step.get('run', '') (e.g., the agentry action identifier or exact CLI
invocation); replace the any(...) expressions with these explicit field checks
so unrelated/malformed steps no longer pass.

In `@docs/specs/ci-workflows/03-proofs/T03-feature-proofs.md`:
- Around line 29-33: Add a language tag to the fenced code block containing the
agentry CLI command so the block is fenced as bash; locate the block showing
"agentry --output-format json run workflows/feature-implement.yaml ..." and
change the opening fence from "```" to "```bash" (the code block around that
command is what triggers MD040).

In `@docs/specs/ci-workflows/04-proofs/T04-proofs.md`:
- Line 30: The summary currently uses the shorthand "issues: [opened]" which is
not valid GitHub Actions trigger syntax; update the summary to the proper
workflow notation used in the file (e.g., use "on: issues: types: [opened]" or
the exact trigger form present in the workflow header) so the documentation
matches the actual workflow trigger (reference the trigger string "issues:
[opened]" and replace it with the correct "on: issues: types: [opened]" form).

In `@workflows/feature-implement.yaml`:
- Around line 41-59: The schema currently lists action, pr_url, and sub_issues
but does not enforce that pr_url is required when action is "implemented" or
that sub_issues is required when action is "decomposed"; update the schema by
adding a JSON Schema conditional (if/then/else) or oneOf block referencing the
existing properties (action, pr_url, sub_issues, reasoning) so that if:
{properties:{action:{const:"implemented"}}} then: {required:["pr_url"]} and if:
{properties:{action:{const:"decomposed"}}} then: {required:["sub_issues"]}; keep
the existing types and enums and ensure sub_issues remains an array of strings.

In `@workflows/prompts/bug-fix-system-prompt.md`:
- Around line 9-15: Insert a mandatory verification step between "Implement the
fix" and "Commit the fix" that requires running relevant automated tests and
checks (unit/integration tests, linters, CI script) and confirming they pass
before proceeding, and update the subsequent "Commit the fix" / "Open a pull
request" instructions to state commits/PRs must only be created after this
verification is successful; modify the items named "Implement the fix", "Commit
the fix", "Open a pull request", and "Post a comment" accordingly so the flow
enforces "run and pass relevant tests/checks first" before committing or opening
the PR.

In `@workflows/prompts/feature-implement-system-prompt.md`:
- Around line 8-9: Step 2 in
workflows/prompts/feature-implement-system-prompt.md incorrectly assumes
planning-pipeline comments will exist; update the prompt text to first check for
planning-pipeline issue comments and, if none are found, fall back to using the
pipeline file outputs described in workflows/planning-pipeline.yaml and
workflows/task-decompose.yaml (or to trigger/consume the task-decompose output)
as the implementation plan; mention both the comment check and the deterministic
fallback so the prompt does not depend solely on issue-comment publication.

---

Nitpick comments:
In @.github/workflows/agentry-bug-fix.yml:
- Around line 25-27: The "agentry" job currently has no timeout, which can lead
to hung runs and extra cost; add a job-level timeout by inserting a
"timeout-minutes: 30" (or other agreed limit) under the agentry job block
(alongside "if:" and "runs-on:") so the workflow will be automatically
terminated after the specified number of minutes.

In @.github/workflows/agentry-feature-implement.yml:
- Line 27: Add a job-level timeout to the agentry GitHub Actions job to prevent
stuck runners: in the job block with id "agentry" (the block containing the
runs-on: ubuntu-latest line), add a top-level key like "timeout-minutes: 30" (or
an appropriate number for your workflow) so the job is automatically canceled
after that duration.

In @.github/workflows/agentry-planning-pipeline.yml:
- Around line 27-29: Pin the global installation of the Claude Code package by
updating the workflow step named "Install Claude Code" to install a fixed
version of `@anthropic-ai/claude-code` (use 2.1.86); replace the current npm
install -g `@anthropic-ai/claude-code` invocation with an explicit version
specifier `@2.1.86` to stabilize CI.

In `@docs/specs/ci-workflows/04-proofs/T04-03-no-remaining-references.txt`:
- Around line 5-6: Replace the informational grep command so it fails the CI on
unexpected matches: instead of "grep -r agentry-issue-triage .github/ || echo
'No references found'", run a grep that excludes planning-pipeline.yml (or
explicitly ignores its explanatory comment) and returns non‑zero on any match
(e.g., use grep -r --exclude=planning-pipeline.yml agentry-issue-triage .github/
and if it finds matches print an error and exit 1); update the command string in
the proof (the grep invocation referencing "agentry-issue-triage" and the
mention of planning-pipeline.yml) so CI fails when stray references are present.

In `@workflows/prompts/bug-fix-system-prompt.md`:
- Around line 25-26: Update the output contract string "Output format: JSON
object with keys `diagnosis`, `root_cause`, `suggested_fix`, and `confidence`."
to include a `pr_url` field, e.g. add `pr_url` to the top-level keys so
downstream automation can trace the PR; keep the existing `suggested_fix`
requirement (must contain `file`, `line`, and `change`) and ensure the prompt
text clearly mandates `pr_url` be a fully qualified URL in the produced JSON.
🪄 Autofix (Beta)

✅ Autofix completed


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 06e663b7-e253-4fdd-8e1f-923c35c336c1

📥 Commits

Reviewing files that changed from the base of the PR and between 324e08a and c4acafd.

📒 Files selected for processing (31)
  • .github/workflows/agentry-bug-fix.yml
  • .github/workflows/agentry-feature-implement.yml
  • .github/workflows/agentry-planning-pipeline.yml
  • docs/specs/07-spec-issue-triage/02-proofs/T02-01-file.txt
  • docs/specs/07-spec-issue-triage/02-proofs/T02-02-file.txt
  • docs/specs/07-spec-issue-triage/02-proofs/T02-03-file.txt
  • docs/specs/07-spec-issue-triage/02-proofs/T02-04-file.txt
  • docs/specs/07-spec-issue-triage/02-proofs/T02-proofs.md
  • docs/specs/07-spec-issue-triggered-triage/02-proofs/T02-01-file.txt
  • docs/specs/07-spec-issue-triggered-triage/02-proofs/T02-02-file.txt
  • docs/specs/07-spec-issue-triggered-triage/02-proofs/T02-proofs.md
  • docs/specs/ci-workflows/03-proofs/T03-01-file-creation.txt
  • docs/specs/ci-workflows/03-proofs/T03-02-yaml-validation.txt
  • docs/specs/ci-workflows/03-proofs/T03-03-workflow-structure.txt
  • docs/specs/ci-workflows/03-proofs/T03-feature-01-file-creation.txt
  • docs/specs/ci-workflows/03-proofs/T03-feature-02-yaml-validation.txt
  • docs/specs/ci-workflows/03-proofs/T03-feature-03-workflow-structure.txt
  • docs/specs/ci-workflows/03-proofs/T03-feature-proofs.md
  • docs/specs/ci-workflows/03-proofs/T03-proofs.md
  • docs/specs/ci-workflows/04-proofs/T04-01-file-deletion.txt
  • docs/specs/ci-workflows/04-proofs/T04-02-workflow-updated.txt
  • docs/specs/ci-workflows/04-proofs/T04-03-no-remaining-references.txt
  • docs/specs/ci-workflows/04-proofs/T04-proofs.md
  • docs/specs/triage/01-proofs/T01-01-file.txt
  • docs/specs/triage/01-proofs/T01-02-file.txt
  • docs/specs/triage/01-proofs/T01-proofs.md
  • workflows/bug-fix.yaml
  • workflows/feature-implement.yaml
  • workflows/planning-pipeline.yaml
  • workflows/prompts/bug-fix-system-prompt.md
  • workflows/prompts/feature-implement-system-prompt.md

Comment on lines +18 to +20
'steps_checkout': any('checkout' in str(s) for s in workflow.get('jobs', {}).get('agentry', {}).get('steps', [])),
'steps_python': any('setup-python' in str(s) for s in workflow.get('jobs', {}).get('agentry', {}).get('steps', [])),
'steps_agentry': any('agentry' in str(s) for s in workflow.get('jobs', {}).get('agentry', {}).get('steps', [])),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Harden step detection to avoid false-positive PASS results.

At Line 18–Line 20, using substring checks on str(step) is too permissive. A malformed or unrelated step can still pass. Prefer explicit matching on uses/run fields.

Proposed fix
-    'steps_checkout': any('checkout' in str(s) for s in workflow.get('jobs', {}).get('agentry', {}).get('steps', [])),
-    'steps_python': any('setup-python' in str(s) for s in workflow.get('jobs', {}).get('agentry', {}).get('steps', [])),
-    'steps_agentry': any('agentry' in str(s) for s in workflow.get('jobs', {}).get('agentry', {}).get('steps', [])),
+    'steps_checkout': any(
+        s.get('uses', '').startswith('actions/checkout@')
+        for s in workflow.get('jobs', {}).get('agentry', {}).get('steps', [])
+        if isinstance(s, dict)
+    ),
+    'steps_python': any(
+        s.get('uses', '').startswith('actions/setup-python@')
+        for s in workflow.get('jobs', {}).get('agentry', {}).get('steps', [])
+        if isinstance(s, dict)
+    ),
+    'steps_agentry': any(
+        'agentry --output-format json run workflows/bug-fix.yaml' in s.get('run', '')
+        for s in workflow.get('jobs', {}).get('agentry', {}).get('steps', [])
+        if isinstance(s, dict)
+    ),
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/specs/ci-workflows/03-proofs/T03-03-workflow-structure.txt` around lines
18 - 20, The current boolean checks (steps_checkout, steps_python,
steps_agentry) use substring matching on str(step) which yields false positives;
update the checks to iterate the agentry job's steps and explicitly inspect
step.get('uses') and step.get('run') fields: for steps_checkout check if
step.get('uses', '').startswith('actions/checkout@'), for steps_python check if
step.get('uses', '').startswith('actions/setup-python@'), and for steps_agentry
check for a precise match in step.get('uses', '') or a recognizable command in
step.get('run', '') (e.g., the agentry action identifier or exact CLI
invocation); replace the any(...) expressions with these explicit field checks
so unrelated/malformed steps no longer pass.

Comment on lines +29 to +33
```
agentry --output-format json run workflows/feature-implement.yaml \
--input repository-ref=. \
--binder github-actions
```

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Add a language tag to the fenced code block.

At Line 29, the code fence is missing a language identifier, which triggers MD040 and weakens markdown tooling support.

Proposed fix
-```
+```bash
 agentry --output-format json run workflows/feature-implement.yaml \
   --input repository-ref=. \
   --binder github-actions
</details>

<details>
<summary>🧰 Tools</summary>

<details>
<summary>🪛 markdownlint-cli2 (0.22.0)</summary>

[warning] 29-29: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

</details>

</details>

<details>
<summary>🤖 Prompt for AI Agents</summary>

Verify each finding against the current code and only fix it if needed.

In @docs/specs/ci-workflows/03-proofs/T03-feature-proofs.md around lines 29 -
33, Add a language tag to the fenced code block containing the agentry CLI
command so the block is fenced as bash; locate the block showing "agentry
--output-format json run workflows/feature-implement.yaml ..." and change the
opening fence from "" to "bash" (the code block around that command is
what triggers MD040).


</details>

<!-- fingerprinting:phantom:triton:hawk:8549180a-b638-40f6-b477-2ce640dd19d7 -->

<!-- This is an auto-generated comment by CodeRabbit -->

## Implementation Details

The planning pipeline workflow (agentry-planning-pipeline.yml) is the replacement for the triage-only workflow. It:
- Triggers on the same event: `issues: [opened]`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Use accurate GitHub Actions trigger notation in the summary.

issues: [opened] is not the workflow syntax used in the actual file and may mislead readers.

Suggested change
-- Triggers on the same event: `issues: [opened]`
+- Triggers on the same event: `on.issues.types: [opened]`
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/specs/ci-workflows/04-proofs/T04-proofs.md` at line 30, The summary
currently uses the shorthand "issues: [opened]" which is not valid GitHub
Actions trigger syntax; update the summary to the proper workflow notation used
in the file (e.g., use "on: issues: types: [opened]" or the exact trigger form
present in the workflow header) so the documentation matches the actual workflow
trigger (reference the trigger string "issues: [opened]" and replace it with the
correct "on: issues: types: [opened]" form).

Comment thread workflows/feature-implement.yaml
Comment thread workflows/prompts/bug-fix-system-prompt.md Outdated
Comment on lines +8 to +9
2. Check for any comments on the issue from the planning-pipeline, which may have already decomposed the feature into a task list. Use those tasks as your implementation plan if present.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Prompt depends on planning comments that may never exist.

Line 8 assumes planning-pipeline decomposition is available in issue comments, but current pipeline contracts (see workflows/planning-pipeline.yaml and workflows/task-decompose.yaml) describe file outputs, not guaranteed issue-comment publication. This can make Step 1 nondeterministic.

Proposed prompt fallback clarification
-2. Check for any comments on the issue from the planning-pipeline, which may have already decomposed the feature into a task list. Use those tasks as your implementation plan if present.
+2. Check for planning-pipeline decomposition in issue comments. If no decomposition comment exists, proceed from the issue body alone and explicitly note that absence in `reasoning`.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
2. Check for any comments on the issue from the planning-pipeline, which may have already decomposed the feature into a task list. Use those tasks as your implementation plan if present.
2. Check for planning-pipeline decomposition in issue comments. If no decomposition comment exists, proceed from the issue body alone and explicitly note that absence in `reasoning`.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@workflows/prompts/feature-implement-system-prompt.md` around lines 8 - 9,
Step 2 in workflows/prompts/feature-implement-system-prompt.md incorrectly
assumes planning-pipeline comments will exist; update the prompt text to first
check for planning-pipeline issue comments and, if none are found, fall back to
using the pipeline file outputs described in workflows/planning-pipeline.yaml
and workflows/task-decompose.yaml (or to trigger/consume the task-decompose
output) as the implementation plan; mention both the comment check and the
deterministic fallback so the prompt does not depend solely on issue-comment
publication.

- Harden feature-implement output schema with allOf conditional
  (pr_url required when implemented, sub_issues when decomposed)
- Add validation step to bug-fix prompt before commit/PR creation
- Feature-implement prompt: graceful fallback when no planning comments exist
- Remove 41 proof artifact files from git tracking (already gitignored)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Agentry Code Review


Tokens: 31,236 in / 2,338 out

The _format_output_comment method produced empty comments when the agent
returned no structured output or a raw string response. Now:
- Extracts JSON from markdown-fenced or preamble-wrapped text
- Shows raw agent response when no structured JSON found
- Displays diagnostic message when output is completely empty

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Agentry Code Review

Looking at this git diff, I can see significant changes to GitHub Actions workflows, workflow configurations, and code for handling agent outputs. Let me analyze this for potential issues.

```json
{
  "findings": [
    {
      "file": ".github/workflows/agentry-bug-fix.yml",
      "line": 22,
      "severity": "critical",
      "category": "security",
      "description": "Workflow now has 'contents: write' and 'pull-requests: write' permissions and can be triggered by any user who can apply labels to issues. This creates a potential security risk where malicious users could trigger automated code changes.",
      "suggestion": "Add additional validation in the workflow condition to ensure only trusted users can trigger these workflows, or require manual approval for automated PR creation."
    },
    {
      "file": ".github/workflows/agentry-bug-fix.yml",
      "line": 26,
      "severity": "warning",
      "category": "correctness",
      "description": "The conditional trigger relies on exact string matching of label name 'category:bug' without any fallback or validation. Typos in labels will cause the workflow to silently fail to trigger.",
      "suggestion": "Consider using a more robust label matching strategy, such as checking for labels that start with 'category:' or provide fallback behavior."
    },
    {
      "file": ".github/workflows/agentry-planning-pipeline.yml",
      "line": 8,
      "severity": "warning",
      "category": "correctness",
      "description": "The planning pipeline now handles 'issues: [opened]' events, but the original triage workflow that handled this was renamed to bug-fix (which only triggers on labeled events). This could leave newly opened issues without any automated processing.",
      "suggestion": "Verify that the planning pipeline workflow properly handles the responsibility previously covered by the triage workflow, or ensure proper workflow orchestration."
    },
    {
      "file": "src/agentry/binders/github_actions.py",
      "line": 1041,
      "severity": "warning",
      "category": "performance",
      "description": "The _extract_json_from_text method uses text.rfind('}') to find the last closing brace, which could match unrelated text in large inputs and doesn't validate proper JSON structure.",
      "suggestion": "Implement proper brace counting or use a more robust JSON extraction algorithm that validates structure, especially for large text inputs."
    },
    {
      "file": "src/agentry/binders/github_actions.py",
      "line": 994,
      "severity": "info",
      "category": "correctness",
      "description": "When agent_output is a string, the code tries to extract JSON but falls back to displaying raw text. However, it truncates to 3000 characters which might cut off important information.",
      "suggestion": "Consider using a smarter truncation strategy that preserves JSON structure, or provide a way to access the full output when truncated."
    },
    {
      "file": "work

Tokens: 32,482 in / 1,930 out

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
src/agentry/binders/github_actions.py (2)

1029-1036: The fence regex could be slightly more permissive.

The regex r"```(?:json)?\s*\n(.*?)```" requires a newline after the opening fence. Some LLM outputs may omit the newline (e.g., ```json{"key": "value"}```). However, the subsequent brace-matching fallback (lines 1038-1049) handles this case, so this is a minor observation rather than an issue.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/agentry/binders/github_actions.py` around lines 1029 - 1036, Relax the
fence regex used in the re.search call that assigns fence_match so it also
matches cases where there is no newline after the opening ```json fence
(currently it requires a newline). Update the pattern in the re.search (the
expression that produces fence_match) to allow optional whitespace/newline after
the opening fence and still capture the inner block (keep re.DOTALL and
non-greedy capture), so `fence_match` will succeed for both ```json\n{...}```
and ```json{...}``` outputs; leave the existing brace-matching fallback
untouched.

1007-1051: Consider moving import re to module level.

The re import at line 1018 is inside the method. While functionally correct, placing it at the module level (around line 10-15) would be more consistent with the codebase style and avoid re-importing on each call.

♻️ Suggested change

At the top of the file (around line 14):

 from pathlib import Path
 from typing import Any, cast
+import re
 
 import httpx

Then remove the import from inside the method:

     def _extract_json_from_text(text: str) -> dict[str, Any] | None:
         ...
-        import re
-
         # Try direct parse first.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/agentry/binders/github_actions.py` around lines 1007 - 1051, Move the
inline import for the regex module out of the _extract_json_from_text method and
place a single top-level "import re" with the other module imports; then remove
the "import re" line from inside the _extract_json_from_text function so the
function uses the module-level re symbol instead of re-importing on each call.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@src/agentry/binders/github_actions.py`:
- Around line 1029-1036: Relax the fence regex used in the re.search call that
assigns fence_match so it also matches cases where there is no newline after the
opening ```json fence (currently it requires a newline). Update the pattern in
the re.search (the expression that produces fence_match) to allow optional
whitespace/newline after the opening fence and still capture the inner block
(keep re.DOTALL and non-greedy capture), so `fence_match` will succeed for both
```json\n{...}``` and ```json{...}``` outputs; leave the existing brace-matching
fallback untouched.
- Around line 1007-1051: Move the inline import for the regex module out of the
_extract_json_from_text method and place a single top-level "import re" with the
other module imports; then remove the "import re" line from inside the
_extract_json_from_text function so the function uses the module-level re symbol
instead of re-importing on each call.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: c58862e9-6872-4d13-b4a7-40d0339801d1

📥 Commits

Reviewing files that changed from the base of the PR and between 37f58c2 and 63f74a3.

📒 Files selected for processing (1)
  • src/agentry/binders/github_actions.py

@coderabbitai

coderabbitai Bot commented Mar 28, 2026

Copy link
Copy Markdown

Note

Autofix is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it.

Fixes Applied Successfully

Fixed 1 file(s) based on 3 unresolved review comments.

A stacked PR containing fixes has been created.

  • Stacked PR: #9
  • Files modified:
  • workflows/prompts/feature-implement-system-prompt.md

Time taken: 6m 23s

@norrietaylor norrietaylor merged commit 177a910 into main Mar 28, 2026
7 checks passed
@norrietaylor norrietaylor deleted the feature/issue-triage branch March 28, 2026 04:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant