Conversation
…ap step When a Feature contains requirements that cannot be planned due to missing inputs, the skill now explicitly documents them in an "Excluded requirements" section of the impact map instead of silently omitting them. Implements TC-5056 Assisted-by: Claude Code
Reviewer's guide (collapsed on small PRs)Reviewer's GuideAdds explicit guidance in the plan-feature skill to flag non-plannable requirements and document them under an 'Excluded requirements' section in the impact map, ensuring missing-input requirements are surfaced rather than silently omitted. Flow diagram for handling non-plannable requirements in Step 4 impact mapflowchart TD
A[Feature requirements] --> B[Evaluate requirement]
B --> C{Can be decomposed into actionable tasks?}
C -->|Yes| D[Add change to impact map]
C -->|No| E[Add to Excluded requirements section]
E --> F[State missing inputs preventing planning]
D --> G[Present full impact map to user]
F --> G
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- The new guidance paragraph in Step 4 is quite dense; consider breaking it into a short introductory sentence followed by bullet points (e.g., when to flag, how to document, examples of missing inputs) to improve scanability for users following the workflow.
- You repeat the idea of not silently omitting requirements in a couple of phrases; tightening this wording into a single, clear directive (e.g., one sentence about explicit exclusion and rationale) would make the instruction more concise without losing emphasis.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The new guidance paragraph in Step 4 is quite dense; consider breaking it into a short introductory sentence followed by bullet points (e.g., when to flag, how to document, examples of missing inputs) to improve scanability for users following the workflow.
- You repeat the idea of not silently omitting requirements in a couple of phrases; tightening this wording into a single, clear directive (e.g., one sentence about explicit exclusion and rationale) would make the instruction more concise without losing emphasis.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Eval Results
Eval Results: plan-feature
| Eval | Passed | Failed | Pass Rate |
|---|---|---|---|
| eval-1 | 18/19 | 1 | 95% |
| eval-2 | 15/16 | 1 | 94% |
| eval-3 | 14/15 | 1 | 93% |
| eval-4 | 13/13 | 0 | 100% |
| eval-5 | 13/15 | 2 | 87% |
| eval-6 | 13/14 | 1 | 93% |
Failed Assertions
eval-1: 1 failing assertion
- Assertion: "For each task created, a description digest is produced — evidenced by a separate digest file (e.g., task-N-digest.md), a digest entry in the impact map, or a '[sdlc-workflow] Description digest: sha256-md:<64-char-hex>' marker in any output file. The digest must contain a format-tagged SHA-256 hash — exactly 64 lowercase hex characters prefixed by 'sha256-md:' or 'sha256-adf:', not a placeholder, abbreviated value, or example string"
Evidence: "Digest files exist for all 7 tasks (task-1-digest.md through task-7-digest.md). However, all contain the literal placeholder text '[sdlc-workflow] Description digest: sha256-adf:<64-char-hex-digest>' rather than an actual 64-character hexadecimal hash value. The string '<64-char-hex-digest>' is a placeholder/example string, not a valid SHA-256 hash. Each digest file explicitly states: 'Since this eval does not interact with Jira, the digest cannot be computed.' The assertion requires 'exactly 64 lowercase hex characters prefixed by sha256-md: or sha256-adf:, not a placeholder, abbreviated value, or example string.'"
eval-2: 1 failing assertion
- Assertion: "For each task created, a description digest is produced — evidenced by a separate digest file (e.g., task-N-digest.md), a digest entry in the impact map, or a '[sdlc-workflow] Description digest: sha256-md:<64-char-hex>' marker in any output file. The digest must contain a format-tagged SHA-256 hash — exactly 64 lowercase hex characters prefixed by 'sha256-md:' or 'sha256-adf:', not a placeholder, abbreviated value, or example string"
Evidence: "Digest files exist for all three tasks (task-1-digest.md, task-2-digest.md, task-3-digest.md), but all contain the placeholder string 'sha256-adf:<computed-64-char-hex-digest>' instead of an actual 64-character hex digest. For example, task-1-digest.md line 13: '[sdlc-workflow] Description digest: sha256-adf:<computed-64-char-hex-digest>'. The assertion requires 'exactly 64 lowercase hex characters prefixed by sha256-md: or sha256-adf:, not a placeholder, abbreviated value, or example string.' The angle-bracket placeholder fails this requirement."
eval-3: 1 failing assertion
- Assertion: "For each task created, a description digest is produced — evidenced by a separate digest file (e.g., task-N-digest.md), a digest entry in the impact map, or a '[sdlc-workflow] Description digest: sha256-md:<64-char-hex>' marker in any output file. The digest must contain a format-tagged SHA-256 hash — exactly 64 lowercase hex characters prefixed by 'sha256-md:' or 'sha256-adf:', not a placeholder, abbreviated value, or example string"
Evidence: "All 8 digest files (task-1-digest.md through task-8-digest.md) exist but contain placeholder hashes instead of actual 64-character hex strings. For example, task-1-digest.md contains: '[sdlc-workflow] Description digest: sha256-md:<computed-after-jira-creation>' with a note explaining 'In a live run, this digest would be computed by...' The string '<computed-after-jira-creation>' is not a valid 64-character lowercase hex hash. The assertion requires 'exactly 64 lowercase hex characters prefixed by sha256-md:', not a placeholder."
eval-5: 2 failing assertions
-
Assertion: "Each non-documentation task file contains all required template sections: Repository, Target Branch, Description, at least one of Files to Modify or Files to Create, Implementation Notes, Acceptance Criteria, Test Requirements. Documentation tasks are exempt from requiring Files to Modify, Files to Create, and Implementation Notes — they must still include Repository, Target Branch, Description, Acceptance Criteria, and Test Requirements"
Evidence: "Bookend tasks (Task 1 and Task 8) are non-documentation tasks but lack required sections. Task 1 (task-1-create-feature-branch.md) is missing 'Files to Modify', 'Files to Create', and 'Implementation Notes' sections. Task 8 (task-8-merge-feature-branch.md) is similarly missing 'Files to Modify', 'Files to Create', and 'Implementation Notes' sections. The assertion exempts only documentation tasks, not bookend tasks. Non-bookend intermediate tasks (2-6) all have the required sections. Task 7 (documentation) correctly includes Repository, Target Branch, Description, Acceptance Criteria, and Test Requirements." -
Assertion: "For each task created, a description digest is produced — evidenced by a separate digest file (e.g., task-N-digest.md), a digest entry in the impact map, or a '[sdlc-workflow] Description digest: sha256-md:<64-char-hex>' marker in any output file. The digest must contain a format-tagged SHA-256 hash — exactly 64 lowercase hex characters prefixed by 'sha256-md:' or 'sha256-adf:', not a placeholder, abbreviated value, or example string"
Evidence: "Digest files exist for all 8 tasks (task-1-digest.md through task-8-digest.md). However, every digest file contains a placeholder instead of an actual SHA-256 hash. For example, task-1-digest.md line 9: '[sdlc-workflow] Description digest: sha256-adf:<64-char-hex-digest>' — the string '<64-char-hex-digest>' is a placeholder, not 64 lowercase hex characters. The assertion explicitly requires 'not a placeholder, abbreviated value, or example string'. No actual computed hash appears in any output file."
eval-6: 1 failing assertion
- Assertion: "For each task created, a description digest is produced — evidenced by a separate digest file (e.g., task-N-digest.md), a digest entry in the impact map, or a '[sdlc-workflow] Description digest: sha256-md:<64-char-hex>' marker in any output file. The digest must contain a format-tagged SHA-256 hash — exactly 64 lowercase hex characters prefixed by 'sha256-md:' or 'sha256-adf:', not a placeholder, abbreviated value, or example string"
Evidence: "All 9 digest files (task-1-digest.md through task-9-digest.md) exist, but each contains a placeholder: '[sdlc-workflow] Description digest: sha256-md:<computed-after-jira-creation>' followed by a note explaining the digest would be computed in a live execution. The value '<computed-after-jira-creation>' is not 64 lowercase hex characters — it is a placeholder string. The assertion explicitly requires 'not a placeholder, abbreviated value, or example string'."
Pass rate: 94% · Tokens: 84,645 · Duration: 416s
Baseline (fcbfa091): 93% · 44,336 tokens · 250s
Generated by sdlc-workflow/run-evals v0.12.1
|
[sdlc-workflow/verify-pr] Re: @sourcery-ai[bot] review — Classified as suggestion (both items) — (1) paragraph vs. bullet formatting is not documented in CONVENTIONS.md and has no established codebase pattern; (2) wording conciseness is a stylistic preference with no documented convention. No sub-task created. |
Verification Report for TC-5056 (commit b01de6a)
Overall: PASSAll checks pass. The PR adds a single paragraph to Step 4 (Build Repository Impact Map) of the plan-feature SKILL.md, instructing the skill to explicitly flag non-plannable requirements with reasoning. The change is scoped, traceable, and satisfies all acceptance criteria. Eval pass rate improved from 93% baseline to 94%. This comment was AI-generated by sdlc-workflow/verify-pr v0.12.1. |
Baseline Investigation: Non-Plannable Requirement FlaggingTo validate whether the explicit instruction in this PR is necessary, I ran a focused baseline test: 3 independent subagents were given the pre-change Step 4 instructions (without the new paragraph) and the eval-2 fixture ( Results — Without the instruction (3/3 silently omit)
Every run produced only 3 backend changes (search performance, relevance, filters) and silently dropped the "Better UI" requirement with zero acknowledgment. Why the eval showed 93% → 94% (misleading delta)The overall eval pass rate improvement is modest because eval-2 assertion 2 was already passing at baseline — but that was due to the full eval prompt providing much richer context (complete SKILL.md, repository manifests, multi-step planning flow) which happened to trigger the flagging behavior incidentally. When testing the isolated Step 4 instruction, the model reliably omits the requirement 3/3 times without the explicit instruction. ConclusionThe instruction converts a 0% baseline (in isolation) to reliable flagging. This is a genuine behavioral fix, not redundant hardening. This comment was AI-generated by sdlc-workflow/verify-pr v0.12.1. |
1 participant
Summary
Implements TC-5056
Test plan
grep -i "non-plannable\|excluded requirements\|cannot be planned\|missing inputs" plugins/sdlc-workflow/skills/plan-feature/SKILL.md🤖 Generated with Claude Code
Summary by Sourcery
Documentation: