Skip to content

epic-04/05: add discovery reasoning guardrails - #28

Merged
mtalvi merged 2 commits into
rh-ai-quickstart:mainfrom
mtalvi:epic-04/05-reasoning-guardrails
Jul 29, 2026
Merged

epic-04/05: add discovery reasoning guardrails#28
mtalvi merged 2 commits into
rh-ai-quickstart:mainfrom
mtalvi:epic-04/05-reasoning-guardrails

Conversation

@mtalvi

@mtalvi mtalvi commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Related Ticket:
APPENG-5567

Summary

Adds reasoning-guardrails.md to rh-qs-discovery — the mental framework the agent uses during PRD generation to catch common pitfalls without reducing quality checks to a mechanical checklist.

What this file does

Defines 5 core concern areas the agent should reason about during discovery:

  1. Scope Creep — did the user actually ask for this, or am I adding it?
  2. Technology Bias — am I defaulting to RAG/complex stacks when simpler options work?
  3. GPU Assumptions — does this actually need GPU, or am I assuming "AI = GPU"?
  4. Completeness Without Over-specification — enough detail for architecture, but not pre-deciding the design
  5. User Voice Fidelity — does the PRD reflect what the user said, not what I think they meant?

Plus context-specific sections for gap analysis mode and uploaded documents.

Design decisions

  • Concern areas, not checklists. Follows the reasoning-guardrails-template — each area has "what to consider," "key questions," and "where to look" instead of pass/fail items.
  • Dynamic reasoning example included. Shows how guardrails integrate into natural thinking flow (lines 144-176), so future agents understand the intended usage pattern.
  • Self-check before PRD generation. A lightweight verification list (lines 187-192) the agent runs right before writing the final PRD to disk.

Test plan

  • File follows the reasoning-guardrails-template structure: concern areas with "what to consider" / "key questions" / "where to look" (not checklists)
  • All 5 concern areas are domain-specific to discovery (not generic boilerplate)
  • Cross-references to prd-structurer subagent match the file name from PR-02
  • No stale file paths or references to files that don't exist in the skill directory
  • Gap analysis and uploaded document sections are clearly marked as context-specific (not always applied)

@mtalvi
mtalvi marked this pull request as ready for review July 27, 2026 07:27
@mtalvi
mtalvi force-pushed the epic-04/05-reasoning-guardrails branch from a6623c0 to a2346db Compare July 28, 2026 07:34

@YosiElias YosiElias left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks @mtalvi ! 🚀
I've left 2 comments on the new file of this PR, pls LMK what you think.

Comment thread core/skills/rh-qs-discovery/reasoning-guardrails.md Outdated
Comment thread core/skills/rh-qs-discovery/reasoning-guardrails.md
@mtalvi
mtalvi force-pushed the epic-04/05-reasoning-guardrails branch from a2346db to 22426ac Compare July 28, 2026 13:36
Five concern areas the agent reasons about organically during PRD
generation: scope creep, technology bias, GPU assumptions, completeness
without over-specification, and user voice fidelity. Follows the
concern-area format from docs/foundation/reasoning-guardrails-template.md.

Co-authored-by: Cursor <cursoragent@cursor.com>
@mtalvi
mtalvi force-pushed the epic-04/05-reasoning-guardrails branch from 22426ac to 84fc3ee Compare July 29, 2026 11:37
@github-actions

Copy link
Copy Markdown

🔍 Skill Validation (Advisory — does not block merge)

Errors

  • broken internal link: ./references/industry-trends.md (file not found)

⚠️ Warnings

  • unexpected file at root: output-templates.md — if agents need this file, move it into references/ or assets/ as appropriate; otherwise remove it to avoid unnecessary context window usage
  • unexpected file at root: reasoning-guardrails.md — if agents need this file, move it into references/ or assets/ as appropriate; otherwise remove it to avoid unnecessary context window usage
  • unexpected file at root: spec-template.md — if agents need this file, move it into references/ or assets/ as appropriate; otherwise remove it to avoid unnecessary context window usage
  • unknown directory: subagents/ (contains 3 files) — agents using the standard skill structure won't discover these files; should this be assets/?
  • potentially unreferenced file: references/gap-analysis-template.md — agents may not discover this file without an explicit reference in SKILL.md or a referenced file

📋 Full report

The Technology Bias and GPU Assumptions sections only flagged
"assuming on-cluster serving" as a bias to watch for, with no
equivalent check for assuming remote endpoints. This nudged the
agent toward recommending remote inference by default, undermining
RHOAI's on-cluster model serving as a showcase feature. Reworded to
make both directions equally dependent on user input rather than
agent assumption.

Co-authored-by: Cursor <cursoragent@cursor.com>
@mtalvi
mtalvi requested a review from YosiElias July 29, 2026 11:41
@github-actions

Copy link
Copy Markdown

🔍 Skill Validation (Advisory — does not block merge)

Errors

  • broken internal link: ./references/industry-trends.md (file not found)

⚠️ Warnings

  • unexpected file at root: output-templates.md — if agents need this file, move it into references/ or assets/ as appropriate; otherwise remove it to avoid unnecessary context window usage
  • unexpected file at root: reasoning-guardrails.md — if agents need this file, move it into references/ or assets/ as appropriate; otherwise remove it to avoid unnecessary context window usage
  • unexpected file at root: spec-template.md — if agents need this file, move it into references/ or assets/ as appropriate; otherwise remove it to avoid unnecessary context window usage
  • unknown directory: subagents/ (contains 3 files) — agents using the standard skill structure won't discover these files; should this be assets/?
  • potentially unreferenced file: references/gap-analysis-template.md — agents may not discover this file without an explicit reference in SKILL.md or a referenced file

📋 Full report

@YosiElias YosiElias left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Thanks @mtalvi !

@mtalvi
mtalvi merged commit a5bb9db into rh-ai-quickstart:main Jul 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants