Skip to content

feat: wire threat-model.md into Phase 6 reporting context#54

Merged
pruiz merged 1 commit into
masterfrom
feature/37-threat-model-phase6
Jun 12, 2026
Merged

feat: wire threat-model.md into Phase 6 reporting context#54
pruiz merged 1 commit into
masterfrom
feature/37-threat-model-phase6

Conversation

@pruiz

@pruiz pruiz commented Jun 12, 2026

Copy link
Copy Markdown
Owner

Summary

Wires itemdb/notes/threat-model.md into Phase 6 (reporting) so the report can draw on the operational threat model for methodology context, attacker assumptions, trust boundaries, affected assets, and limitations — without duplicating the artifact or inflating severity beyond what confirmed evidence supports.

Closes #37.

Changes

File Change
prompts/phase-6-report.md Added threat-model.md to required reading (conditional), added reporting rule for grounding Methodology/Scope/Limitations from the threat model with an explicit guardrail against speculative severity inflation, added threat-model limitation example
.opencode/agents/reporter.md Added conditional threat-model.md to required reading, added guidance to Methodology (attacker model summary), Scope (trust boundaries, assets, security objectives), and Limitations (open assumptions affecting severity or exploitability)
.opencode/skills/report-writing/SKILL.md Added conditional threat-model.md to inputs
templates/report.md Added threat-model placeholders to Methodology, Scope, and Limitations sections (all with conditional qualifiers)
tests/test_prompts_threat_model.py Added 14 new tests covering Phase 6 prompt, reporter agent (including Limitations), report-writing skill, and report template

Design decisions

  • No new top-level report section — threat-model context is woven into existing Methodology, Scope, and Limitations sections. This satisfies "summarize relevant context without duplicating the whole artifact."
  • Explicit guardrail — a reporting rule ensures severity/context language remains tied to confirmed findings and evidence, NOT speculative abuse-path themes from the threat model (acceptance criteria docs(README): add screenshots section #3).
  • All references are conditional (If present, when available) — projects without threat-model.md see no change in behavior (acceptance criteria web: spaces #4).
  • Follows established pattern — same wiring approach as Phases 2–5.

Acceptance criteria coverage

  • Phase 6 prompt explicitly references itemdb/notes/threat-model.md when present.
  • Reports may summarize relevant threat-model context without duplicating the whole artifact (woven into Methodology/Scope/Limitations).
  • Severity/context language remains tied to confirmed findings and evidence, not speculative abuse-path themes (explicit guardrail in reporting rules).
  • Existing Phase 6 behavior remains compatible with projects that do not yet have threat-model artifacts (all conditional).

Review feedback addressed

All three Greptile P2 findings from the first review round have been fixed:

  1. templates/report.md — Limitations example now conditional: Added (only when \itemdb/notes/threat-model.md` is present)` qualifier, consistent with Methodology and Scope additions in the same file.
  2. .opencode/agents/reporter.md — Limitations guidance added: Reporter agent now includes open-assumption guidance in its Limitations section, closing the gap vs. the prompt and template.
  3. tests/test_prompts_threat_model.py — Guardrail test tightened: Removed the redundant "do not" assert that would pass trivially from pre-existing directives. Added a dedicated test for reporter agent Limitations guidance (test_reporter_agent_mentions_threat_model_in_limitations).

Testing

make tests passes (814 tests, frontmatter validation, artifact checks).

Summary by CodeRabbit

  • Documentation

    • Security reports now conditionally incorporate threat model context, including attacker assumptions, trust boundaries, and in-scope assets across Methodology, Scope, and Limitations sections.
    • Added safeguards to prevent severity inflation from speculative scenarios and ensure findings are grounded in confirmed evidence.
  • Tests

    • Added test coverage for threat model integration across reporting components.

@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 67a19efa-8f64-429a-8ecd-c8494855b273

📥 Commits

Reviewing files that changed from the base of the PR and between abee9b8 and d3cbeb5.

📒 Files selected for processing (5)
  • .opencode/agents/reporter.md
  • .opencode/skills/report-writing/SKILL.md
  • prompts/phase-6-report.md
  • templates/report.md
  • tests/test_prompts_threat_model.py

📝 Walkthrough

Walkthrough

Phase 6 reporting now optionally uses itemdb/notes/threat-model.md to ground methodology, scope, and limitations sections. Agent instructions, prompt rules, and report templates are updated to reference threat-model attacker context, trust boundaries, and assumptions while guarding against speculative severity inflation. Integration tests validate all components together.

Changes

Phase 6 threat-model integration

Layer / File(s) Summary
Threat-model input declaration
.opencode/agents/reporter.md, .opencode/skills/report-writing/SKILL.md
Agent-level Required reading and skill-level Inputs sections now recognize itemdb/notes/threat-model.md (when present) as an optional input for Phase 6 reporting context.
Phase 6 prompt threat-model rules
prompts/phase-6-report.md
Phase 6 prompt explicitly incorporates threat-model.md, instructs use for grounding Methodology/Scope/Limitations, and adds anti-speculation guards to prevent abuse-path themes from inflating severity beyond confirmed evidence.
Methodology, Scope, Limitations section guidance
.opencode/agents/reporter.md
Agent instructions provide section-specific guidance: Methodology summarizes attacker-model capabilities/constraints, Scope references trust boundaries and assets, Limitations includes open assumptions affecting severity or exploitability.
Report template conditional blocks
templates/report.md
Report template adds conditional subsections (when threat-model.md is present): Methodology documents assumed attacker model, Scope lists trust boundaries and in-scope assets/security objectives, and Limitations notes threat-model assumption impact.
Integration test validation
tests/test_prompts_threat_model.py
Test assertions across agent instructions, skills, Phase 6 prompt, and report templates verify threat-model references, conditional availability language, attacker-model terminology, trust-boundary content, and speculation guards.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

  • pruiz/CodeCome#53: Both PRs wire itemdb/notes/threat-model.md into adjacent workflow phases (Phase 6 reporting vs Phase 5 exploitation) and both extend threat-model test coverage across phase-specific prompts and templates.

Poem

🐰 A threat model emerges from Phase One's deep thought,
Now Phase Six reports weave what the threat space has taught—
Trust bounds and assumptions, in scope, laid out clean,
No speculative shadows where evidence hasn't been,
Reports grounded solid, from threat model to test!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat: wire threat-model.md into Phase 6 reporting context' clearly and concisely summarizes the main objective of integrating the threat model artifact into Phase 6 reporting.
Linked Issues check ✅ Passed All acceptance criteria from issue #37 are met: threat-model.md is referenced in Phase 6 prompts/templates when present, reports can summarize threat-model context without duplication, severity language is guarded against speculation, and Phase 6 behavior remains compatible with projects without threat-model.md via conditional references.
Out of Scope Changes check ✅ Passed All changes are directly related to integrating threat-model.md into Phase 6 reporting context; no out-of-scope modifications to unrelated components or features are present.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/37-threat-model-phase6

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown

Coverage Report

Metric Value
Line Coverage 75.8%
Lines Covered 0 / 0

Download detailed HTML coverage reports per OS/Python from the workflow artifacts.

Generated by pytest-cov on 2026-06-12T18:16:16.147Z

@greptile-apps

greptile-apps Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR wires itemdb/notes/threat-model.md into Phase 6 reporting by adding conditional references and guidance to the reporter agent, report-writing skill, Phase 6 prompt, and report template, along with 13 new tests covering the integration.

  • All threat-model additions are correctly gated behind conditional language (If present, when available, when the threat model is available), preserving backward compatibility for projects without a threat-model artifact.
  • The Limitations section is updated in all three instruction layers (prompt, reporter agent, template) with inline conditional qualifiers, and the guardrail against speculative severity inflation is present in prompts/phase-6-report.md.

Confidence Score: 5/5

All changes are additive, documentation-only, and correctly conditioned on artifact presence — no behavior change for projects without a threat model.

Every instruction layer (prompt, reporter agent, skill, template) is updated consistently. Conditional language is present in all four files. Previously raised concerns about missing Limitations guidance in the reporter agent and the unconditional Limitations example in the template are both addressed in this revision.

No files require special attention.

Important Files Changed

Filename Overview
prompts/phase-6-report.md Adds conditional required-reading entry, a reporting rule with explicit severity-inflation guardrail, and a conditional Limitations example — all properly gated.
.opencode/agents/reporter.md Adds conditional threat-model guidance to Required reading, Methodology, Scope, and Limitations sections — all four sections updated consistently.
.opencode/skills/report-writing/SKILL.md Adds conditional threat-model.md to the skill's inputs list — minimal, correct addition.
templates/report.md Adds conditional guidance blocks to Methodology and Scope, and a correctly-qualified inline Limitations example with the 'only when' conditional qualifier.
tests/test_prompts_threat_model.py Adds 13 presence-check tests for the threat-model integration across all four changed artifacts; guardrail test now uses specific terms ('inflate'/'abuse-path') rather than the previous generic 'do not' check.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    TM[("itemdb/notes/threat-model.md\n(optional)")]

    subgraph Phase6["Phase 6 – Report Generation"]
        P6["prompts/phase-6-report.md\n(required reading + guardrail rule)"]
        RA[".opencode/agents/reporter.md\n(required reading + section guidance)"]
        SK[".opencode/skills/report-writing/SKILL.md\n(inputs list)"]
    end

    subgraph Template["Report Template"]
        RPT["templates/report.md\n(Methodology / Scope / Limitations)"]
    end

    TM -->|"If present"| P6
    TM -->|"If present"| RA
    TM -->|"If present"| SK

    P6 --> RPT
    RA --> RPT

    RPT --> M["Methodology\n(attacker model summary)"]
    RPT --> S["Scope\n(trust boundaries, assets)"]
    RPT --> L["Limitations\n(open assumptions — conditional)"]

    style TM fill:#f9f,stroke:#999,stroke-dasharray:5
    style M fill:#dff,stroke:#aaa
    style S fill:#dff,stroke:#aaa
    style L fill:#dff,stroke:#aaa
Loading

Reviews (2): Last reviewed commit: "feat: wire threat-model.md into Phase 6 ..." | Re-trigger Greptile

Comment thread templates/report.md Outdated
Comment thread .opencode/agents/reporter.md
Comment thread tests/test_prompts_threat_model.py
Closes #37.

- prompts/phase-6-report.md: add threat-model.md to required reading
  (conditional), add reporting rule for grounding Methodology/Scope/
  Limitations from the threat model without inflating severity beyond
  confirmed evidence, add threat-model limitation example.
- .opencode/agents/reporter.md: add conditional threat-model.md to
  required reading, add guidance to Methodology (attacker model summary),
  Scope (trust boundaries, assets, security objectives), and Limitations
  (open assumptions affecting severity or exploitability) sections.
- .opencode/skills/report-writing/SKILL.md: add conditional
  threat-model.md to inputs.
- templates/report.md: add threat-model placeholders to Methodology
  (attacker model, threat assumptions, trust boundaries), Scope
  (trust boundaries, assets, attacker postures), and Limitations
  (conditional qualifier).
- tests/test_prompts_threat_model.py: add 14 tests covering Phase 6
  prompt, reporter agent (including Limitations), report-writing skill,
  and report template.

All references are conditional (if present / when available) so projects
without threat-model.md continue to work unchanged.
@pruiz pruiz force-pushed the feature/37-threat-model-phase6 branch from 51e0162 to d3cbeb5 Compare June 12, 2026 18:14
@pruiz

pruiz commented Jun 12, 2026

Copy link
Copy Markdown
Owner Author

All three P2 findings addressed in d3cbeb5:

  1. templates/report.md — unconditional Limitations example: Fixed. Now includes (only when \itemdb/notes/threat-model.md` is present)` qualifier, consistent with Methodology and Scope additions.

  2. .opencode/agents/reporter.md — missing Limitations guidance: Fixed. Added open-assumption guidance to the Limitations section, closing the gap vs. the prompt and template.

  3. tests/test_prompts_threat_model.py — overly broad guardrail test: Fixed. Removed the redundant "do not" assert. Also added test_reporter_agent_mentions_threat_model_in_limitations to cover the new Limitations guidance in the agent.

814 tests passing.

@pruiz pruiz merged commit cb84c51 into master Jun 12, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use threat-model.md in Phase 6 reporting context

1 participant