Skip to content

Add detailed PR description skill#3

Merged
biw merged 1 commit into
mainfrom
biw/detailed-pr-description-skill
Jun 27, 2026
Merged

Add detailed PR description skill#3
biw merged 1 commit into
mainfrom
biw/detailed-pr-description-skill

Conversation

@biw

@biw biw commented Jun 27, 2026

Copy link
Copy Markdown
Owner

Summary

This PR adds a new installable Agent Skill named detailed-pr-description. The skill turns an ad hoc prompt for writing rich PR descriptions into a reusable workflow that tells agents how to inspect a PR, gather tracked repository context, write a self-contained PR body, include reviewer-focused gotchas and code snippets, and explicitly assess whether the branch's tests are strong enough to catch regressions.

The only runtime surface added here is skill documentation and Codex/OpenAI skill metadata. There are no application code changes, no repository validator changes, and no new helper scripts.

What This PR Does

  • Adds skills/detailed-pr-description/SKILL.md, an installable skill file with valid frontmatter and a complete workflow for writing or updating GitHub PR descriptions.
  • Adds skills/detailed-pr-description/agents/openai.yaml, so Codex/OpenAI surfaces can display and implicitly invoke the skill for PR-description work.
  • Defines a concrete PR-description workflow:
    • identify the PR and comparison base,
    • inspect tracked project artifacts rather than relying on chat memory,
    • treat .context/ as untracked scratch context that must be restated in the PR body when relevant,
    • review changed code from a reviewer perspective,
    • assess tests directly,
    • update the PR body with gh pr edit when available.
  • Provides a default PR body structure with sections for summary, scope, improvements, non-goals, important context, code snippets, gotchas, test coverage, and follow-up work.
  • Sets a quality bar that asks agents to keep claims factual, distinguish verified facts from assumptions, include failed or skipped validation, and avoid unrelated code changes while updating PR descriptions.

What This Improves

  • Makes a common review-prep request reusable instead of relying on a one-off prompt copied between conversations.
  • Pushes agents to write PR descriptions that stand alone for reviewers who cannot see the chat transcript or local .context/ notes.
  • Adds an explicit regression-coverage assessment to the workflow, which is the part of the original prompt most likely to be skipped by a generic "write a PR description" instruction.
  • Gives reviewers a predictable shape for PR bodies, including the "what this does not do" and "follow-up work" sections that help prevent scope confusion.
  • Improves discovery in Codex/OpenAI skill lists through agents/openai.yaml.

What This Does Not Do

  • Does not add executable helper scripts for PR analysis or GitHub API calls.
  • Does not change existing skills, templates, validation scripts, package metadata, or CI.
  • Does not add automated behavioral tests that prove an agent will follow the skill well on arbitrary future PRs.
  • Does not require every PR description to use this exact template; the skill says to use the structure unless the target repository has a stronger PR template.
  • Does not make PR-description updates possible without GitHub access. The skill falls back to writing a draft body under .context/ when gh pr edit cannot update the PR.

Important Context

  • This repository publishes skills from skills/<skill-name>/SKILL.md. The new skill follows that convention with skills/detailed-pr-description/SKILL.md.
  • The skill frontmatter name exactly matches the parent directory, which is required by scripts/validate-skills.mjs.
  • The frontmatter description is the trigger surface for skill discovery. It explicitly mentions updating GitHub PR descriptions, self-contained change context, gotchas, follow-up work, code snippets, and test coverage assessment.
  • .context/ is intentionally not tracked. The skill calls that out directly so future agents do not leave important review context only in local scratch files.
  • The skill intentionally has no scripts/, references/, or assets/ directory. This workflow is judgment-heavy and repository-specific, so the reusable value is the procedure and output standard rather than deterministic local automation.

Code To Review Closely

skills/detailed-pr-description/SKILL.md

The frontmatter is what determines installability and discovery. Reviewers should confirm the skill name matches the folder and that the description is specific enough to trigger on PR-description requests without being so broad that it fires for unrelated GitHub work.

---
name: detailed-pr-description
description: Write or update GitHub PR descriptions with self-contained change context, gotchas, follow-up work, code snippets, and test coverage assessment.
---

The core workflow is the most important part of the skill. It tells the agent to inspect tracked artifacts, treat .context/ as non-authoritative scratch space, and explicitly answer whether the tests are sufficient.

2. Build the context from tracked project artifacts.
   - Inspect `git status --short`, `git diff --stat <base>...HEAD`, `git diff --name-only <base>...HEAD`, relevant commits, changed files, tests, docs, migrations, config, and generated artifacts that are part of the branch.
   - Read `.context/` only as local scratch context. Anything important from `.context/` must be restated in the PR description because `.context/` is not tracked.
   - Do not rely on memory or prior chat alone for claims about behavior.

4. Assess the tests directly.
   - Inventory existing tests affected by the diff and any new tests added by the branch.
   - Run the narrowest reliable validation first, then the repository's documented broader validation when feasible.
   - Answer the regression question explicitly: are the tests sufficient, partially sufficient, or insufficient? Explain which regressions they would catch and which important regressions could still slip through.

The body template deliberately includes sections that the original prompt asked for, including non-goals, gotchas, code snippets, follow-up work, and test coverage. The outer fence uses four backticks so the example can safely contain an inner code fence.

## What This PR Does
## What This Improves
## What This Does Not Do
## Important Context
## Code To Review Closely
## Gotchas, Risks, And Edge Cases
## Tests And Regression Coverage
## Follow-Up Work

skills/detailed-pr-description/agents/openai.yaml

This metadata makes the skill show up cleanly in Codex/OpenAI skill UI and allows implicit invocation when the user asks for this kind of PR-description work.

interface:
  display_name: "Detailed PR Description"
  short_description: "Write review-ready PR descriptions"
  default_prompt: "Use $detailed-pr-description to update my PR description with detailed context, gotchas, follow-up work, and a test coverage assessment."
policy:
  allow_implicit_invocation: true

Gotchas, Risks, And Edge Cases

  • The skill is procedural, not executable. Repository validation can prove the skill is structurally valid and discoverable, but it cannot prove a future agent will make the right judgment calls on a complex PR.
  • allow_implicit_invocation: true is intentional because users often ask for "update the PR description" without naming the skill. Reviewers should still check whether the description is scoped tightly enough for that implicit trigger.
  • The skill tells agents to use gh pr edit for updates, but GitHub auth and network conditions remain external dependencies. The fallback path is to leave a draft in .context/pr-description.md.
  • The PR body template is intentionally verbose. That matches the original request to err on the side of too much context, but it means agents should still trim sections that are truly irrelevant or mark them Not applicable.
  • The standalone Python skill validator requires PyYAML; this PR used uv run --with pyyaml ... to avoid changing the repository's Python environment.

Tests And Regression Coverage

Validation Run

  • uv run --with pyyaml python /Users/ben/.codex/skills/.system/skill-creator/scripts/quick_validate.py skills/detailed-pr-description: passed with Skill is valid!
  • pnpm validate: passed with Validated 6 skills.
  • pnpm skills add . --list --yes: passed and listed 6 available skills, including detailed-pr-description.

Coverage Assessment

The current checks are partially sufficient for this branch.

They are strong enough for the mechanical contract of publishing a skill: they verify frontmatter shape, folder/name alignment, allowed installable location, description length constraints, OpenAI metadata presence, YAML validity, required interface fields, and local CLI discovery. For a docs-only skill addition, that is the highest-risk structural surface.

They are not strong enough to guarantee the skill's behavioral quality. They do not automatically prove that a future agent will gather enough PR context, include the right code snippets, make a sound regression-coverage judgment, preserve useful existing PR body content, or choose the right level of detail for every repository.

Regressions These Tests Should Catch

  • The skill file is moved outside an allowed installable path.
  • The frontmatter is missing, malformed, or not valid YAML.
  • name stops matching the detailed-pr-description directory.
  • The description becomes too long for the repository validator's compatibility limit.
  • agents/openai.yaml is missing, malformed, or lacks required display fields.
  • policy.allow_implicit_invocation is present but not a boolean.
  • The skills CLI can no longer discover the repository's installable skills locally.

Gaps And Recommended Follow-Up Tests

  • Add a lightweight content check if this repository wants to enforce required skill sections for workflow-heavy skills, such as checking for Workflow, Quality Bar, and Final Response.
  • Add a fixture-based or snapshot-style test for expected pnpm skills add . --list --yes output if CLI discovery regressions have been recurring.
  • Forward-test the skill on one or two real PRs after it lands, using fresh agent runs, to see whether the workflow produces the intended level of PR-description detail without leaking local .context/ assumptions.
  • Consider a future helper script only if repeated usage shows agents are rewriting the same PR-inspection commands incorrectly.

Follow-Up Work

  • Use the skill on another non-trivial PR and refine the instructions if agents miss important context or overproduce boilerplate.
  • Consider adding a short reference file with examples of excellent PR descriptions if future usage shows the body-template guidance is too abstract.
  • Consider adding repository-level validation for required agents/openai.yaml policy defaults if more skills should consistently allow or disallow implicit invocation.

Screenshots, Rollout, Migrations, And Release Notes

  • Screenshots: not applicable; this is a text skill and metadata change.
  • Rollout: merging to main publishes the skill from this repository in the same way existing skills are published.
  • Migrations: not applicable.
  • Release notes: mention that detailed-pr-description is available for agents that need to write or update detailed GitHub PR descriptions.

@biw biw marked this pull request as ready for review June 27, 2026 21:02
@biw biw merged commit 01be8f1 into main Jun 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant