feat(review-plugin-creator): fetch the default rubric instead of bundling it#15
Merged
Merged
Conversation
…ling it The creator shipped its own copy of the default reviewer (rubrics, config, reviewer SKILL.md, schemas), which drifts from the published default. Drop all of it and fetch tessl/default-skill-review from the public GitHub repo on demand (codeload tarball, one subdir, no install) for both the fork and scratch paths. Single source of truth: the creator carries no default-rubric copy. Also drops the Trust-boundary guardrail from the fork instructions — tessl/default-skill-review is now the source of truth for the reviewer prompt. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…shipping fixtures Drop the bundled default/schema/reviewer fixtures from scenario-0 and scenario-1 and the `include` that staged them. The agent now fetches tessl/default-skill-review from GitHub per the skill, so the evals exercise the real fetch+fork path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (25)
💤 Files with no reviewable changes (20)
📝 WalkthroughWalkthroughRemoves all bundled default rubric JSON files, schema definitions, and reviewer-SKILL templates from the ChangesRemove bundled default rubric; fetch on demand
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
abheer-t
reviewed
Jun 29, 2026
abheer-t
left a comment
abheer-t
left a comment
Contributor
There was a problem hiding this comment.
Approving it works as is, however, is the pattern of installing downloaded skills / plugins in a temporary directory reasonable, or does it pollute the filesystem for users?
| "$PLUGIN_DIR/skills/skill-reviewer/references/schemas/" | ||
| cp <path-to-this-skill>/references/schemas/results.schema.json \ | ||
| "$PLUGIN_DIR/skills/skill-reviewer/references/schemas/" | ||
| DEFAULT_DIR=$(mktemp -d) |
Contributor
There was a problem hiding this comment.
Query: Why doesn't this use tessl install, and instead downloads it temporarily?
Contributor
Author
There was a problem hiding this comment.
tessl install would install the plugin (i.e. it would become avaliable as a skill to your agent), whereas we just want the files for reference.
Contributor
Author
Could direct it just to read the files directly without downloading them - I'll have a look at this next. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stops
review-plugin-creatorshipping its own copy of the default reviewer, which drifts from the published default. The fork and scratch flows now fetchtessl/default-skill-reviewfrom the public GitHub repo on demand — a single source of truth.What changed
references/default-rubric/,reviewer-SKILL.md,schemas/— the wholereferences/dir).default-skill-reviewplugin fromcodeload.github.com(one subdir, public, no auth, no install, into a temp dir); both the fork (Path A) and scratch (Path B) scaffold from it.tessl/default-skill-reviewis now the source of truth for the reviewer prompt too.include) so the agent fetches the default live, giving real coverage of the fetch+fork path.0.2.0 → 0.3.0.Validation
Ran the two scenarios as a real eval (avg 99% — scenario-0 100/100, scenario-1 98/100). Pulled the produced workspaces and confirmed the agent fetched the real default from GitHub in-sandbox: the forked plugin's
content.json/description.jsonmatch the canonical default, itsSKILL.mdis the no-guardrail default extended for the new judge, andconfig.jsonweights sum to 1.0.Closes OPT-1413
Summary by CodeRabbit
Documentation
Chores