feat(default-skill-review): publish the default tessl review rubric as a plugin

README.md

@@ -10,6 +10,7 @@ These are R&D-maintained plugins intended for production use.
 | --- | --- | --- |
 | [`skill-optimizer`](skill-optimizer/) | Optimize your skills and plugins: review SKILL.md quality, generate eval scenarios, run evals, compare across models, diagnose gaps, and re-run until scores improve | [![](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2Ftesslio%2Fproduct-plugins%2Fmain%2Fskill-optimizer%2F.tessl-plugin%2Fplugin.json&query=%24.version&label=tessl&color=blue&prefix=v)](https://tessl.io/registry/tessl/skill-optimizer) [![](https://img.shields.io/endpoint?url=https%3A%2F%2Fapi.tessl.io%2Fv1%2Fbadges%2Ftessl%2Fskill-optimizer)](https://tessl.io/registry/tessl/skill-optimizer/evals) |
 | [`review-plugin-creator`](review-plugin-creator/) | Create a custom `tessl review` rubric – fork the default rubric, build one from scratch, or derive it from evidence (existing skills, PR feedback, agent logs) | [![review-plugin-creator version](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2Ftesslio%2Fproduct-plugins%2Fmain%2Freview-plugin-creator%2F.tessl-plugin%2Fplugin.json&query=%24.version&label=tessl&color=blue&prefix=v)](https://tessl.io/registry/tessl/review-plugin-creator) [![review-plugin-creator evals](https://img.shields.io/endpoint?url=https%3A%2F%2Fapi.tessl.io%2Fv1%2Fbadges%2Ftessl%2Freview-plugin-creator)](https://tessl.io/registry/tessl/review-plugin-creator/evals) |
+| [`default-skill-review`](default-skill-review/) | The default `tessl review` rubric – an agentic reviewer that scores a skill's description and content, the same reviewer `tessl review` uses out of the box. Score skills against it, or fork it into a custom rubric | [![default-skill-review version](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2Ftesslio%2Fproduct-plugins%2Fmain%2Fdefault-skill-review%2F.tessl-plugin%2Fplugin.json&query=%24.version&label=tessl&color=blue&prefix=v)](https://tessl.io/registry/tessl/default-skill-review) [![default-skill-review evals](https://img.shields.io/endpoint?url=https%3A%2F%2Fapi.tessl.io%2Fv1%2Fbadges%2Ftessl%2Fdefault-skill-review)](https://tessl.io/registry/tessl/default-skill-review/evals) |
 | [`api`](api/) | Find and call Tessl API endpoints token-efficiently, without loading the whole `openapi.json` spec into context | [![api version](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2Ftesslio%2Fproduct-plugins%2Fmain%2Fapi%2F.tessl-plugin%2Fplugin.json&query=%24.version&label=tessl&color=blue&prefix=v)](https://tessl.io/registry/tessl/api) [![api evals](https://img.shields.io/endpoint?url=https%3A%2F%2Fapi.tessl.io%2Fv1%2Fbadges%2Ftessl%2Fapi)](https://tessl.io/registry/tessl/api/evals) |
 
 ## Install

default-skill-review/.tessl-plugin/plugin.json

@@ -0,0 +1,9 @@
+{
+  "name": "tessl/default-skill-review",
+  "version": "0.1.0",
+  "description": "The default Tessl review rubric. An agentic reviewer that reads a SKILL.md and scores it against rubrics for its description and its content, the same reviewer tessl review uses out of the box. Use it as the reference reviewer to score skills, or as the base to fork a custom rubric from.",
+  "private": false,
+  "skills": [
+    "skills/skill-reviewer"
+  ]
+}

default-skill-review/README.md

@@ -0,0 +1,31 @@
+# default-skill-review
+
+The default `tessl review` rubric, packaged as a plugin. It is the exact reviewer `tessl review` uses out of the box.
+
+## Install
+
+```bash
+tessl install tessl/default-skill-review
+```
+
+## What it does
+
+`tessl review` scores a skill against a **reviewer plugin**. This is that reviewer: an agent that reads a `SKILL.md` and scores it against rubrics grounded in Anthropic best practices.
+
+It ships two rubrics (their weights live in `config.json`):
+
+| Rubric | What it measures |
+|--------|------------------|
+| `description` | How well the skill's description drives activation |
+| `content` | Quality of the SKILL.md body |
+
+Use it two ways:
+
+- **As the reference reviewer** — the reviewer `tessl review` resolves when no other reviewer is selected.
+- **As a base to fork** — start from this rubric and adjust weights, anchors, or judges with [`tessl/review-plugin-creator`](../review-plugin-creator/).
+
+## Skills
+
+| Skill | Description |
+|-------|-------------|
+| `skill-reviewer` | An agent that reads a SKILL.md and scores it against the description and content rubrics |

default-skill-review/skills/skill-reviewer/SKILL.md

@@ -0,0 +1,102 @@
+---
+name: skill-reviewer
+description: Evaluate a SKILL.md for quality. An agent that dynamically discovers the rubric files and scores the skill against each, producing one result per rubric. Use when running a skill review workflow or testing skill quality against configurable rubrics.
+---
+
+You are an expert skill quality evaluator. Your task is to assess the quality of a `SKILL.md` file by scoring it against each rubric file found in `./rubrics/` — one evaluation per rubric.
+
+## Step 1 — Read the skill
+
+Read `./SKILL.md`. Parse it into two parts:
+- **YAML frontmatter**: everything between the opening and closing `---` delimiters. Extract `description` (and `name` if present).
+- **Content body**: all markdown after the closing `---`.
+
+Then list any bundle files present in `./references/`, `./scripts/`, and `./assets/`. Do **not** load all bundle file contents into memory — read only files that are directly relevant to scoring `progressive_disclosure` (e.g., to verify references in the body are real files).
+
+## Step 2 — Discover rubrics and scoring config
+
+List all `.json` files in `./rubrics/`. Each file is one judge. Read each rubric file. Use the dimension `id`, `weight`, `scores`, and `scale` from these files — do not rely on memory.
+
+The file stem (filename without `.json`) is the judge name (e.g. `description.json` → judge name `description`).
+
+Read `./config.json`. This file contains:
+- `judges`: a map of rubric stem → `{ weight }` expressing each judge's contribution to the final score
+
+Construct the `scoring.components` list (judge components only — one entry per rubric file in discovery order):
+- `{ id: "<stem>", weight: config.judges[stem].weight, normalized: <judge normalizedScore> }`
+
+## Step 3 — Run judges
+
+For each rubric file discovered in Step 2, run one judge against the appropriate part of the skill:
+- Rubrics whose `evaluation_target` is `"description"` → evaluate the frontmatter `description` field.
+- Rubrics whose `evaluation_target` is `"content"` → evaluate the markdown body.
+- For rubrics with any other `evaluation_target`, use your judgment about what part of the skill to evaluate.
+
+For each judge, follow this process:
+1. Quote the specific phrases from the target text relevant to each dimension.
+2. Compare against the rubric anchors — which example is the closest match?
+3. Confirm why it is not the level above or below.
+4. Assign your score (must be within `scale.min`–`scale.max`).
+
+Produce one evaluation object per judge:
+```json
+{
+  "scores": {
+    "<dimension_id>": { "score": <number>, "reasoning": "<1-2 sentences>" }
+  },
+  "overall_assessment": "<2-3 sentence summary>",
+  "suggestions": []
+}
+```
+
+Every dimension `id` from the rubric must appear in `scores`. For strong results leave `suggestions` as `[]`. For weaker ones provide 2–3 actionable suggestions tied to the lowest-scoring dimensions.
+
+## Step 4 — Compute scores
+
+For **each judge**:
+
+**Weighted score** (using the rubric's dimension weights):
+```text
+weightedScore = sum(dimension.score * dimension.weight)
+```
+(All weights sum to 1.0.)
+
+**Normalized score** (maps weighted score to [0, 1] using rubric scale):
+```text
+normalizedScore = (weightedScore - scale.min) / (scale.max - scale.min)
+```
+
+## Step 5 — Write results
+
+Write `./results.json` conforming to `schemas/results.schema.json`.
+
+```json
+{
+  "judges": {
+    "<rubric stem>": {
+      "success": true,
+      "scale": { "min": <scale.min from rubric>, "max": <scale.max from rubric> },
+      "evaluation": <evaluation object from Step 3>,
+      "weightedScore": <computed in Step 4>,
+      "normalizedScore": <computed in Step 4>
+    }
+  },
+  "scoring": {
+    "components": [
+      { "id": "<rubric stem>", "weight": config.judges[stem].weight, "normalized": <judge normalizedScore> }
+    ]
+  }
+}
+```
+
+The `judges` object key is the rubric file stem (e.g. `description`, `content`). The scoring component `id` matches the same stem.
+
+If a judge fails (e.g., cannot parse the skill), set `success: false` and populate `errorMessage`. Do not omit the judge key — include it with `success: false`.
+
+## Important reminders
+
+- Read bundle files selectively — list them for context, read only what is needed.
+- Every dimension `id` from each rubric must appear in the corresponding judge's `scores` object.
+- All scores must be within `scale.min`–`scale.max`.
+- `results.json` must be valid JSON conforming to `schemas/results.schema.json`.
+- Do not include any commentary outside the `results.json` file.

default-skill-review/skills/skill-reviewer/references/config.json

@@ -0,0 +1,8 @@
+{
+  "$schema": "../schemas/config.schema.json",
+  "validation_weight": 0.2,
+  "judges": {
+    "description": { "weight": 0.4 },
+    "content": { "weight": 0.4 }
+  }
+}

default-skill-review/skills/skill-reviewer/references/rubrics/content.json

@@ -0,0 +1,136 @@
+{
+  "$schema": "../schemas/rubric.schema.json",
+  "evaluation_target": "content",
+  "scope": "SKILL.md body (markdown content after YAML frontmatter)",
+  "scale": {
+    "min": 1,
+    "max": 3
+  },
+  "reference_examples": {
+    "judging_guidelines": [
+      "Evaluate what is written, not what could be inferred or added.",
+      "Penalize explanations of concepts Claude already knows (e.g., what a PDF is, how libraries work, basic programming concepts).",
+      "Reward brevity that preserves clarity; penalize verbosity even if accurate.",
+      "Code examples should be executable and complete, not pseudocode, unless flexibility is explicitly justified.",
+      "Time-sensitive information (specific dates, version numbers) should penalize conciseness unless placed in an 'old patterns' or 'deprecated' section.",
+      "For simple skills (single task, no multi-step process), workflow clarity can score 3 if the single action is unambiguous.",
+      "For skills under 50 lines with no need for external references, progressive disclosure can score 3 with just well-organized sections.",
+      "Missing validation/verification steps in workflows involving destructive or batch operations should cap workflow clarity at 2.",
+      "When bundle files are provided, score `progressive_disclosure` against the actual bundle structure (file listing + any referenced paths) rather than on references alone."
+    ],
+    "good_overall_examples": [
+      "## Quick start\n\nExtract text with pdfplumber:\n```python\nimport pdfplumber\nwith pdfplumber.open(\"file.pdf\") as pdf:\n    text = pdf.pages[0].extract_text()\n```\n\n## Advanced features\n\n**Form filling**: See [FORMS.md](FORMS.md) for complete guide\n**API reference**: See [REFERENCE.md](REFERENCE.md) for all methods",
+      "## Commit message format\n\nGenerate commit messages following these examples:\n\n**Example 1:**\nInput: Added user authentication with JWT tokens\nOutput:\n```\nfeat(auth): implement JWT-based authentication\n\nAdd login endpoint and token validation middleware\n```\n\nFollow this style: type(scope): brief description, then detailed explanation.",
+      "## Document editing process\n\n1. Make your edits to `word/document.xml`\n2. **Validate immediately**: `python ooxml/scripts/validate.py unpacked_dir/`\n3. If validation fails:\n   - Review the error message carefully\n   - Fix the issues in the XML\n   - Run validation again\n4. **Only proceed when validation passes**\n5. Rebuild: `python ooxml/scripts/pack.py unpacked_dir/ output.docx`"
+    ],
+    "bad_overall_examples": [
+      "PDF (Portable Document Format) files are a common file format that contains text, images, and other content. To extract text from a PDF, you'll need to use a library. There are many libraries available for PDF processing, but we recommend pdfplumber because it's easy to use and handles most cases well. First, you'll need to install it using pip...",
+      "You can use pypdf, or pdfplumber, or PyMuPDF, or pdf2image, or camelot, or tabula-py depending on your needs. Each has different strengths...",
+      "Process the document.\n\nSee advanced.md for more details.\n\n(In advanced.md: See details.md for the actual instructions.)",
+      "If you're doing this before August 2025, use the old API. After August 2025, use the new API."
+    ]
+  },
+  "dimensions": [
+    {
+      "id": "conciseness",
+      "name": "Conciseness / Token Efficiency",
+      "weight": 0.3,
+      "question": "Does it respect token budget and assume Claude's intelligence?",
+      "rationale": "The context window is a public good. Every unnecessary token competes with conversation history and other context. Skills should add only what Claude doesn't already know.",
+      "scores": [
+        {
+          "score": 1,
+          "anchor": "Verbose; explains concepts Claude knows; padded with unnecessary context",
+          "example": "PDF (Portable Document Format) files are a common file format that contains text, images, and other content. To extract text from a PDF, you'll need to use a library. There are many libraries available..."
+        },
+        {
+          "score": 2,
+          "anchor": "Mostly efficient but includes some unnecessary explanation or could be tightened",
+          "example": "Use pdfplumber for text extraction. It's a good library that handles most PDF formats well.\n\n```python\nimport pdfplumber\nwith pdfplumber.open(\"file.pdf\") as pdf:\n    text = pdf.pages[0].extract_text()\n```"
+        },
+        {
+          "score": 3,
+          "anchor": "Lean and efficient; assumes Claude's competence; every token earns its place",
+          "example": "## Extract PDF text\n\n```python\nimport pdfplumber\nwith pdfplumber.open(\"file.pdf\") as pdf:\n    text = pdf.pages[0].extract_text()\n```"
+        }
+      ]
+    },
+    {
+      "id": "actionability",
+      "name": "Actionability",
+      "weight": 0.3,
+      "question": "Does it provide concrete, executable guidance vs vague direction?",
+      "rationale": "Skills succeed when Claude knows exactly what to do. Concrete code, specific commands, and clear examples beat abstract descriptions.",
+      "scores": [
+        {
+          "score": 1,
+          "anchor": "Vague or abstract; no concrete code/commands; describes rather than instructs",
+          "example": "Use a PDF library to extract the text content from the document and process it as needed."
+        },
+        {
+          "score": 2,
+          "anchor": "Some concrete guidance but incomplete; pseudocode instead of executable code; missing key details",
+          "example": "Use pdfplumber to extract text:\n```\n# pseudocode\nopen pdf\nfor each page:\n    extract text\n```"
+        },
+        {
+          "score": 3,
+          "anchor": "Fully executable code/commands; specific examples; copy-paste ready",
+          "example": "```python\nimport pdfplumber\nwith pdfplumber.open(\"file.pdf\") as pdf:\n    for page in pdf.pages:\n        print(page.extract_text())\n```\n\nFor tables: `page.extract_tables()` returns list of lists."
+        }
+      ]
+    },
+    {
+      "id": "workflow_clarity",
+      "name": "Workflow Clarity",
+      "weight": 0.25,
+      "question": "Are multi-step processes clearly sequenced with validation checkpoints?",
+      "rationale": "Complex operations fail when steps are unclear or validation is missing. Clear workflows with explicit checkpoints catch errors early and guide Claude through fragile operations.",
+      "scores": [
+        {
+          "score": 1,
+          "anchor": "Steps unclear or missing; no sequence for multi-step tasks; no validation mentioned for risky operations",
+          "example": "Edit the XML file, then rebuild the document. Make sure it's valid."
+        },
+        {
+          "score": 2,
+          "anchor": "Steps listed but validation gaps; sequence present but checkpoints missing or implicit",
+          "example": "1. Unpack the document\n2. Edit word/document.xml\n3. Repack the document\n4. Test the output"
+        },
+        {
+          "score": 3,
+          "anchor": "Clear sequence with explicit validation steps; feedback loops for error recovery; checklists for complex processes",
+          "example": "1. Unpack: `python scripts/unpack.py input.docx`\n2. Edit `word/document.xml`\n3. **Validate**: `python scripts/validate.py unpacked/`\n4. If errors: fix and re-validate\n5. **Only when valid**: `python scripts/pack.py unpacked/ output.docx`"
+        }
+      ]
+    },
+    {
+      "id": "progressive_disclosure",
+      "name": "Progressive Disclosure",
+      "weight": 0.15,
+      "question": "Does it appropriately structure content across files with clear navigation?",
+      "rationale": "SKILL.md should be an overview pointing to detailed materials. References should be one level deep, clearly signaled, and organized for discovery.",
+      "scores": [
+        {
+          "score": 1,
+          "anchor": "Monolithic wall of text; deeply nested references (2+ levels); poor organization",
+          "example": "See advanced.md for details.\n\n(In advanced.md: See details.md for the actual information.)\n\n(In details.md: The real content is here.)"
+        },
+        {
+          "score": 2,
+          "anchor": "Some structure but could be better organized; references present but not clearly signaled; content that should be separate is inline",
+          "example": "## Overview\n[200 lines of API reference that could be in a separate file]\n\n## Advanced\nSee ADVANCED.md"
+        },
+        {
+          "score": 3,
+          "anchor": "Clear overview with well-signaled one-level-deep references; content appropriately split; easy navigation",
+          "example": "## Quick start\n[Concise getting-started content]\n\n## Advanced features\n**Form filling**: See [FORMS.md](FORMS.md)\n**API reference**: See [REFERENCE.md](REFERENCE.md)\n**Examples**: See [EXAMPLES.md](EXAMPLES.md)"
+        }
+      ]
+    }
+  ],
+  "scoring_notes": {
+    "simple_skills": "For simple, single-purpose skills (under 50 lines, single task), workflow_clarity and progressive_disclosure can score 3 if the content is clear and well-organized, even without explicit multi-step workflows or external file references.",
+    "code_vs_instruction_skills": "Apply the same rubric to both. Code skills should have executable examples; instruction-only skills should have concrete, specific guidance. Absence of code in an instruction-only skill is not penalized if the guidance is actionable.",
+    "feedback_loops": "Feedback loops (validate -> fix -> retry) are especially important for: batch operations, destructive changes, XML/document manipulation, database operations. Missing feedback loops in these contexts should cap workflow_clarity at 2."
+  }
+}