Skip to content

test(skills): fail the build when a skill is not installable - #93

Merged
VickyXAI merged 1 commit into
mainfrom
test/marketplace-registration-guard
Aug 5, 2026
Merged

test(skills): fail the build when a skill is not installable#93
VickyXAI merged 1 commit into
mainfrom
test/marketplace-registration-guard

Conversation

@VickyXAI

@VickyXAI VickyXAI commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Closes the recurrence risk left open by #91.

The gap

Nine of thirteen skills were unreachable until 0.37.1 — on disk, shipped inside the npm package, and installable by nobody, because /plugin marketplace add shows exactly the entries in .claude-plugin/marketplace.json. #91 backfilled all thirteen but nothing stops the fourteenth from drifting the same way. The original cause was not a decision: an entry got added alongside whichever skill prompted it, and the rest were never backfilled. Nothing failed, because nothing looked.

What this checks

Both directions, because both are silent:

Invariant What it catches
every skills/<name>/SKILL.md has an entry the actual bug — shipped but unreachable
every entry has a skills/<name>/SKILL.md a renamed or moved directory: a broken install, not a missing one
source == ./skills/<name> the wrong skill installed under the right label, which looks like it worked
non-empty description registered but unchosen — it's the only text shown before install
unique names

Why a test and not a script

npm test already gates PRs and publish.yml, so this needs no new CI step. brand-numbers.test.ts established the pattern in this repo: assert the invariant against the real files, in the repo that can fix it. And nothing here is usefully auto-fixable — a marketplace description is hand-written prose, so generating one from the frontmatter would produce worse text than the omission it replaces.

It is the sibling of skill-frontmatter.test.ts, which proves a SKILL.md can load. This proves it can be reached. Neither implies the other. Both take "a directory with SKILL.md at its root" as the definition of a skill — the same definition on purpose, or one of them can pass vacuously.

Verification

Green at 288/288, and confirmed it can actually fail — each invariant broken in turn, each producing the matching failure and nothing else:

### drop the 'rpc' entry        -> not ok 2 - every skill on disk is registered
### entry naming a ghost skill  -> not ok 2, not ok 3
### wrong source path           -> not ok 4 - source path matches its name
### empty description           -> not ok 5
### duplicate entry             -> not ok 6

The unregistered-skill failure names the directories and prints the JSON to paste:

these skills exist on disk but are NOT installable — nobody can reach them:
  skills/modal/
  skills/rpc/
  Add an entry to .claude-plugin/marketplace.json:
    { "name": "modal", "source": "./skills/modal", "description": "Use when …" }

Test-only — skills/ and dist ship to npm, test/ does not, so this needs no version bump or release.

Nine of thirteen skills were unreachable until 0.37.1 — shipped, published to
npm, and listed by nobody, because /plugin marketplace add shows exactly the
entries in .claude-plugin/marketplace.json. An entry got added alongside
whichever skill prompted it and the rest were never backfilled. Nothing failed,
because nothing looked.

Guards both directions, since both are silent: a skills/<name>/SKILL.md with no
entry is invisible to users, and an entry whose directory moved or was renamed
is a broken install rather than a missing one. Also pins source == ./skills/
<name> (a mismatch installs the wrong skill under the right label, which looks
like it worked), non-empty descriptions, and name uniqueness.

Written as a test rather than a new script + CI step because npm test already
gates PRs and publish, and because brand-numbers.test.ts established the
pattern: assert the invariant against the real files in the repo that can fix
it. Nothing here is auto-fixable anyway — a marketplace description is
hand-written prose, and generating one from the frontmatter would produce worse
text than the omission it replaces.

Sibling to skill-frontmatter.test.ts, which proves a SKILL.md can LOAD. This
proves it can be REACHED. Neither implies the other. Both take "a directory
with SKILL.md at its root" as the definition of a skill — deliberately the same
definition, or one of them can pass vacuously.

Verified by breaking each invariant in turn and confirming the matching test
fails: dropped entry, entry naming a nonexistent skill, wrong source path,
empty description, duplicate name. The unregistered-skill failure names the
directories and prints the JSON to paste.

Test-only; skills/ and dist ship to npm but test/ does not, so this needs no
version bump.
@VickyXAI
VickyXAI merged commit 8a5671d into main Aug 5, 2026
1 check passed
@VickyXAI
VickyXAI deleted the test/marketplace-registration-guard branch August 5, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant