feat(resources): linkAuth validator wiring and per-host outcome codes (#113 slice 2)#125
Closed
ejdutton wants to merge 3 commits into
Closed
feat(resources): linkAuth validator wiring and per-host outcome codes (#113 slice 2)#125ejdutton wants to merge 3 commits into
ejdutton wants to merge 3 commits into
Conversation
…#113 slice 2) Wires the slice-1 pure engine into ExternalLinkValidator and adds the five LINK_AUTH_* outcome codes. End-to-end: when an adopter sets resources.linkAuth in vibe-agent-toolkit.config.yaml, vat resources validate bypasses markdown-link-check for any URL whose host is claimed by a provider, issues an authenticated fetch against the rewritten URL, and classifies the response per design §7 into one of: LINK_AUTH_DEAD (404/410 from honest-404 hosts) LINK_AUTH_DEAD_OR_UNAUTHORIZED (404 from ambiguous hosts like GitHub) LINK_AUTH_FORBIDDEN (403) LINK_AUTH_UNAUTHORIZED (401) LINK_AUTH_UNVERIFIED (no token resolved) New surface: fetchAuthenticated() with cross-origin Authorization stripping (§8, sticky across chains) and 429/Retry-After honoring (§5.2, 60s DoS cap + 250ms good-neighbor floor); pure classifier per §7; project-config → engine bridge with post-expansion validation against InlineProviderSchema (catches typo'd macro overrides); auth cache keyed by rewritten URL and scoped to a per-OS-user subdirectory (§6.3) with an explicit version field forward-compat for slice 3. 196 new tests across 6 files. Doc-anchor coverage iterator (packages/agent-schema/test/docs/validation-codes.test.ts) iterates CODE_REGISTRY and asserts each code has a matching docs section. Adopter-visible breaking changes: the LinkAuthConfig type exported from @vibe-agent-toolkit/resources renames to LinkAuthProjectConfig (resolves auto-import ambiguity with the engine type of the same name); the external-link cache layout adds an auth-${osUser}/ subdirectory and a version field that invalidates pre-existing entries on first read. Fixed: ExternalLinkValidator.clearCache()/getCacheStats() now operate on both anonymous and authenticated caches (previously the auth cache survived a manual clear, surfacing stale 401/403 entries after token rotation). Deferred to later slices: content-fetch primitive and content cache (slice 3); cross-platform .cmd-shim system test, VAT_LINKAUTH_ALLOW_COMMAND=0 opt-out, and contributor docs (slice 4). See CHANGELOG.md [Unreleased] for full details. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #125 +/- ##
==========================================
+ Coverage 81.94% 82.18% +0.24%
==========================================
Files 227 230 +3
Lines 17632 17955 +323
Branches 3468 3561 +93
==========================================
+ Hits 14449 14757 +308
- Misses 3183 3198 +15
🚀 New features to boost your workflow:
|
Owner
Code audit — 3 findings to address before mergeNice work on this slice — well-structured and well-tested, CI green. Three items from review: 1. 🔴 Postel's Law inverted: adopter config schema is
|
…nCommand, fail-soft cache IO Addresses jdutton's three review findings on PR #125: 1. Postel's Law: the adopter-facing linkAuth schemas (InlineProviderSchema, LinkAuthConfigSchema, and the five nested object schemas) were `.strict()` but parse external input. Switched all seven to `.passthrough()` to match the repo CLAUDE.md rule for adopter configs — forward-compatible / typo'd fields now degrade rather than crash `vat resources validate`. Adjusted the file header comment that misread Postel's Law, and updated four schema tests to assert the new passthrough semantics. The compile-time _KeysAgree drift check moved from `keyof z.infer<...>` to `keyof Schema.shape` so passthrough's index signature doesn't defeat it. Post-expansion validation still catches missing required fields and wrong types on declared fields (e.g. invalid notFoundMeaning enum value); typo-catching DX belongs in a separate lint pass. 2. Token memoization: ExternalLinkValidator now wraps the linkAuth deps' runCommand with a Map<JSON-argv, RunResult>, so validating N URLs from the same host runs `gh auth token` (or any command-source token) at most once per validator instance. Treats *all* token sources as potentially expensive per the review. Engine's DEFAULT_RUN_COMMAND exported as `defaultRunCommand` so the wrapper calls through to a single source of truth (avoids a duplicate-implementation jscpd clone). Two new tests pin the memoization (single provider → 1 call across N URLs; distinct providers → separate calls). 3. Fail-soft cache IO: ExternalLinkCache.loadCache() previously rethrew anything other than ENOENT/SyntaxError; saveCache() had no try/catch. An EACCES/EROFS on the cache file aborted the whole validate run. Both paths now swallow IO errors — read returns empty cache, write no-ops while the in-memory cache stays authoritative for the rest of the run. Two POSIX-skipped tests using chmod simulate EACCES on read and write. CHANGELOG updated to reflect the final post-review behavior. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4 tasks
…ats, signal, defaultSleep Covers Codecov-flagged patch lines in PR #125: - external-link-validator: catch block in validateAuthenticatedLink (Error, null/falsy, plain object, empty {}) — exercises all safeSerializeError branches - external-link-validator: clearCache() and getCacheStats() happy paths - external-link-validator: resolveOsUser() default path (no osUser option) - link-auth-fetch: signal pass-through to fetchImpl RequestInit - link-auth-fetch: defaultSleep body via fake timers (no sleep injection) - link-auth-config-build: TypeError when use value is not a string Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Collaborator
Author
|
Closing in favour of #136, which was built on top of this branch and already contains all slice 2 changes. Coverage tests have been ported to the slice 3 branch. |
|
ejdutton
added a commit
that referenced
this pull request
Jun 26, 2026
…113 slice 3) (#136) * feat(resources): linkAuth validator wiring and per-host outcome codes (#113 slice 2) Wires the slice-1 pure engine into ExternalLinkValidator and adds the five LINK_AUTH_* outcome codes. End-to-end: when an adopter sets resources.linkAuth in vibe-agent-toolkit.config.yaml, vat resources validate bypasses markdown-link-check for any URL whose host is claimed by a provider, issues an authenticated fetch against the rewritten URL, and classifies the response per design §7 into one of: LINK_AUTH_DEAD (404/410 from honest-404 hosts) LINK_AUTH_DEAD_OR_UNAUTHORIZED (404 from ambiguous hosts like GitHub) LINK_AUTH_FORBIDDEN (403) LINK_AUTH_UNAUTHORIZED (401) LINK_AUTH_UNVERIFIED (no token resolved) New surface: fetchAuthenticated() with cross-origin Authorization stripping (§8, sticky across chains) and 429/Retry-After honoring (§5.2, 60s DoS cap + 250ms good-neighbor floor); pure classifier per §7; project-config → engine bridge with post-expansion validation against InlineProviderSchema (catches typo'd macro overrides); auth cache keyed by rewritten URL and scoped to a per-OS-user subdirectory (§6.3) with an explicit version field forward-compat for slice 3. 196 new tests across 6 files. Doc-anchor coverage iterator (packages/agent-schema/test/docs/validation-codes.test.ts) iterates CODE_REGISTRY and asserts each code has a matching docs section. Adopter-visible breaking changes: the LinkAuthConfig type exported from @vibe-agent-toolkit/resources renames to LinkAuthProjectConfig (resolves auto-import ambiguity with the engine type of the same name); the external-link cache layout adds an auth-${osUser}/ subdirectory and a version field that invalidates pre-existing entries on first read. Fixed: ExternalLinkValidator.clearCache()/getCacheStats() now operate on both anonymous and authenticated caches (previously the auth cache survived a manual clear, surfacing stale 401/403 entries after token rotation). Deferred to later slices: content-fetch primitive and content cache (slice 3); cross-platform .cmd-shim system test, VAT_LINKAUTH_ALLOW_COMMAND=0 opt-out, and contributor docs (slice 4). See CHANGELOG.md [Unreleased] for full details. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(resources): address #125 review — passthrough schemas, memoize runCommand, fail-soft cache IO Addresses jdutton's three review findings on PR #125: 1. Postel's Law: the adopter-facing linkAuth schemas (InlineProviderSchema, LinkAuthConfigSchema, and the five nested object schemas) were `.strict()` but parse external input. Switched all seven to `.passthrough()` to match the repo CLAUDE.md rule for adopter configs — forward-compatible / typo'd fields now degrade rather than crash `vat resources validate`. Adjusted the file header comment that misread Postel's Law, and updated four schema tests to assert the new passthrough semantics. The compile-time _KeysAgree drift check moved from `keyof z.infer<...>` to `keyof Schema.shape` so passthrough's index signature doesn't defeat it. Post-expansion validation still catches missing required fields and wrong types on declared fields (e.g. invalid notFoundMeaning enum value); typo-catching DX belongs in a separate lint pass. 2. Token memoization: ExternalLinkValidator now wraps the linkAuth deps' runCommand with a Map<JSON-argv, RunResult>, so validating N URLs from the same host runs `gh auth token` (or any command-source token) at most once per validator instance. Treats *all* token sources as potentially expensive per the review. Engine's DEFAULT_RUN_COMMAND exported as `defaultRunCommand` so the wrapper calls through to a single source of truth (avoids a duplicate-implementation jscpd clone). Two new tests pin the memoization (single provider → 1 call across N URLs; distinct providers → separate calls). 3. Fail-soft cache IO: ExternalLinkCache.loadCache() previously rethrew anything other than ENOENT/SyntaxError; saveCache() had no try/catch. An EACCES/EROFS on the cache file aborted the whole validate run. Both paths now swallow IO errors — read returns empty cache, write no-ops while the in-memory cache stays authoritative for the rest of the run. Two POSIX-skipped tests using chmod simulate EACCES on read and write. CHANGELOG updated to reflect the final post-review behavior. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(resources): linkAuth content-fetch primitive and content cache (#113 slice 3) Implements §6.2 dual-mode headers, §6.3 content cache with 30-min TTL, and the public fetchAuthenticated primitive that wires the engine, transport, and cache together for authenticated binary content retrieval. Key changes: - rename link-auth-fetch.ts → link-auth-transport.ts; rename fetchAuthenticated → authTransport to free the name for the public primitive - engine (resolve.ts): add Provider.fetch?, dual-expand fetch.headers alongside auth.headers against the same token/capture context (§6.2) - engine: add LinkAuthConfig.cache? so cache.ttlMinutes rides the same config object without a second source of truth - content-cache.ts: per-entry two-file layout (<sha256>.json + .bin), write .bin before .json (commit-marker discipline), pickMetadata() whitelist against token persistence (§8), fail-soft IO, schema versioning - link-auth-deps-memo.ts: extract wrapLinkAuthDepsWithMemo from validator so both slice 2 and slice 3 share the same per-argv Map memoization - link-auth-content-fetch.ts: public fetchAuthenticated(); Object.hasOwn discrimination, fetch.headers merged over auth.headers, unverified/unsupported never touch cache (§6.3), cache write-through after full body read - cross-slice integration test: one adopter config + one shared memo feeds both ExternalLinkValidator and fetchAuthenticated; asserts wire-level Accept headers, token memo runs exactly once across 3 calls Bumps version to v0.1.39-rc.7. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(resources): add coverage for catch block, clearCache, getCacheStats, signal, defaultSleep Covers Codecov-flagged patch lines (ported from slice 2): - external-link-validator: catch block in validateAuthenticatedLink (Error, null/falsy, plain object, empty {}) — exercises all safeSerializeError branches - external-link-validator: clearCache() and getCacheStats() happy paths - external-link-validator: resolveOsUser() default path (no osUser option) - link-auth-transport: signal pass-through to fetchImpl RequestInit - link-auth-transport: defaultSleep body via fake timers (no sleep injection) - link-auth-config-build: TypeError when use value is not a string Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
github-actions Bot
pushed a commit
that referenced
this pull request
Jun 28, 2026
### Added
- **Dogfood eval suites for the whole `vat-development-agents` skill set, plus the fixes that dogfooding surfaced.** Every published VAT dev skill now ships a committed `vat skill test` eval suite (`evals/<skill>/`): `vat-audit`, `vat-skill-authoring`, `vat-knowledge-resources`, `vat-skill-distribution`, `vat-rag`, `vat-agent-authoring`, and `markdown-rewriting` (joining the existing `vat-skill-review` suite), wired via `skills.config.<skill>.test`. Final grades: vat-skill-distribution 25/25, vat-agent-authoring 24/24, vat-rag 22/22, vat-knowledge-resources 22/22, markdown-rewriting 18/18, vat-skill-authoring 21/22 (one capability-headroom miss), vat-audit 33/40 baseline A/B (the without-skill failures demonstrate the skill's lift on CI-gating/compat knowledge). Running the suites caught real skill/doc bugs, now fixed:
- **`markdown-rewriting` is now actually published.** It lived in the skills dir and `vat-skill-authoring` told agents to load `[[markdown-rewriting]]`, but the discovery glob (`vat-*.md`) didn't match its name, so it never shipped — a dangling skill reference. Added it to `skills.include` and `package.json` `vat.skills`; it now builds and ships.
- **`vat-skill-authoring`** gained the conservative-frontmatter-keys rule (the standard key set; stamp `version`/`team`/ownership under `metadata:` or in config.yaml, never as bare top-level keys) — the agent was inventing top-level `version:`/`team:` fields.
- **`vat-skill-review`** corrected a factual error: it claimed a `metadata:` field "will be rejected," but `metadata` is an allowed standard key (the sanctioned home for custom data per `SKILL_FRONTMATTER_EXTRA_FIELDS`).
- **`vat-rag`** removed a nonexistent `vat rag index --rebuild` flag (the real reset is `vat rag clear`; indexing is incremental) and added the missing `OnnxEmbeddingProvider` to the providers table.
- **`vat-knowledge-resources`** now states that `strict` mode only rejects extra fields when the schema sets `"additionalProperties": false`, and that collection validation defaults to `permissive`.
- **Collection-validation docs** corrected: `mode` defaults to `permissive` (matching `validateAgainstCollectionSchema`), not `strict` as previously documented.
- **Skill-test harness:** `buildForwardedEnv` now forwards `USER`/`LOGNAME` (see below) and eval fixtures (including intentionally-broken `.ts` files) are excluded from ESLint.
- **`vat skill test run` / `vat skill test configure` — behavioral skill testing in a context-isolated harness (#132).** Stage a packaged skill plus its declared dependencies into a throwaway, locked-down harness and run a canned, non-interactive evaluation that grades the skill against your `evals.json` (reusing skill-creator's grading rubric and JSON shapes) and writes `grading.json` (with a published [JSON Schema](docs/skill-test-grading-schema.md)), `friction.json`, and full transcripts you can inspect. `configure` writes a per-skill `test:` block to your config as a surgical edit — only the keys you pass change; surrounding formatting and comments are byte-preserved; a first `run` with no `evals.json` writes a template for you to fill in. Runs end-to-end against `claude` 2.x. **Security:** the harness runs the skill's own code with your account's privileges — it is *context* isolation, not an OS sandbox — so `run` requires `--i-understand-this-runs-skill-code`, enforced *before* anything runs (including the optional pre-stage build), and you should only test skills you trust. The pass/fail verdict is recomputed from the graded expectations, so a failing or empty grade is never silently reported as a pass; add `--fail-on-eval-failure` to make a failing eval exit non-zero and gate CI on it. See the new `vibe-agent-toolkit:vat-skill-testing` skill for auth modes, budget/turn/timeout caps, `--baseline` A/B runs, and exit codes.
- **Pre-stage `build:` hook + plugin-root staging.** An optional `test.build` command runs once before staging, so a skill that depends on a generated, un-committed artifact has it present (a non-zero build fails fast at preflight, before any tokens are spent). Plugin-distributed skills stage under their real plugin-root layout with `CLAUDE_PLUGIN_ROOT` set; standalone skills stage flat.
- **Declared test-env passthrough.** `passEnv` / `--pass-env` forwards host variables; `env` / `--env` injects values with `${fixturesDir}` / `${stagedSkillDir}` / `${harnessRoot}` / `${resultsDir}` interpolation. Both apply *after* the security allowlist — protected names always win, so committed test config can neither reroute your account credentials nor inject code: auth credentials, `PATH`, and credential-routing variables (`ANTHROPIC_BASE_URL` and the other endpoint/proxy overrides, `NODE_OPTIONS`, `NODE_EXTRA_CA_CERTS`) cannot be overridden. Fixtures under the skill's `evals/fixtures/` auto-stage with the eval tree.
- **Project-aware subject resolution.** Name a skill declared in `vibe-agent-toolkit.config.yaml` and `run` builds it first and tests the shipping **dist** — link-following, reference-rewriting, nav-stripping, and `files:` injection all applied — so you exercise exactly what installs, not the source tree. A path (including an already-built dist dir), or a `workspace:` / `npm:` / `url:` / `path:` / `vendored` source, is tested as-is; use `./<name>` to force a local directory over a colliding declared name. `--no-build` stages an existing dist without rebuilding (and errors if it is absent); `--dry-run` assembles the command without building and flags when the previewed dist may be stale, and — when no `evals.json` exists yet — reports where a real run *would* scaffold the template (exit 3) instead of writing it, so a dry run never touches your tree. A build failure fails fast at preflight (exit 2), before any tokens are spent.
- **Eval `files` are now provisioned.** Each eval's declared input files are staged into a per-eval working directory the executor operates on, enabling realistic "drop the agent in a project" evals. Files resolve relative to the `evals.json` directory and are materialized under `<harnessRoot>/workspaces/<id>/`; the experimenter prompt hands the executor that directory via a new `{{WORKSPACES_ROOT}}` token. A declared-but-missing input file fails fast at preflight (exit 2). Previously `files` was documented but inert.
- **Merge-readiness: liberal eval-suite schema, macOS subscription-auth fix, expanded skill, first dogfood suite.** (1) `evals.json` is adopter-authored data VAT *reads*, so its schema is now liberal per VAT's Postel's Law: `EvalSuiteSchema`/`EvalEntrySchema` are `.passthrough()` and `id` accepts a descriptive **string** or an int — only the fields VAT consumes (`prompt`, `expected_output`, `expectations`) stay required. This reverses the earlier strict-parser call that rejected real adopter suites three ways (string `id`, `category`, `_category_note`) and restores compatibility for the flagship adopter (app-platform/dxa). The persisted `test:` *config* block stays **strict** (it's VAT-produced config) — the deliberate inverse. (2) **macOS subscription-auth fix:** the harness env allowlist (`buildForwardedEnv`) dropped the POSIX `USER`/`LOGNAME` vars, so on macOS `claude auth status` could not read the login Keychain with the API key scrubbed — `--auth subscription` (and `inherit`'s subscription fallback) wrongly failed preflight, and the experimenter child could not authenticate. `USER`/`LOGNAME` are now forwarded (non-secret; already derivable from the forwarded `HOME`). (3) The `vibe-agent-toolkit:vat-skill-testing` skill gains a research-grounded "Authoring `evals.json`" section (blind realistic prompts, discriminating + negative expectations, categories, fixtures, `--baseline` skill-lift, grading) and a full flag⇄config knob table. (4) Ships the first committed VAT dogfood suite (`vat-skill-review`, 5 evals across catch-violation / no-false-positive / guidance-correctness) wired via `skills.config.vat-skill-review.test`, with eval fixtures excluded from `vat resources validate`.
- **`files:` entries now support glob sources and an optional `integrity` byte-verify.** A `source` containing glob magic (`*`, `**`, `?`, `[`) fans out into a directory `dest`, preserving the directory structure below the static base (glob is VAT's existing idiom, as in `skills.include` — no `recursive` flag). Globbed dests are late-bound, so `SKILL.md` links into them are treated as deferred artifacts at validate time (no `LINK_TO_GITIGNORED_FILE` allowlist needed). Add `integrity: true` to byte-verify the copy at build time and assert an exact dest subtree for glob entries.
- **`NON_PORTABLE_ASSET_REFERENCE` validation code (default `warning`) — a portability check family.** `vat skills validate` / `vat audit` now flag a skill document that references a bundled script/asset via a non-portable anchor, scanning the `SKILL.md` body **and every reachable bundled markdown doc** (agents copy invocations from reference files too). It's a family of sub-checks under one code — `claude-plugin-root`, `claude-project-dir`, and `absolute-script-path` — each finding names the variant and carries a tailored fix, and a single `validation.allow` entry silences the whole family for a file. These anchors don't exist when a skill is mounted standalone (claude.ai upload, API container), so the path breaks on the agent's first invocation; reference bundled files relative to the skill directory instead. See [`NON_PORTABLE_ASSET_REFERENCE`](docs/validation-codes.md#non_portable_asset_reference).
- **Skill-authoring guidance: portable bundled-script paths.** The `vibe-agent-toolkit:vat-skill-authoring` skill now documents how to reference bundled scripts/assets portably (relative to the skill directory, never `CLAUDE_PLUGIN_ROOT`/absolute/env-var anchors), and `vibe-agent-toolkit:vat-skill-review` carries the matching pre-publication checklist item.
- **Skill-review guidance: reserved words `claude`/`anthropic` in skill names.** The `vibe-agent-toolkit:vat-skill-review` skill's Naming section now carries the reserved-word rule as a canonical `[A]` item — Anthropic's authoring guidance states a skill `name` "Cannot contain reserved words: 'anthropic', 'claude'", and Claude Code refuses to load a non-certified skill named that way, so it fails at install/validation, not just review (`[RESERVED_WORD_IN_NAME]`). Surfaced by dogfooding the skill against its own eval suite (the reviewer was noting the prefix as "redundant" but missing the install-blocking consequence). The rule directs the reviewer to surface that consequence when reviewing such a name and to include the warning when advising on naming.
- **`NON_PORTABLE_COMMAND` validation code (default `warning`) — a portability check family.** `vat skills validate` / `vat audit` now flag a skill document that tells an agent to run a GNU/Linux-only shell command, scanning the `SKILL.md` body **and every reachable bundled markdown doc** (agents copy invocations from reference files too). It's a family of sub-checks under one code — `timeout`, `grep-pcre` (`grep -P`), `sed-i-no-backup` (`sed -i` with no suffix), `readlink-f`, and `date-d` (GNU `date -d`) — each finding names the variant and carries a tailored fix, and a single `validation.allow` entry silences the whole family for a file. Patterns match commands in command position only (not bare prose), so `grep -E`/`sed -i.bak` and nouns like "the request will timeout" are not flagged. Promotes a former manual `vat skill review` checklist line into an automated check. See [`NON_PORTABLE_COMMAND`](docs/validation-codes.md#non_portable_command).
- **linkAuth content-fetch primitive + content cache (issue #113, slice 3).** Ships the public `fetchAuthenticated(url, config, options) → { bytes, metadata, cached } | { outcome: 'unsupported' | 'unverified' }` primitive (`packages/resources/src/link-auth-content-fetch.ts`), per design §6.2 — *sibling to* the slice-2 health-check path, both reading from the same engine config and rewrite pipeline. No consumer wiring (asset-references, bundling) lands in this slice; the primitive ships standalone so future callers can adopt it without reworking the contract. **Two-mode headers (§6.2):** `Provider` gains an optional `fetch: { headers }` block alongside `auth: { headers }`; `resolveAuthenticatedUrl` now dual-expands both header sets against the same context (URL captures + resolved token), surfacing them on the success outcome as `headers` (auth, for health-check) and `fetchHeaders` (fetch, for content retrieval). The primitive merges `fetchHeaders` over `headers` so fetch-mode overrides on conflict — the canonical case being GitHub, where `Accept: application/vnd.github+json` returns 200 for any size but omits bytes >1 MiB (good for health-check) while `Accept: application/vnd.github.raw` streams the bytes inline (required for content). Adopter schema (`InlineProviderSchema`) gains a parallel `ProviderFetchSchema` (passthrough, like the rest of the adopter linkAuth tree per the repo Postel's Law rule), and the compile-time `_KeysAgree` drift check picks up the new field automatically. The resolved-token-wins precedence (URL-capture-named `token` cannot beat the resolved value) and the null-prototype hardening apply to `fetchHeaders` too. **`ContentCache` (§6.3, `packages/resources/src/content-cache.ts`):** new persistence class for the content-fetch primitive — distinct from slice 2's `ExternalLinkCache` (which is a status cache). Per-entry layout: `<sha256(rewrittenUrl)>.json` (metadata: status, content-type, etag, last-modified, fetchedAt, rewrittenUrl, `version: 1`) + `<sha256(rewrittenUrl)>.bin` (raw bytes), under a caller-supplied `cacheDir` (the validator-style `<cacheDir>/content/auth-${osUser}/` scoping is the caller's responsibility — the class only knows about the directory it was given, mirroring §6.3's "cross-user isolation = OS user, not cache key"). 30-minute default TTL (`§6.3`), tunable via constructor and via the threaded-through `resources.linkAuth.cache.ttlMinutes` adopter config (`buildLinkAuthEngineConfig` now copies the adopter `cache` block onto the engine `LinkAuthConfig`, which previously dropped it silently). Write order is `.bin` first, then `.json` as the commit marker — a partial-write crash leaves either no entry or `.bin` ahead of `.json` (reads as a miss), never `.json` ahead of `.bin` (which would serve stale bytes under new metadata). On-disk metadata fields are whitelisted via a single `pickMetadata()` helper used by both `set()` (strip smuggled fields before write) and `get()` (strip the on-disk `version` before return), so token-bearing fields a caller might smuggle through structural typing cannot land on disk — defense in depth on top of the closed `ContentMetadata` interface. TTL boundary is `>`, not `>=` — entries are valid at exactly the TTL, expire at TTL + 1 ms; tests pin both boundary cases. Fail-soft IO per #125 review: `EACCES` / `EROFS` / corrupted JSON degrade to a miss (read) or no-op (write), never throw. Forward-compat `version: 1` mirrors `ExternalLinkCache`. **Primitive behavior:** the four outcome branches — `unsupported` and `unverified` short-circuit with no fetch and **no cache touch** (§6.3: never cache `unverified`, since the result flips the moment a token appears); cache-hit returns `{ bytes, metadata, cached: true }` with no fetch; otherwise fetch via `authTransport` (cross-origin auth strip + 429 retry inherited from slice 2), read the body binary-clean via `arrayBuffer()`, build metadata from response headers (content-type / etag / last-modified default to `null` when absent), write through to the cache if supplied. `forceRefresh: true` bypasses cache reads but still writes through. `AbortSignal` propagates to the transport. The token value is interpolated into request headers in-memory and never flows into `ContentMetadata`; an end-to-end test reads every file in the cache directory after a fetch and asserts the literal token string is absent. **`wrapLinkAuthDepsWithMemo` lifted to its own module** (`packages/resources/src/link-auth-deps-memo.ts`) and exported from the resources barrel — slice 2 originally housed it private inside `external-link-validator.ts`, but the standalone primitive needs the same memoization, and the lift centralizes the implementation so jscpd cannot flag a clone. Validators and primitive callers iterating many URLs from the same provider wrap their `deps` once and reuse, so `gh auth token` / any `command`-source resolver runs at most once across the iteration. **File rename for slice 2's transport:** `packages/resources/src/link-auth-fetch.ts` → `link-auth-transport.ts`, and the exported function `fetchAuthenticated` → `authTransport` (with `AuthFetchOptions` → `AuthTransportOptions`) — frees the `fetchAuthenticated` name for the spec-documented primitive and aligns the filename with the symbol's role as the lower-level auth-safe HTTP wrapper. **47 new tests:** `link-auth-content-fetch` (15 covering short-circuits, header merge with fetch.headers override, cache hit/miss/forceRefresh, unverified-never-cached, binary-clean round-trip, signal pass-through, token-never-persisted), `content-cache` (14 covering round-trip, binary safety, distinct-URL isolation, overwrite, TTL boundary at `=` and `=+1ms`, version-mismatch eviction, corrupted-JSON tolerance, POSIX-skipped `EACCES` fail-soft on read and write, and the whitelist-on-write check), `link-auth-deps-memo` (5 covering single-source memo, distinct-argv independence, default-runCommand fallback, deps pass-through, undefined-deps handling), and 13 augmenting tests on the slice 2 surface for the new `provider.fetch` block (engine dual-expansion, schema acceptance/rejection, cache field propagation). The slice 3 primitive does not wire into any existing CLI command — `--refresh` / `--no-cache` ships with the first consumer slice.
- **linkAuth validator wiring + per-host outcome codes (issue #113, slice 2).** The slice-1 pure engine is now end-to-end: when an adopter sets `resources.linkAuth` in `vibe-agent-toolkit.config.yaml`, `vat resources validate` bypasses the anonymous `markdown-link-check` path for any URL whose host is claimed by a provider, issues an authenticated `fetch()` against the rewritten URL with the configured token, and classifies the response per design §7 into one of five new `CODE_REGISTRY` entries: `LINK_AUTH_DEAD` (404/410 from an honest-404 host — `error`-severity, the only such code in the slice; design §7 establishes that an authenticated 404 against e.g. SharePoint is high-confidence link rot, satisfying the rule-design corpus-evidence bar), `LINK_AUTH_DEAD_OR_UNAUTHORIZED` (404 from an ambiguous host like GitHub that masks `403`s — warn), `LINK_AUTH_FORBIDDEN` (`403` — warn), `LINK_AUTH_UNAUTHORIZED` (`401` — warn, promote to `error` on strict CI lanes), and `LINK_AUTH_UNVERIFIED` (no token resolved — warn, never cached per §6.3). New files: `packages/resources/src/link-auth-fetch.ts` (`fetchAuthenticated()` — bounded redirect loop with **cross-origin `Authorization` stripping (§8)** that is sticky across the rest of the chain to defeat token-laundering, **429/`Retry-After` honoring (§5.2)** parsing both delta-seconds and HTTP-date forms with a 60s DoS cap *and* a 250ms good-neighbor floor, all dependency-injectable via `fetchImpl`/`sleep`/`signal`); `packages/resources/src/link-auth-classify.ts` (pure `(status, providerCheck) → outcome+code` per §7's table); `packages/resources/src/link-auth-config-build.ts` (bridge from adopter config to engine — runs `expandMacro` on `{ use: <macro>, ...overrides }` entries; the adopter schemas are passthrough per the repo's Postel's Law rule, so the post-expansion `InlineProviderSchema.safeParse` catches missing required fields and wrong types on declared fields but lets unknown extras through, matching how the rest of project-config treats adopter input; a compile-time `_KeysAgree` assertion locks the schema's top-level field set to the engine's `Provider` interface). New cache architecture: a second `ExternalLinkCache` instance for auth-branch results, **keyed by the rewritten URL** (the original `blob/` URL 404s — caching it would poison results) and scoped to a per-OS-user subdirectory `cacheDir/auth-${sanitizedOsUser}/` so two users on a shared CI host never read each other's authenticated results (§6.3); the OS user resolves through `os.userInfo()` → `USER`/`USERNAME` env → `'default'` with a one-shot `console.warn` on the last fallback so the cross-user-leak risk is observable. Cache entries gain an explicit `version: 1` field; reads of any other version produce a miss, so slice 3's content-cache evolution can change the entry shape without misparsing pre-existing files. Doc-anchor coverage iterator test (`packages/agent-schema/test/docs/validation-codes.test.ts`) iterates `CODE_REGISTRY` and asserts each code has a matching `### \`CODE\`` heading in `docs/validation-codes.md` plus a convention-matching `entry.reference` — 126 assertions covering all 63 codes, future-proofs the per-code docs requirement. Five new doc sections under "Authenticated External Link Codes" in `docs/validation-codes.md`. Engine surface gains: `Provider.check` now flows through on the verified `ResolveOutcome` so the classifier can route per-provider `notFoundMeaning`; `ExternalLinkValidatorOptions` gains `linkAuthConfig`, `fetchImpl`, `linkAuthDeps`, `sleep`, and `osUser` (the first two are adopter-usable for corporate-proxy/custom-TLS injection, the rest test-only); `LinkValidationResult` gains a `code?: IssueCode` field that the `resource-registry.ts` consumer prefers over the existing status-code-to-`EXTERNAL_URL_*` mapping. The validator memoizes `runCommand` results per unique argv for the duration of a `validate()` run, so validating N URLs from the same host runs `gh auth token` (and any other command-source token) at most once. **196 new tests across `link-auth-classify` (13), `link-auth-fetch` (19), `external-link-validator-auth` (25), `link-auth-config-build` (10), `validation-codes` (+126 doc-anchor iterator, +2 LINK_AUTH_* registry), and `external-link-cache` (+1 version-gate).** Security-load-bearing tests pin the cross-origin Authorization strip (case-insensitive, sticky across chains, with userinfo/relative-Location edge cases), the path-traversal sanitizer (table-driven over 9 pathological `osUser` inputs), the unverified-no-cache invariant, the cache-hit re-classification (cache hits re-run the classifier against the current provider so a `notFoundMeaning` flip between runs surfaces the new code, not the old one), the runCommand memoization (N URLs from the same provider → 1 command invocation), and an Object.hasOwn-based prototype-pollution defense on the `{ use }` discriminator in `buildLinkAuthEngineConfig`. Slice 4 (cross-platform `.cmd`-shim system test, `VAT_LINKAUTH_ALLOW_COMMAND=0` opt-out, contributor docs) and slice 3 (content-fetch primitive + content cache) are downstream.
- **linkAuth pure engine foundation (issue #113, slice 1).** Adds a config-driven engine for authenticated external URL resolution, scoped to the pure-logic layer with no consumer wiring yet (the `ExternalLinkValidator` integration and the `LINK_AUTH_*` `CODE_REGISTRY` entries are slice 2; the content-fetch primitive is slice 3). New `link-auth/` module under `@vibe-agent-toolkit/utils` with eight files: `transforms.ts` (closed allowlist — `base64url`, `urlencode`, `lower` — with `Object.hasOwn`-based prototype-chain defense), `template.ts` (tiny `${name}` / `${transform(name)}` renderer, deliberately separate from the Handlebars renderer in `utils/template.ts`), `rewrite.ts` (ordered `when → vars → to` pipeline with fragment/query stripping per design §5.2), `build-headers.ts` (header rendering plus structural `Authorization` redaction), `select-provider.ts` (host-glob matching via picomatch with `excludeHost`), `expand-macro.ts` (YAML loader + deep-merge expander), `resolve-token.ts` (ordered env / `safeExecResult`-backed argv-command sources, first-non-empty wins, no shell), and `resolve.ts` (the public `resolveAuthenticatedUrl(url, config)` entry returning one of `{fetchUrl, headers}` / `{outcome: 'unsupported'}` / `{outcome: 'unverified', reason}`). Ships the `github` and `sharepoint` macros as a YAML data asset (`src/link-auth/macros.yaml`), with a new cross-platform `packages/dev-tools/src/copy-yaml-assets.ts` post-build step bundling `.yaml` into `dist/` — first YAML-asset shipping pattern in the utils package. Adds `yaml` as a utils dependency. Companion Zod schema in `@vibe-agent-toolkit/resources` (`src/schemas/link-auth.ts`) validates the `resources.linkAuth` config block (strict; accepts either `{ use: <macro>, ...overrides }` or full inline providers), wired as an optional field on `ResourcesConfigSchema`. 140 unit tests in utils + 29 schema tests in resources, all pure-logic with no network or filesystem dependencies; security-load-bearing tests pin the closed-allowlist guarantee, the `${__proto__}` bypass defense, the token-never-leaks-into-Authorization invariant, and `shell: false` literal-argv handling.
- **Corpus seed expanded from 9 → 237 entries via a new committed importer at `packages/dev-tools/src/import-marketplace.ts` (`bun run import-marketplace [--allow-shrink]`).** The script fetches `.claude-plugin/marketplace.json` from `anthropics/claude-plugins-official` (205 of 209 raw entries kept) and `anthropics/knowledge-work-plugins` (30 of 60 — the knowledge-work catalog turns out to be ≈50% mirror entries of the official catalog) via `gh api`, maps each upstream entry to a `PluginEntry`, deduplicates by `source` URL (preserved VAT-owned entries always win; otherwise alphabetical-first-name wins within each duplicate cluster), and rewrites `corpus/seed.yaml`. Mapping rules: `bucket: official` uniformly (both catalogs are anthropics-curated marketplaces — `bucket` is the *reporting posture* per slice 1a, not code provenance); `confidence: first-party` for catalog-internal string sources and `github.com/anthropics/...` object sources, else `curated`; the `./partner-built/` knowledge-work convention overrides to `curated`; `maturity: production` for all entries. URL composition handles all five upstream source shapes (string, `git-subdir` ± `ref`, `url` ± `path`, `github`), throwing on unknown discriminators. The seven sample entries from slice 1a are regenerated from upstream manifests on every re-import. Re-import safety: the importer refuses to overwrite `corpus/seed.yaml` if either upstream catalog returned 0 plugins or the new entry count would drop more than 20% vs. the existing seed; `--allow-shrink` bypasses both gates for the rare case where shrinkage is real. The generated `seed.yaml` header dropped its earlier per-entry `validation:` claim (the importer throws on validation blocks today) and now states explicitly that entry `source` URLs pin a fragment ref (typically the default branch), not a per-entry commit SHA — the catalog SHAs in the header are this run's audit provenance. Issue #99 slice 1b — follows the schema change from PR #111 (slice 1a).
- **Empirical compatibility harness (`packages/dev-tools/src/compat-empirical/`).** Per-#100 research scaffold for measuring skill compatibility across `claude-code`, `claude-cowork`, and `claude-chat`: a CLI (`predict`/`run`/`judge`/`report`/`all`) that joins VAT's static predictions with deterministic runtime observations and an LLM-judge semantic read into a reality-vs-prediction matrix — an evidence artifact for proposing detector improvements that each cite specific (skill, runtime) cells. Probe coverage: multi-prompt + repeat-N with adaptive N=3→N=5 extension, mandatory positive+negative prompt pairing per corpus entry, and negative-prompt agreement inversion so false-positive triggers surface as `vat-optimistic`. Evidence quality: the deterministic class is widened from 6 to 9 values (splitting `error` into `install-failed`/`runtime-error`, `not-invoked` into `not-invoked-engaged`/`not-invoked-empty`, adding `refused`), with a v2 judge prompt that adds a `refused` verdict. Report fidelity: coverage stats, per-bucket headline (own/official/community × ran/agree/optimistic/pessimistic/gray-zone), gray-zone (mixed-signal) and high-variance subsections, and per-attempt variance rendered inline (`runtime-error (2/3) / failed (3/3)`). Judge replay persists `judge-calls/<skillId>-<promptId>-<target>-<attemptIdx>.json` artifacts that a new `re-judge` subcommand re-executes against an optionally different model or freshly-edited system prompt — without re-spending operator hours on the runtime side. Also landed: `git fetch --tags --force` before named-ref fetch (annotated tag refresh) and `setup()` teardown-first idempotency for the manual driver. No detector code or `RUNTIME_PROFILES` changes; lives entirely in the private `@vibe-agent-toolkit/dev-tools` package with no adopter-facing surface. Design: [the v2 harness design](./docs/research/2026-05-23-compat-empirical-harness-v2-design.md). Corpus authoring, the first real run, and the docs deliverable are the downstream work.
- **Cowork driver spike.** Added [`docs/contributing/cowork-driver-spike.md`](docs/contributing/cowork-driver-spike.md) — a time-boxed investigation (per §4a of the harness v2 design) of whether `claude-cowork` can be driven programmatically by the empirical compat harness today. Verdict: **not feasible**; cowork is a Claude Desktop app product with no public API/CLI surface. The `claude-cowork` runtime stays on `scripted-assisted` until Anthropic ships a Cowork CLI mode, Sessions API, or documented filesystem-import path. Adjacent finding (not a cowork replacement): the public-beta Skills API (`POST /v1/skills` + `container.skills[]` on `/v1/messages`) supports a fully-automatable *new* runtime — captured in the spike doc as a potential follow-up, gated on a separate design decision.
- **Subscription-only compat harness billing.** The harness now bills a Claude Pro/Max subscription instead of the API: both token-consuming surfaces (the `claude-code` runtime driver and the LLM judge) route through one shared `claude` CLI invoker (`runtimes/shared/claude-cli.ts`) that injects the operator's `CLAUDE_CODE_OAUTH_TOKEN` and deletes every API credential from the child env, so the CLI cannot fall back to API billing. The operator's own token is sourced at preflight — env var if set, otherwise an interactive prompt — so a run only ever spends the operator's personal plan. The judge was migrated off `@anthropic-ai/sdk` (dependency removed) onto the CLI, parsing a strict JSON verdict with one retry instead of the SDK's forced-tool call (`judge-system.md` now asks for a JSON object). `RunMetadata` gains `authMode` and the report methodology discloses subscription auth + parsed-not-forced verdicts. Premise (zero API billing under the OAuth token) still pending the manual smoke test.
- **First-class local HTML resources (#112).** `.html`/`.htm` files are now discovered, parsed, link- and anchor-validated, checked for well-formedness, and link-rewritten on bundle — using the same `ParseResult` contract and validation framework as markdown. A parse5-backed parser extracts `<a href>` and `<img src>` links plus `id`/`name` fragment anchors; `ResourceRegistry` routes HTML through it and persists optional `anchors`/`parseErrors` on `ResourceMetadata`. Anchor validation now uses a format-neutral fragment index (each file's markdown heading slugs or HTML `id`/`name`, with its case-matching policy carried per entry), enabling cross-format anchor checks (md↔html) with HTML ids matched case-sensitively and markdown slugs case-insensitively. A new `MALFORMED_HTML` code (default `info`) surfaces parser well-formedness diagnostics. On bundle, `<a href>`/`<img src>` values are rewritten by offset-splicing the original source (never re-serialized), so unchanged markup round-trips byte-for-byte and original attribute quoting is preserved (a rewritten value that would be unsafe unquoted is wrapped in quotes). Scope is `<a href>` + `<img src>` only; `<link>`/`<script>`/`<iframe>`/`<source srcset>`/CSS `url(...)` are deferred (asset/machinery references, not the content link graph). `<base href>` is not honored — relative hrefs resolve against the file's own directory (see the breaking note below for the `ResourceMetadataSchema` tightening that shipped with this work).
- **`DUPLICATE_RESOURCE_ID` validation code (default `error`).** When two files resolve to the same resource id after path normalization (e.g. `My Guide.md` and `my-guide.md` both → `my-guide-md`), `vat resources validate` now reports it as an `error` issue naming both files, instead of aborting the entire run with an uncaught `Duplicate resource ID` exception. Documented under [Resource Registry Codes](./docs/validation-codes.md).
- **Live audit/validate now sees source HTML links (issue #129 AC2).** `vat audit` / `vat skills validate` previously crawled `**/*.md` only, so links inside source `.html`/`.htm` files were invisible until build time. The live crawl now includes HTML (the registry already parses it via parse5), so the link-graph walker traverses HTML references and a broken local link inside a source HTML file surfaces as `LINK_MISSING_TARGET` at validate time, at parity with the built path's `PACKAGED_BROKEN_LINK`.
- **`LINK_DEFERRED_ARTIFACT` info code (issue #127, slice 2 of #129).** A `SKILL.md` link to a `files:`-declared artifact that doesn't exist yet (a dest built later, or a not-yet-created source) is no longer reported as a broken link — it downgrades from `LINK_MISSING_TARGET` to the new [`LINK_DEFERRED_ARTIFACT`](docs/validation-codes.md#link_deferred_artifact) info code at validate time, and `vat skills build` preserves and rewrites the link to the materialized dest instead of stripping it.
### Changed (breaking, pre-1.0)
- **`computeDeferredPaths` return type changed (issue #127, slice 2 of #129).** `computeDeferredPaths(files)` now returns `{ destPaths, sourcePaths }` instead of a flat `Set<string>` — a breaking API change (pre-1.0, intentional). Both `vat skills validate` and `vat skills build` now consume the deferred-path set (previously `deferredAssets` was silently dropped), and deferred dest/source paths resolve project-root-relative so the new behavior works for skills in subdirectories, not only at the project root. Plugin-local `files:` deferred paths remain out of scope for this slice (see [AC-10d](docs/architecture/skill-packaging.md#ac-10d--plugin-local-files-deferred-paths-are-out-of-scope-for-issue-127--slice-2-of-129)).
- **Directory links are now valid targets; `LINK_TARGETS_DIRECTORY` is narrowed to typed single-file slots (issue #126, slice 1 of #129).** A navigational local link that resolves to an existing directory (e.g. `[docs/](docs/)` in a ToC, README, or SKILL.md body) is no longer an error in `vat resources validate` or the skill-bundling link walk — previously any local link to a directory was a hard error. A renamed/deleted directory still fails via the ordinary broken-link path. `LINK_TARGETS_DIRECTORY` (still `error`) now fires **only** for a packaging `files:` *source* entry that resolves to a directory (the contract demands exactly one file). GitHub-style directory-index resolution (`docs/` → `docs/README.md`) is intentionally not implemented. Known limit (tracked for #129): a no-slash link such as `[Concepts](concepts)` that resolves to a directory is still treated as a file link; the slash form is the navigational case this slice covers.
- **`ResourceMetadataSchema` is now `strict()`.** Shipped with first-class HTML support (#112): the resource-metadata schema rejects unknown top-level fields instead of silently accepting them, so a typo or stale field in code that constructs `ResourceMetadata` now fails at parse time rather than passing through. Move any extra data into a recognized field or drop it.
- **Resource ids now carry a file-extension suffix.** `generateIdFromPath` appends `-<ext>` to every resource id (e.g. `guide.md` → `guide-md`, `guide.html` → `guide-html`, `README.md` → `readme-md`). This makes a markdown file and a same-stem HTML file distinct resources instead of colliding — the prerequisite for first-class HTML resources sharing a directory with their markdown source. Resource ids are internal, path-derived identifiers (never hand-authored in config or frontmatter), but anything that referenced an id by its old bare form must use the suffixed form — most visibly `vat rag query --resource-id` filters and re-indexed chunk ids (re-index to regenerate).
- **`vat resources validate` gains per-code severity configuration, and external-URL findings no longer fail the build by default.** Resource findings now use the same configurable severity framework as `vat skills`: each is a documented code (e.g. `LINK_BROKEN_FILE`, `EXTERNAL_URL_DEAD`) with a default severity, overridable per project under `resources.validation.severity` / `resources.validation.allow`. External-URL findings now default to `warning` and no longer flip the exit code (fixing a bug where they always failed the command); set their severity to `error` to restore failing. Severity now also accepts an `info` level. The never-implemented `resources.validation.checkLinks`/`checkAnchors`/`allowExternal` keys are removed.
- **`validation.severity` / `validation.allow` keys are validated against real codes.** A mistyped code key (e.g. `LNIK_OUTSIDE_PROJECT`) is now a config-load error instead of a silent no-op.
- **Corpus seed entries now require `bucket`, `confidence`, and `maturity` metadata fields.** `PluginEntrySchema` in `vat corpus scan`'s seed loader gains three required enum fields: `bucket: 'official' | 'community'`, `confidence: 'first-party' | 'curated' | 'listed'`, and `maturity: 'production' | 'experimental' | 'example'`. The bundled `corpus/seed.yaml` is updated; downstream callers running custom seeds must add the fields to every entry. `bucket` is the load-bearing discriminator (`official` entries report named findings; `community` entries are aggregate-only in follow-up work). The other two are descriptive metadata used by triage tooling.
- **`vat claude marketplace publish` no longer reports the project root `package.json` version in the CLI banner, commit message, status YAML, or CHANGELOG section lookup.** The label is now derived from the staged `marketplace.json`. Single-plugin marketplaces use the plugin's version — banner reads `Publishing marketplace "X" v0.0.4`, commit subject reads `publish v0.0.4`. Multi-plugin marketplaces drop the `v<X>` entirely — banner reads `Publishing marketplace "X"`, commit subject reads `publish X` — since the per-plugin `version` fields in the published `marketplace.json` are the source of truth for which plugin moved to which version. Two visible side-effects follow: (1) the status YAML's `published[*].version` field is now absent for multi-plugin marketplaces (previously it carried the misleading project version) — automation should read per-plugin versions from the published `marketplace.json` instead; (2) the stamped `## [X.Y.Z]` CHANGELOG lookup now uses the plugin's version rather than the project's, so a previously-ignored matching section will now be picked up as the commit body for single-plugin marketplaces. The `marketplace.json` schema's optional top-level `version` field is not yet consumed — that is a separate follow-up.
- **Adopter-facing `LinkAuthConfig` type renamed to `LinkAuthProjectConfig` (issue #113).** Both `@vibe-agent-toolkit/utils` (engine) and `@vibe-agent-toolkit/resources` (Zod-inferred adopter shape) previously exported a type named `LinkAuthConfig`, causing IDE auto-import ambiguity in any code that touched both. The adopter type — accessible as `import type { LinkAuthProjectConfig } from '@vibe-agent-toolkit/resources/schemas/link-auth'` — is the one renamed; the engine's `LinkAuthConfig` is unchanged (more API surface depends on it). Migration: rename the import. The Zod schema's name (`LinkAuthConfigSchema`) is unchanged.
- **External-link cache directory layout adds an `auth-${osUser}/` subdirectory and an entry `version: 1` field (issue #113 §6.3).** When `vat resources validate` runs with `resources.linkAuth` configured, authenticated-fetch results land under `<cacheDir>/auth-${sanitizedOsUser}/external-links.json` rather than the shared `external-links.json` used by the anonymous `markdown-link-check` path — two users on the same host (e.g. shared CI runners) cannot read each other's authenticated cache entries. All cache entries now carry an explicit `version: 1` field; entries written under a different (or missing) version are treated as a cache miss, so any pre-existing `external-links.json` triggers a one-time re-fetch on first run after upgrade. The `version` gate is forward-compat for slice 3's content-cache shape evolution.
- **`vat claude marketplace publish` no longer pushes per-plugin `<name>-v<version>` source-repo tags.** The post-publish tagging step (introduced alongside multi-plugin versioning) is removed entirely — no tags are created or pushed, and the misleading `Repository not found` / "tag already exists at a different commit" warnings it emitted on every cross-repo publish are gone ([#121](https://github.com/jdutton/vibe-agent-toolkit/issues/121)). The tags were pushed to the marketplace remote rather than a source remote, never landed anywhere useful, and there was no opt-in demand. Which plugin moved to which version is now determined solely by the per-plugin `version` fields in the published `marketplace.json`. No config key or flag is involved; if you relied on these tags, create them in your own release workflow.
### Fixed
- **Skill-test eval-suite schema hardened after an adversarial review of the Postel liberalization.** Four issues the `id`/passthrough widening introduced or left open, all verified against the real `dxa` adopter suites in `app-platform`:
- **String eval ids are now validated as filesystem-safe path segments** (`[A-Za-z0-9_-]+`). A string `id` names a per-eval working directory, and the experimenter substitutes it verbatim into `<workspaces>/<id>`; an id like `year:extraction` previously passed parse, then failed on Windows (illegal filename) — surfacing as a *misleading* "escapes the eval directory" copy error. Rejected at parse with a clear message instead. dxa's hyphenated ids (`dollar-quote-recovery`) are unaffected.
- **Numeric `1` and string `"1"` no longer slip past the uniqueness check.** Ids are deduped on their stringified form, since both name the same workspace directory and would otherwise silently clobber each other's staged files.
- **A near-miss typo of the optional `files` field is now flagged** (`filez` → "did you mean files?"). Under plain `.passthrough()` such a typo was silently swallowed and the eval ran in an empty workspace. The check is a single-edit match scoped to recognized fields, so legitimate adopter extras (`name`, `category`, `notes`, `_category_note`) still pass through untouched.
- **`stageEvalWorkspaces` no longer mislabels copy failures as containment escapes.** Containment (`joinUnderRoot`) and the filesystem copy are now in separate try/catch blocks, so a permission/illegal-filename/disk error reports accurately instead of as "escapes the eval directory."
- **Skill-test `expected_output` is now optional, and is fed to the grader as context when present.** The pass/fail verdict is always decided per `expectations` entry, so `expected_output` is no longer required (per Postel's Law) — this unblocks real adopter suites (e.g. `dxa-consumption`) that grade with `expectations` alone. Previously the field was accepted but consumed by nothing; the experimenter prompt now passes it to the grader as the author's prose description of a correct result, informing judgment without becoming a checklist item. Still validated as a non-empty string when present.
- **Skill-test eval fixtures are now excluded from the remaining two link/structure validators that scanned them.** The intentionally-broken eval fixtures (under `resources/skills/evals/`) were excluded from the repo-root resource validation, ESLint, and repo-structure checks, but two validators still scanned them and failed on a clean (uncached) run: the `vat-development-agents` package config (so `vat verify`'s resources phase reported the fixtures' deliberate `LINK_BROKEN_FILE`s) and the `project-validation` dogfooding system test (whose hardcoded exclude list omitted the dir). Both now exclude the eval fixtures, and every exclusion site cross-references the others.
- **`vat claude plugin build` now copies a tree-copied skill's `files:` artifacts into the distributed plugin (#127).** A skill that ships build-provided artifacts in its own directory via `files: [{ source, dest }]` but lives in a plugin's source tree was distributed by a verbatim tree-copy that skipped its `files:` step, so the shipped plugin was missing those artifacts. Build now applies each tree-copied skill's `files:` config into `skills/<name>/`, exactly as it already does for shared-pool skills — removing the need for an external inject-into-dist script (which VAT couldn't see, producing false `LINK_TO_GITIGNORED_FILE` and `missing-bundled-file` findings).
- **`vat verify` no longer false-flags skills in plugins distributed by verbatim tree-copy (`vat build --only claude`).** A plugin that ships its skills by copying its own `skills/` tree (`source:` set, `skills: []`) builds correctly, but two verify checks still assumed the shared-skill-pool model and failed a byte-correct artifact: `files-config-dests` looked for a skill's `files:` dests only under `dist/skills/<name>/` and missed the plugin tree where build actually wrote them, and `PUBLISHED_SKILL_NOT_IN_PLUGIN` was blind to `source:`, flagging every skill a tree-copy plugin ships. Both checks (and `vat build`) now agree on where a tree-copied skill lands, so the false failures are gone. (Whether private `.claude/skills/**` skills should count as "published" is unchanged and tracked separately.)
- **`ExternalLinkValidator.clearCache()` and `getCacheStats()` now operate on both caches (issue #113).** Slice 2 introduced a second cache instance for authenticated-link results (per-OS-user scoping); the existing `clearCache()` / `getCacheStats()` methods continued to touch only the anonymous cache, so an adopter rotating a token would see stale `401`/`403` entries until the auth cache TTL expired. Both methods now clear/sum across both caches.
- **`ExternalLinkCache` IO errors degrade to a cache miss instead of aborting validation (issue #113).** `loadCache()` previously threw on anything other than `ENOENT` / `SyntaxError` (e.g. `EACCES` on a permissions-restricted cache file, `EROFS` on a read-only filesystem); `saveCache()` had no try/catch (write errors propagated). A failed read / write on the status-cache file would abort the whole `vat resources validate` run. Both paths are now fail-soft: a read failure returns an empty in-memory cache, a write failure no-ops while the in-memory cache stays authoritative for the remainder of the run. Cost of a bad cache entry: one extra fetch. Cost of a bad cache entry under the previous behavior: the whole run.
- **Lazy-loaded embedding providers no longer mislabel model/runtime failures as "not installed" ([#118](https://github.com/jdutton/vibe-agent-toolkit/issues/118)).** `loadPipeline` in `transformers-embedding-provider.ts` wrapped both the dynamic `import('@xenova/transformers')` and the model download/inference in a single `catch` that always rethrew a fixed `@xenova/transformers is not installed` message, swallowing the real error (not even as `cause`) — so a model-download or `onnxruntime-node` native-backend failure on an installed package was reported as a missing dependency. The two failure modes are now separated: an import failure keeps the actionable install hint (now with the original error attached as `cause`), while a model/inference failure throws `Failed to load transformers model '<model>'` preserving `cause`. The sibling `onnx-embedding-provider.ts` was audited: its install-hint `catch` was already correctly scoped to the import alone, but its model download (`ensureModelFiles`) and session creation (`InferenceSession.create`) previously bubbled raw errors with no provider/model context, so they now throw `Failed to download ONNX model '<model>'` / `Failed to load ONNX model '<model>'` with `cause` preserved.
- **Transformers.js integration tests now skip on Windows CI instead of flaking.** `transformers-embedding-provider.integration.test.ts` and the Transformers.js block of `comparison.integration.test.ts` skip on Windows (in addition to skipping when the optional `@xenova/transformers` dependency is absent), matching the existing `onnx-embedding-provider` test. These tests download a model over the network and load the `onnxruntime-node` native backend — both flaky in Windows CI. Such a failure was previously mislabeled `@xenova/transformers is not installed` by an over-broad `catch` in the provider's `loadPipeline` (the package was installed; the model download/inference is what failed), which is also why an availability-only guard did not prevent it.
- **Config-first skill discovery now honors `..` in `skills.include` patterns.** `vat build`, `vat verify`, and `vat skills validate` all funnel through `discoverSkillsFromConfig`, which previously passed every include pattern to a single downward-only crawl rooted at `projectRoot` — so an include like `"../../docs/skills/*/SKILL.md"` (common in monorepos where SKILL.md sources live alongside, not inside, the package) silently matched zero skills. `vat audit` accepted the same config only because it has a separate filesystem-first walker. Each include pattern is now split into a literal base + glob remainder via `picomatch.scan`, patterns are grouped by their resolved absolute base, and the crawler runs once per base — making config-first discovery agree with audit. User-supplied excludes stay anchored to `projectRoot` so patterns like `docs/private/**` keep their original meaning, and a pattern resolving to a nonexistent base now silently produces zero matches.
- **Anchor validation no longer reports a false `LINK_BROKEN_ANCHOR` for un-indexed target files (#112).** Previously a fragment link to any file the resource registry had not parsed (e.g. a target outside the crawl) was reported as a broken anchor. Anchor checks now skip targets absent from the fragment index — affecting markdown and HTML alike — while genuinely missing fragments in indexed files are still reported.
- **`vat resources validate` no longer crashes on same-stem `.md` + `.html` sibling files (#116).** Making HTML first-class added `.html`/`.htm` to the crawl, and same-stem siblings (e.g. `index.md` + `index.html`) previously produced an uncaught `Duplicate resource ID` exception that aborted the whole command. Fixed by the extension-suffixed ids above (siblings now get distinct ids), with `DUPLICATE_RESOURCE_ID` as a graceful backstop for any genuine post-normalization collision.
- **Post-build link checks now cover bundled HTML (#116).** `checkBrokenPackagedLinks` and the unreferenced-file check previously scanned only `.md`, so a broken `<a href>`/`<img src>` inside a packaged `.html`/`.htm` file shipped with a green build. Both checks — and the reachability traversal — now extract HTML links via the same parser, so broken links in packaged HTML surface as `PACKAGED_BROKEN_LINK` (failing the build) and an HTML file referenced only by other HTML is no longer falsely flagged `PACKAGED_UNREFERENCED_FILE`.
- **Deferred-artifact existence parity in the link walker (issue #129 carry-forward).** `walk-link-graph`'s `checkDeferred` guarded only the `files:` *source* branch with `!existsSync`; the *dest* branch deferred unconditionally. An existing real file at a `files:` dest (e.g. a gitignored artifact already on disk) was therefore silently downgraded to the `LINK_DEFERRED_ARTIFACT` info code, masking a genuine `LINK_TO_GITIGNORED_FILE` / directory-target signal. Both branches now share the existence guard: a path is treated as deferred only when it does not yet exist on disk.
- **`computeDeferredPaths` resolves `files:` sources exactly as the packager does (issue #129 carry-forward).** The deferred-source set was computed with `resolve(projectRoot, source)`, which let an absolute-looking source escape the project root, while the packager copies with `resolve(join(projectRoot, source))`. The two now use the identical expression, so an absolute-looking source roots under the project root in both places and the deferred set matches what the build actually copies.
### Internal
- **Unified `resolveSkillSource` skill-source resolver (#132, foundation).** A `skill-source/` module in `@vibe-agent-toolkit/agent-skills` that materializes a typed source union (`workspace` / `npm` / `url(+sha256)` / `path` / `vendored`) to a hardened, content-addressed staged directory through a per-user, `0700`, uid-checked fetch cache. The git-URL parser moved from `@vibe-agent-toolkit/cli` to `@vibe-agent-toolkit/utils`. No user-facing CLI surface yet — this is the resolver consumed by `vat skill test`.
- **Authenticated external-URL resolution foundation (issue #113, slice 1).** A pure `link-auth/` engine in `@vibe-agent-toolkit/utils` (host-glob provider selection, ordered token sources with no shell, header rendering with `Authorization` redaction, `github`/`sharepoint` macros) plus a strict `resources.linkAuth` config schema in `@vibe-agent-toolkit/resources`. Not yet wired into validation — consumer integration and the `LINK_AUTH_*` codes land in later slices — so there is no user-facing behavior yet.
- **`corpus/seed.yaml` is now generated from the upstream Anthropic marketplaces (issue #99, slice 1b).** A committed importer (`bun run import-marketplace [--allow-shrink]`) fetches the `claude-plugins-official` and `knowledge-work-plugins` catalogs, deduplicates by `source` URL, and rewrites the seed — replacing the previously hand-maintained list. Re-import is guarded against accidental shrinkage (refuses to overwrite on a 0-plugin fetch or a >20% drop unless `--allow-shrink`); current entry counts and audit provenance live in the generated seed header.
- **Empirical compatibility harness (issue #100).** A research scaffold (`packages/dev-tools/src/compat-empirical/`) for measuring skill compatibility across `claude-code`, `claude-cowork`, and `claude-chat` — it joins VAT's static predictions with deterministic runtime observations and an LLM-judge read into a reality-vs-prediction matrix, as evidence for future detector improvements. Lives entirely in the private `dev-tools` package with no adopter-facing surface; no detector or `RUNTIME_PROFILES` changes. [Design](./docs/research/2026-05-23-compat-empirical-harness-v2-design.md).
- **Cowork driver spike.** [`docs/contributing/cowork-driver-spike.md`](docs/contributing/cowork-driver-spike.md) records a time-boxed finding that `claude-cowork` cannot currently be driven programmatically (no public API/CLI surface), so it stays on `scripted-assisted` in the compat harness. Notes the public-beta Skills API as a separate, fully-automatable runtime worth a future follow-up.
- **Subscription-only compat harness billing.** The compat harness now bills a Claude Pro/Max subscription via a shared `claude` CLI invoker (uses the operator's `CLAUDE_CODE_OAUTH_TOKEN` and strips all API credentials from the child env), instead of the API; the LLM judge migrated off `@anthropic-ai/sdk` onto the same CLI. Private `dev-tools` only — no adopter-facing surface.
- **Intent-aware skill-resource verdict engine (issue #129, slice 3).** Skill-resource validation now routes through a pure verdict engine (`packages/agent-skills/src/validators/rule-engine/`): `evaluate(ctx)` maps an intent-aware context to at most one validation code, and a single `materializeIssue` constructor sources severity/description/fix/reference from `CODE_REGISTRY` so docs, runtime, and tests cannot drift. This is a refactor of how the existing codes are produced — the built and live paths now share one engine instead of duplicated literals, with no change to which codes fire — guarded by a table-driven scenario harness that enforces one-code-per-context, registry equality, and an anti-workaround invariant on every code's `fix`.
- **Single-source rule catalog (issue #129 AC5).** `docs/validation-codes.md` gains a machine-readable skill-resource rule catalog (between `<!-- BEGIN:rule-catalog -->` markers) and a disambiguation map; a docs test enforces full cell-equality (severity/description/fix) with `CODE_REGISTRY` so the registry, docs, and runtime cannot drift.
github-actions Bot
pushed a commit
that referenced
this pull request
Jul 2, 2026
### Added
- **Dogfood eval suites for the whole `vat-development-agents` skill set, plus the fixes that dogfooding surfaced.** Every published VAT dev skill now ships a committed `vat skill test` eval suite (`evals/<skill>/`): `vat-audit`, `vat-skill-authoring`, `vat-knowledge-resources`, `vat-skill-distribution`, `vat-rag`, `vat-agent-authoring`, and `markdown-rewriting` (joining the existing `vat-skill-review` suite), wired via `skills.config.<skill>.test`. Final grades: vat-skill-distribution 25/25, vat-agent-authoring 24/24, vat-rag 22/22, vat-knowledge-resources 22/22, markdown-rewriting 18/18, vat-skill-authoring 21/22 (one capability-headroom miss), vat-audit 33/40 baseline A/B (the without-skill failures demonstrate the skill's lift on CI-gating/compat knowledge). Running the suites caught real skill/doc bugs, now fixed:
- **`markdown-rewriting` is now actually published.** It lived in the skills dir and `vat-skill-authoring` told agents to load `[[markdown-rewriting]]`, but the discovery glob (`vat-*.md`) didn't match its name, so it never shipped — a dangling skill reference. Added it to `skills.include` and `package.json` `vat.skills`; it now builds and ships.
- **`vat-skill-authoring`** gained the conservative-frontmatter-keys rule (the standard key set; stamp `version`/`team`/ownership under `metadata:` or in config.yaml, never as bare top-level keys) — the agent was inventing top-level `version:`/`team:` fields.
- **`vat-skill-review`** corrected a factual error: it claimed a `metadata:` field "will be rejected," but `metadata` is an allowed standard key (the sanctioned home for custom data per `SKILL_FRONTMATTER_EXTRA_FIELDS`).
- **`vat-rag`** removed a nonexistent `vat rag index --rebuild` flag (the real reset is `vat rag clear`; indexing is incremental) and added the missing `OnnxEmbeddingProvider` to the providers table.
- **`vat-knowledge-resources`** now states that `strict` mode only rejects extra fields when the schema sets `"additionalProperties": false`, and that collection validation defaults to `permissive`.
- **Collection-validation docs** corrected: `mode` defaults to `permissive` (matching `validateAgainstCollectionSchema`), not `strict` as previously documented.
- **Skill-test harness:** `buildForwardedEnv` now forwards `USER`/`LOGNAME` (see below) and eval fixtures (including intentionally-broken `.ts` files) are excluded from ESLint.
- **`vat skill test run` / `vat skill test configure` — behavioral skill testing in a context-isolated harness (#132).** Stage a packaged skill plus its declared dependencies into a throwaway, locked-down harness and run a canned, non-interactive evaluation that grades the skill against your `evals.json` (reusing skill-creator's grading rubric and JSON shapes) and writes `grading.json` (with a published [JSON Schema](docs/skill-test-grading-schema.md)), `friction.json`, and full transcripts you can inspect. `configure` writes a per-skill `test:` block to your config as a surgical edit — only the keys you pass change; surrounding formatting and comments are byte-preserved; a first `run` with no `evals.json` writes a template for you to fill in. Runs end-to-end against `claude` 2.x. **Security:** the harness runs the skill's own code with your account's privileges — it is *context* isolation, not an OS sandbox — so `run` requires `--i-understand-this-runs-skill-code`, enforced *before* anything runs (including the optional pre-stage build), and you should only test skills you trust. The pass/fail verdict is recomputed from the graded expectations, so a failing or empty grade is never silently reported as a pass; add `--fail-on-eval-failure` to make a failing eval exit non-zero and gate CI on it. See the new `vibe-agent-toolkit:vat-skill-testing` skill for auth modes, budget/turn/timeout caps, `--baseline` A/B runs, and exit codes.
- **Pre-stage `build:` hook + plugin-root staging.** An optional `test.build` command runs once before staging, so a skill that depends on a generated, un-committed artifact has it present (a non-zero build fails fast at preflight, before any tokens are spent). Plugin-distributed skills stage under their real plugin-root layout with `CLAUDE_PLUGIN_ROOT` set; standalone skills stage flat.
- **Declared test-env passthrough.** `passEnv` / `--pass-env` forwards host variables; `env` / `--env` injects values with `${fixturesDir}` / `${stagedSkillDir}` / `${harnessRoot}` / `${resultsDir}` interpolation. Both apply *after* the security allowlist — protected names always win, so committed test config can neither reroute your account credentials nor inject code: auth credentials, `PATH`, and credential-routing variables (`ANTHROPIC_BASE_URL` and the other endpoint/proxy overrides, `NODE_OPTIONS`, `NODE_EXTRA_CA_CERTS`) cannot be overridden. Fixtures under the skill's `evals/fixtures/` auto-stage with the eval tree.
- **Project-aware subject resolution.** Name a skill declared in `vibe-agent-toolkit.config.yaml` and `run` builds it first and tests the shipping **dist** — link-following, reference-rewriting, nav-stripping, and `files:` injection all applied — so you exercise exactly what installs, not the source tree. A path (including an already-built dist dir), or a `workspace:` / `npm:` / `url:` / `path:` / `vendored` source, is tested as-is; use `./<name>` to force a local directory over a colliding declared name. `--no-build` stages an existing dist without rebuilding (and errors if it is absent); `--dry-run` assembles the command without building and flags when the previewed dist may be stale, and — when no `evals.json` exists yet — reports where a real run *would* scaffold the template (exit 3) instead of writing it, so a dry run never touches your tree. A build failure fails fast at preflight (exit 2), before any tokens are spent.
- **Eval `files` are now provisioned.** Each eval's declared input files are staged into a per-eval working directory the executor operates on, enabling realistic "drop the agent in a project" evals. Files resolve relative to the `evals.json` directory and are materialized under `<harnessRoot>/workspaces/<id>/`; the experimenter prompt hands the executor that directory via a new `{{WORKSPACES_ROOT}}` token. A declared-but-missing input file fails fast at preflight (exit 2). Previously `files` was documented but inert.
- **Merge-readiness: liberal eval-suite schema, macOS subscription-auth fix, expanded skill, first dogfood suite.** (1) `evals.json` is adopter-authored data VAT *reads*, so its schema is now liberal per VAT's Postel's Law: `EvalSuiteSchema`/`EvalEntrySchema` are `.passthrough()` and `id` accepts a descriptive **string** or an int — only the fields VAT consumes (`prompt`, `expected_output`, `expectations`) stay required. This reverses the earlier strict-parser call that rejected real adopter suites three ways (string `id`, `category`, `_category_note`) and restores compatibility for the flagship adopter (app-platform/dxa). The persisted `test:` *config* block stays **strict** (it's VAT-produced config) — the deliberate inverse. (2) **macOS subscription-auth fix:** the harness env allowlist (`buildForwardedEnv`) dropped the POSIX `USER`/`LOGNAME` vars, so on macOS `claude auth status` could not read the login Keychain with the API key scrubbed — `--auth subscription` (and `inherit`'s subscription fallback) wrongly failed preflight, and the experimenter child could not authenticate. `USER`/`LOGNAME` are now forwarded (non-secret; already derivable from the forwarded `HOME`). (3) The `vibe-agent-toolkit:vat-skill-testing` skill gains a research-grounded "Authoring `evals.json`" section (blind realistic prompts, discriminating + negative expectations, categories, fixtures, `--baseline` skill-lift, grading) and a full flag⇄config knob table. (4) Ships the first committed VAT dogfood suite (`vat-skill-review`, 5 evals across catch-violation / no-false-positive / guidance-correctness) wired via `skills.config.vat-skill-review.test`, with eval fixtures excluded from `vat resources validate`.
- **`files:` entries now support glob sources and an optional `integrity` byte-verify.** A `source` containing glob magic (`*`, `**`, `?`, `[`) fans out into a directory `dest`, preserving the directory structure below the static base (glob is VAT's existing idiom, as in `skills.include` — no `recursive` flag). Globbed dests are late-bound, so `SKILL.md` links into them are treated as deferred artifacts at validate time (no `LINK_TO_GITIGNORED_FILE` allowlist needed). Add `integrity: true` to byte-verify the copy at build time and assert an exact dest subtree for glob entries.
- **`NON_PORTABLE_ASSET_REFERENCE` validation code (default `warning`) — a portability check family.** `vat skills validate` / `vat audit` now flag a skill document that references a bundled script/asset via a non-portable anchor, scanning the `SKILL.md` body **and every reachable bundled markdown doc** (agents copy invocations from reference files too). It's a family of sub-checks under one code — `claude-plugin-root`, `claude-project-dir`, and `absolute-script-path` — each finding names the variant and carries a tailored fix, and a single `validation.allow` entry silences the whole family for a file. These anchors don't exist when a skill is mounted standalone (claude.ai upload, API container), so the path breaks on the agent's first invocation; reference bundled files relative to the skill directory instead. See [`NON_PORTABLE_ASSET_REFERENCE`](docs/validation-codes.md#non_portable_asset_reference).
- **Skill-authoring guidance: portable bundled-script paths.** The `vibe-agent-toolkit:vat-skill-authoring` skill now documents how to reference bundled scripts/assets portably (relative to the skill directory, never `CLAUDE_PLUGIN_ROOT`/absolute/env-var anchors), and `vibe-agent-toolkit:vat-skill-review` carries the matching pre-publication checklist item.
- **Skill-review guidance: reserved words `claude`/`anthropic` in skill names.** The `vibe-agent-toolkit:vat-skill-review` skill's Naming section now carries the reserved-word rule as a canonical `[A]` item — Anthropic's authoring guidance states a skill `name` "Cannot contain reserved words: 'anthropic', 'claude'", and Claude Code refuses to load a non-certified skill named that way, so it fails at install/validation, not just review (`[RESERVED_WORD_IN_NAME]`). Surfaced by dogfooding the skill against its own eval suite (the reviewer was noting the prefix as "redundant" but missing the install-blocking consequence). The rule directs the reviewer to surface that consequence when reviewing such a name and to include the warning when advising on naming.
- **`NON_PORTABLE_COMMAND` validation code (default `warning`) — a portability check family.** `vat skills validate` / `vat audit` now flag a skill document that tells an agent to run a GNU/Linux-only shell command, scanning the `SKILL.md` body **and every reachable bundled markdown doc** (agents copy invocations from reference files too). It's a family of sub-checks under one code — `timeout`, `grep-pcre` (`grep -P`), `sed-i-no-backup` (`sed -i` with no suffix), `readlink-f`, and `date-d` (GNU `date -d`) — each finding names the variant and carries a tailored fix, and a single `validation.allow` entry silences the whole family for a file. Patterns match commands in command position only (not bare prose), so `grep -E`/`sed -i.bak` and nouns like "the request will timeout" are not flagged. Promotes a former manual `vat skill review` checklist line into an automated check. See [`NON_PORTABLE_COMMAND`](docs/validation-codes.md#non_portable_command).
- **linkAuth content-fetch primitive + content cache (issue #113, slice 3).** Ships the public `fetchAuthenticated(url, config, options) → { bytes, metadata, cached } | { outcome: 'unsupported' | 'unverified' }` primitive (`packages/resources/src/link-auth-content-fetch.ts`), per design §6.2 — *sibling to* the slice-2 health-check path, both reading from the same engine config and rewrite pipeline. No consumer wiring (asset-references, bundling) lands in this slice; the primitive ships standalone so future callers can adopt it without reworking the contract. **Two-mode headers (§6.2):** `Provider` gains an optional `fetch: { headers }` block alongside `auth: { headers }`; `resolveAuthenticatedUrl` now dual-expands both header sets against the same context (URL captures + resolved token), surfacing them on the success outcome as `headers` (auth, for health-check) and `fetchHeaders` (fetch, for content retrieval). The primitive merges `fetchHeaders` over `headers` so fetch-mode overrides on conflict — the canonical case being GitHub, where `Accept: application/vnd.github+json` returns 200 for any size but omits bytes >1 MiB (good for health-check) while `Accept: application/vnd.github.raw` streams the bytes inline (required for content). Adopter schema (`InlineProviderSchema`) gains a parallel `ProviderFetchSchema` (passthrough, like the rest of the adopter linkAuth tree per the repo Postel's Law rule), and the compile-time `_KeysAgree` drift check picks up the new field automatically. The resolved-token-wins precedence (URL-capture-named `token` cannot beat the resolved value) and the null-prototype hardening apply to `fetchHeaders` too. **`ContentCache` (§6.3, `packages/resources/src/content-cache.ts`):** new persistence class for the content-fetch primitive — distinct from slice 2's `ExternalLinkCache` (which is a status cache). Per-entry layout: `<sha256(rewrittenUrl)>.json` (metadata: status, content-type, etag, last-modified, fetchedAt, rewrittenUrl, `version: 1`) + `<sha256(rewrittenUrl)>.bin` (raw bytes), under a caller-supplied `cacheDir` (the validator-style `<cacheDir>/content/auth-${osUser}/` scoping is the caller's responsibility — the class only knows about the directory it was given, mirroring §6.3's "cross-user isolation = OS user, not cache key"). 30-minute default TTL (`§6.3`), tunable via constructor and via the threaded-through `resources.linkAuth.cache.ttlMinutes` adopter config (`buildLinkAuthEngineConfig` now copies the adopter `cache` block onto the engine `LinkAuthConfig`, which previously dropped it silently). Write order is `.bin` first, then `.json` as the commit marker — a partial-write crash leaves either no entry or `.bin` ahead of `.json` (reads as a miss), never `.json` ahead of `.bin` (which would serve stale bytes under new metadata). On-disk metadata fields are whitelisted via a single `pickMetadata()` helper used by both `set()` (strip smuggled fields before write) and `get()` (strip the on-disk `version` before return), so token-bearing fields a caller might smuggle through structural typing cannot land on disk — defense in depth on top of the closed `ContentMetadata` interface. TTL boundary is `>`, not `>=` — entries are valid at exactly the TTL, expire at TTL + 1 ms; tests pin both boundary cases. Fail-soft IO per #125 review: `EACCES` / `EROFS` / corrupted JSON degrade to a miss (read) or no-op (write), never throw. Forward-compat `version: 1` mirrors `ExternalLinkCache`. **Primitive behavior:** the four outcome branches — `unsupported` and `unverified` short-circuit with no fetch and **no cache touch** (§6.3: never cache `unverified`, since the result flips the moment a token appears); cache-hit returns `{ bytes, metadata, cached: true }` with no fetch; otherwise fetch via `authTransport` (cross-origin auth strip + 429 retry inherited from slice 2), read the body binary-clean via `arrayBuffer()`, build metadata from response headers (content-type / etag / last-modified default to `null` when absent), write through to the cache if supplied. `forceRefresh: true` bypasses cache reads but still writes through. `AbortSignal` propagates to the transport. The token value is interpolated into request headers in-memory and never flows into `ContentMetadata`; an end-to-end test reads every file in the cache directory after a fetch and asserts the literal token string is absent. **`wrapLinkAuthDepsWithMemo` lifted to its own module** (`packages/resources/src/link-auth-deps-memo.ts`) and exported from the resources barrel — slice 2 originally housed it private inside `external-link-validator.ts`, but the standalone primitive needs the same memoization, and the lift centralizes the implementation so jscpd cannot flag a clone. Validators and primitive callers iterating many URLs from the same provider wrap their `deps` once and reuse, so `gh auth token` / any `command`-source resolver runs at most once across the iteration. **File rename for slice 2's transport:** `packages/resources/src/link-auth-fetch.ts` → `link-auth-transport.ts`, and the exported function `fetchAuthenticated` → `authTransport` (with `AuthFetchOptions` → `AuthTransportOptions`) — frees the `fetchAuthenticated` name for the spec-documented primitive and aligns the filename with the symbol's role as the lower-level auth-safe HTTP wrapper. **47 new tests:** `link-auth-content-fetch` (15 covering short-circuits, header merge with fetch.headers override, cache hit/miss/forceRefresh, unverified-never-cached, binary-clean round-trip, signal pass-through, token-never-persisted), `content-cache` (14 covering round-trip, binary safety, distinct-URL isolation, overwrite, TTL boundary at `=` and `=+1ms`, version-mismatch eviction, corrupted-JSON tolerance, POSIX-skipped `EACCES` fail-soft on read and write, and the whitelist-on-write check), `link-auth-deps-memo` (5 covering single-source memo, distinct-argv independence, default-runCommand fallback, deps pass-through, undefined-deps handling), and 13 augmenting tests on the slice 2 surface for the new `provider.fetch` block (engine dual-expansion, schema acceptance/rejection, cache field propagation). The slice 3 primitive does not wire into any existing CLI command — `--refresh` / `--no-cache` ships with the first consumer slice.
- **linkAuth validator wiring + per-host outcome codes (issue #113, slice 2).** The slice-1 pure engine is now end-to-end: when an adopter sets `resources.linkAuth` in `vibe-agent-toolkit.config.yaml`, `vat resources validate` bypasses the anonymous `markdown-link-check` path for any URL whose host is claimed by a provider, issues an authenticated `fetch()` against the rewritten URL with the configured token, and classifies the response per design §7 into one of five new `CODE_REGISTRY` entries: `LINK_AUTH_DEAD` (404/410 from an honest-404 host — `error`-severity, the only such code in the slice; design §7 establishes that an authenticated 404 against e.g. SharePoint is high-confidence link rot, satisfying the rule-design corpus-evidence bar), `LINK_AUTH_DEAD_OR_UNAUTHORIZED` (404 from an ambiguous host like GitHub that masks `403`s — warn), `LINK_AUTH_FORBIDDEN` (`403` — warn), `LINK_AUTH_UNAUTHORIZED` (`401` — warn, promote to `error` on strict CI lanes), and `LINK_AUTH_UNVERIFIED` (no token resolved — warn, never cached per §6.3). New files: `packages/resources/src/link-auth-fetch.ts` (`fetchAuthenticated()` — bounded redirect loop with **cross-origin `Authorization` stripping (§8)** that is sticky across the rest of the chain to defeat token-laundering, **429/`Retry-After` honoring (§5.2)** parsing both delta-seconds and HTTP-date forms with a 60s DoS cap *and* a 250ms good-neighbor floor, all dependency-injectable via `fetchImpl`/`sleep`/`signal`); `packages/resources/src/link-auth-classify.ts` (pure `(status, providerCheck) → outcome+code` per §7's table); `packages/resources/src/link-auth-config-build.ts` (bridge from adopter config to engine — runs `expandMacro` on `{ use: <macro>, ...overrides }` entries; the adopter schemas are passthrough per the repo's Postel's Law rule, so the post-expansion `InlineProviderSchema.safeParse` catches missing required fields and wrong types on declared fields but lets unknown extras through, matching how the rest of project-config treats adopter input; a compile-time `_KeysAgree` assertion locks the schema's top-level field set to the engine's `Provider` interface). New cache architecture: a second `ExternalLinkCache` instance for auth-branch results, **keyed by the rewritten URL** (the original `blob/` URL 404s — caching it would poison results) and scoped to a per-OS-user subdirectory `cacheDir/auth-${sanitizedOsUser}/` so two users on a shared CI host never read each other's authenticated results (§6.3); the OS user resolves through `os.userInfo()` → `USER`/`USERNAME` env → `'default'` with a one-shot `console.warn` on the last fallback so the cross-user-leak risk is observable. Cache entries gain an explicit `version: 1` field; reads of any other version produce a miss, so slice 3's content-cache evolution can change the entry shape without misparsing pre-existing files. Doc-anchor coverage iterator test (`packages/agent-schema/test/docs/validation-codes.test.ts`) iterates `CODE_REGISTRY` and asserts each code has a matching `### \`CODE\`` heading in `docs/validation-codes.md` plus a convention-matching `entry.reference` — 126 assertions covering all 63 codes, future-proofs the per-code docs requirement. Five new doc sections under "Authenticated External Link Codes" in `docs/validation-codes.md`. Engine surface gains: `Provider.check` now flows through on the verified `ResolveOutcome` so the classifier can route per-provider `notFoundMeaning`; `ExternalLinkValidatorOptions` gains `linkAuthConfig`, `fetchImpl`, `linkAuthDeps`, `sleep`, and `osUser` (the first two are adopter-usable for corporate-proxy/custom-TLS injection, the rest test-only); `LinkValidationResult` gains a `code?: IssueCode` field that the `resource-registry.ts` consumer prefers over the existing status-code-to-`EXTERNAL_URL_*` mapping. The validator memoizes `runCommand` results per unique argv for the duration of a `validate()` run, so validating N URLs from the same host runs `gh auth token` (and any other command-source token) at most once. **196 new tests across `link-auth-classify` (13), `link-auth-fetch` (19), `external-link-validator-auth` (25), `link-auth-config-build` (10), `validation-codes` (+126 doc-anchor iterator, +2 LINK_AUTH_* registry), and `external-link-cache` (+1 version-gate).** Security-load-bearing tests pin the cross-origin Authorization strip (case-insensitive, sticky across chains, with userinfo/relative-Location edge cases), the path-traversal sanitizer (table-driven over 9 pathological `osUser` inputs), the unverified-no-cache invariant, the cache-hit re-classification (cache hits re-run the classifier against the current provider so a `notFoundMeaning` flip between runs surfaces the new code, not the old one), the runCommand memoization (N URLs from the same provider → 1 command invocation), and an Object.hasOwn-based prototype-pollution defense on the `{ use }` discriminator in `buildLinkAuthEngineConfig`. Slice 4 (cross-platform `.cmd`-shim system test, `VAT_LINKAUTH_ALLOW_COMMAND=0` opt-out, contributor docs) and slice 3 (content-fetch primitive + content cache) are downstream.
- **linkAuth pure engine foundation (issue #113, slice 1).** Adds a config-driven engine for authenticated external URL resolution, scoped to the pure-logic layer with no consumer wiring yet (the `ExternalLinkValidator` integration and the `LINK_AUTH_*` `CODE_REGISTRY` entries are slice 2; the content-fetch primitive is slice 3). New `link-auth/` module under `@vibe-agent-toolkit/utils` with eight files: `transforms.ts` (closed allowlist — `base64url`, `urlencode`, `lower` — with `Object.hasOwn`-based prototype-chain defense), `template.ts` (tiny `${name}` / `${transform(name)}` renderer, deliberately separate from the Handlebars renderer in `utils/template.ts`), `rewrite.ts` (ordered `when → vars → to` pipeline with fragment/query stripping per design §5.2), `build-headers.ts` (header rendering plus structural `Authorization` redaction), `select-provider.ts` (host-glob matching via picomatch with `excludeHost`), `expand-macro.ts` (YAML loader + deep-merge expander), `resolve-token.ts` (ordered env / `safeExecResult`-backed argv-command sources, first-non-empty wins, no shell), and `resolve.ts` (the public `resolveAuthenticatedUrl(url, config)` entry returning one of `{fetchUrl, headers}` / `{outcome: 'unsupported'}` / `{outcome: 'unverified', reason}`). Ships the `github` and `sharepoint` macros as a YAML data asset (`src/link-auth/macros.yaml`), with a new cross-platform `packages/dev-tools/src/copy-yaml-assets.ts` post-build step bundling `.yaml` into `dist/` — first YAML-asset shipping pattern in the utils package. Adds `yaml` as a utils dependency. Companion Zod schema in `@vibe-agent-toolkit/resources` (`src/schemas/link-auth.ts`) validates the `resources.linkAuth` config block (strict; accepts either `{ use: <macro>, ...overrides }` or full inline providers), wired as an optional field on `ResourcesConfigSchema`. 140 unit tests in utils + 29 schema tests in resources, all pure-logic with no network or filesystem dependencies; security-load-bearing tests pin the closed-allowlist guarantee, the `${__proto__}` bypass defense, the token-never-leaks-into-Authorization invariant, and `shell: false` literal-argv handling.
- **Corpus seed expanded from 9 → 237 entries via a new committed importer at `packages/dev-tools/src/import-marketplace.ts` (`bun run import-marketplace [--allow-shrink]`).** The script fetches `.claude-plugin/marketplace.json` from `anthropics/claude-plugins-official` (205 of 209 raw entries kept) and `anthropics/knowledge-work-plugins` (30 of 60 — the knowledge-work catalog turns out to be ≈50% mirror entries of the official catalog) via `gh api`, maps each upstream entry to a `PluginEntry`, deduplicates by `source` URL (preserved VAT-owned entries always win; otherwise alphabetical-first-name wins within each duplicate cluster), and rewrites `corpus/seed.yaml`. Mapping rules: `bucket: official` uniformly (both catalogs are anthropics-curated marketplaces — `bucket` is the *reporting posture* per slice 1a, not code provenance); `confidence: first-party` for catalog-internal string sources and `github.com/anthropics/...` object sources, else `curated`; the `./partner-built/` knowledge-work convention overrides to `curated`; `maturity: production` for all entries. URL composition handles all five upstream source shapes (string, `git-subdir` ± `ref`, `url` ± `path`, `github`), throwing on unknown discriminators. The seven sample entries from slice 1a are regenerated from upstream manifests on every re-import. Re-import safety: the importer refuses to overwrite `corpus/seed.yaml` if either upstream catalog returned 0 plugins or the new entry count would drop more than 20% vs. the existing seed; `--allow-shrink` bypasses both gates for the rare case where shrinkage is real. The generated `seed.yaml` header dropped its earlier per-entry `validation:` claim (the importer throws on validation blocks today) and now states explicitly that entry `source` URLs pin a fragment ref (typically the default branch), not a per-entry commit SHA — the catalog SHAs in the header are this run's audit provenance. Issue #99 slice 1b — follows the schema change from PR #111 (slice 1a).
- **Empirical compatibility harness (`packages/dev-tools/src/compat-empirical/`).** Per-#100 research scaffold for measuring skill compatibility across `claude-code`, `claude-cowork`, and `claude-chat`: a CLI (`predict`/`run`/`judge`/`report`/`all`) that joins VAT's static predictions with deterministic runtime observations and an LLM-judge semantic read into a reality-vs-prediction matrix — an evidence artifact for proposing detector improvements that each cite specific (skill, runtime) cells. Probe coverage: multi-prompt + repeat-N with adaptive N=3→N=5 extension, mandatory positive+negative prompt pairing per corpus entry, and negative-prompt agreement inversion so false-positive triggers surface as `vat-optimistic`. Evidence quality: the deterministic class is widened from 6 to 9 values (splitting `error` into `install-failed`/`runtime-error`, `not-invoked` into `not-invoked-engaged`/`not-invoked-empty`, adding `refused`), with a v2 judge prompt that adds a `refused` verdict. Report fidelity: coverage stats, per-bucket headline (own/official/community × ran/agree/optimistic/pessimistic/gray-zone), gray-zone (mixed-signal) and high-variance subsections, and per-attempt variance rendered inline (`runtime-error (2/3) / failed (3/3)`). Judge replay persists `judge-calls/<skillId>-<promptId>-<target>-<attemptIdx>.json` artifacts that a new `re-judge` subcommand re-executes against an optionally different model or freshly-edited system prompt — without re-spending operator hours on the runtime side. Also landed: `git fetch --tags --force` before named-ref fetch (annotated tag refresh) and `setup()` teardown-first idempotency for the manual driver. No detector code or `RUNTIME_PROFILES` changes; lives entirely in the private `@vibe-agent-toolkit/dev-tools` package with no adopter-facing surface. Design: [the v2 harness design](./docs/research/2026-05-23-compat-empirical-harness-v2-design.md). Corpus authoring, the first real run, and the docs deliverable are the downstream work.
- **Cowork driver spike.** Added [`docs/contributing/cowork-driver-spike.md`](docs/contributing/cowork-driver-spike.md) — a time-boxed investigation (per §4a of the harness v2 design) of whether `claude-cowork` can be driven programmatically by the empirical compat harness today. Verdict: **not feasible**; cowork is a Claude Desktop app product with no public API/CLI surface. The `claude-cowork` runtime stays on `scripted-assisted` until Anthropic ships a Cowork CLI mode, Sessions API, or documented filesystem-import path. Adjacent finding (not a cowork replacement): the public-beta Skills API (`POST /v1/skills` + `container.skills[]` on `/v1/messages`) supports a fully-automatable *new* runtime — captured in the spike doc as a potential follow-up, gated on a separate design decision.
- **Subscription-only compat harness billing.** The harness now bills a Claude Pro/Max subscription instead of the API: both token-consuming surfaces (the `claude-code` runtime driver and the LLM judge) route through one shared `claude` CLI invoker (`runtimes/shared/claude-cli.ts`) that injects the operator's `CLAUDE_CODE_OAUTH_TOKEN` and deletes every API credential from the child env, so the CLI cannot fall back to API billing. The operator's own token is sourced at preflight — env var if set, otherwise an interactive prompt — so a run only ever spends the operator's personal plan. The judge was migrated off `@anthropic-ai/sdk` (dependency removed) onto the CLI, parsing a strict JSON verdict with one retry instead of the SDK's forced-tool call (`judge-system.md` now asks for a JSON object). `RunMetadata` gains `authMode` and the report methodology discloses subscription auth + parsed-not-forced verdicts. Premise (zero API billing under the OAuth token) still pending the manual smoke test.
- **First-class local HTML resources (#112).** `.html`/`.htm` files are now discovered, parsed, link- and anchor-validated, checked for well-formedness, and link-rewritten on bundle — using the same `ParseResult` contract and validation framework as markdown. A parse5-backed parser extracts `<a href>` and `<img src>` links plus `id`/`name` fragment anchors; `ResourceRegistry` routes HTML through it and persists optional `anchors`/`parseErrors` on `ResourceMetadata`. Anchor validation now uses a format-neutral fragment index (each file's markdown heading slugs or HTML `id`/`name`, with its case-matching policy carried per entry), enabling cross-format anchor checks (md↔html) with HTML ids matched case-sensitively and markdown slugs case-insensitively. A new `MALFORMED_HTML` code (default `info`) surfaces parser well-formedness diagnostics. On bundle, `<a href>`/`<img src>` values are rewritten by offset-splicing the original source (never re-serialized), so unchanged markup round-trips byte-for-byte and original attribute quoting is preserved (a rewritten value that would be unsafe unquoted is wrapped in quotes). Scope is `<a href>` + `<img src>` only; `<link>`/`<script>`/`<iframe>`/`<source srcset>`/CSS `url(...)` are deferred (asset/machinery references, not the content link graph). `<base href>` is not honored — relative hrefs resolve against the file's own directory (see the breaking note below for the `ResourceMetadataSchema` tightening that shipped with this work).
- **`DUPLICATE_RESOURCE_ID` validation code (default `error`).** When two files resolve to the same resource id after path normalization (e.g. `My Guide.md` and `my-guide.md` both → `my-guide-md`), `vat resources validate` now reports it as an `error` issue naming both files, instead of aborting the entire run with an uncaught `Duplicate resource ID` exception. Documented under [Resource Registry Codes](./docs/validation-codes.md).
- **Live audit/validate now sees source HTML links (issue #129 AC2).** `vat audit` / `vat skills validate` previously crawled `**/*.md` only, so links inside source `.html`/`.htm` files were invisible until build time. The live crawl now includes HTML (the registry already parses it via parse5), so the link-graph walker traverses HTML references and a broken local link inside a source HTML file surfaces as `LINK_MISSING_TARGET` at validate time, at parity with the built path's `PACKAGED_BROKEN_LINK`.
- **`LINK_DEFERRED_ARTIFACT` info code (issue #127, slice 2 of #129).** A `SKILL.md` link to a `files:`-declared artifact that doesn't exist yet (a dest built later, or a not-yet-created source) is no longer reported as a broken link — it downgrades from `LINK_MISSING_TARGET` to the new [`LINK_DEFERRED_ARTIFACT`](docs/validation-codes.md#link_deferred_artifact) info code at validate time, and `vat skills build` preserves and rewrites the link to the materialized dest instead of stripping it.
### Changed (breaking, pre-1.0)
- **`computeDeferredPaths` return type changed (issue #127, slice 2 of #129).** `computeDeferredPaths(files)` now returns `{ destPaths, sourcePaths }` instead of a flat `Set<string>` — a breaking API change (pre-1.0, intentional). Both `vat skills validate` and `vat skills build` now consume the deferred-path set (previously `deferredAssets` was silently dropped), and deferred dest/source paths resolve project-root-relative so the new behavior works for skills in subdirectories, not only at the project root. Plugin-local `files:` deferred paths remain out of scope for this slice (see [AC-10d](docs/architecture/skill-packaging.md#ac-10d--plugin-local-files-deferred-paths-are-out-of-scope-for-issue-127--slice-2-of-129)).
- **Directory links are now valid targets; `LINK_TARGETS_DIRECTORY` is narrowed to typed single-file slots (issue #126, slice 1 of #129).** A navigational local link that resolves to an existing directory (e.g. `[docs/](docs/)` in a ToC, README, or SKILL.md body) is no longer an error in `vat resources validate` or the skill-bundling link walk — previously any local link to a directory was a hard error. A renamed/deleted directory still fails via the ordinary broken-link path. `LINK_TARGETS_DIRECTORY` (still `error`) now fires **only** for a packaging `files:` *source* entry that resolves to a directory (the contract demands exactly one file). GitHub-style directory-index resolution (`docs/` → `docs/README.md`) is intentionally not implemented. Known limit (tracked for #129): a no-slash link such as `[Concepts](concepts)` that resolves to a directory is still treated as a file link; the slash form is the navigational case this slice covers.
- **`ResourceMetadataSchema` is now `strict()`.** Shipped with first-class HTML support (#112): the resource-metadata schema rejects unknown top-level fields instead of silently accepting them, so a typo or stale field in code that constructs `ResourceMetadata` now fails at parse time rather than passing through. Move any extra data into a recognized field or drop it.
- **Resource ids now carry a file-extension suffix.** `generateIdFromPath` appends `-<ext>` to every resource id (e.g. `guide.md` → `guide-md`, `guide.html` → `guide-html`, `README.md` → `readme-md`). This makes a markdown file and a same-stem HTML file distinct resources instead of colliding — the prerequisite for first-class HTML resources sharing a directory with their markdown source. Resource ids are internal, path-derived identifiers (never hand-authored in config or frontmatter), but anything that referenced an id by its old bare form must use the suffixed form — most visibly `vat rag query --resource-id` filters and re-indexed chunk ids (re-index to regenerate).
- **`vat resources validate` gains per-code severity configuration, and external-URL findings no longer fail the build by default.** Resource findings now use the same configurable severity framework as `vat skills`: each is a documented code (e.g. `LINK_BROKEN_FILE`, `EXTERNAL_URL_DEAD`) with a default severity, overridable per project under `resources.validation.severity` / `resources.validation.allow`. External-URL findings now default to `warning` and no longer flip the exit code (fixing a bug where they always failed the command); set their severity to `error` to restore failing. Severity now also accepts an `info` level. The never-implemented `resources.validation.checkLinks`/`checkAnchors`/`allowExternal` keys are removed.
- **`validation.severity` / `validation.allow` keys are validated against real codes.** A mistyped code key (e.g. `LNIK_OUTSIDE_PROJECT`) is now a config-load error instead of a silent no-op.
- **Corpus seed entries now require `bucket`, `confidence`, and `maturity` metadata fields.** `PluginEntrySchema` in `vat corpus scan`'s seed loader gains three required enum fields: `bucket: 'official' | 'community'`, `confidence: 'first-party' | 'curated' | 'listed'`, and `maturity: 'production' | 'experimental' | 'example'`. The bundled `corpus/seed.yaml` is updated; downstream callers running custom seeds must add the fields to every entry. `bucket` is the load-bearing discriminator (`official` entries report named findings; `community` entries are aggregate-only in follow-up work). The other two are descriptive metadata used by triage tooling.
- **`vat claude marketplace publish` no longer reports the project root `package.json` version in the CLI banner, commit message, status YAML, or CHANGELOG section lookup.** The label is now derived from the staged `marketplace.json`. Single-plugin marketplaces use the plugin's version — banner reads `Publishing marketplace "X" v0.0.4`, commit subject reads `publish v0.0.4`. Multi-plugin marketplaces drop the `v<X>` entirely — banner reads `Publishing marketplace "X"`, commit subject reads `publish X` — since the per-plugin `version` fields in the published `marketplace.json` are the source of truth for which plugin moved to which version. Two visible side-effects follow: (1) the status YAML's `published[*].version` field is now absent for multi-plugin marketplaces (previously it carried the misleading project version) — automation should read per-plugin versions from the published `marketplace.json` instead; (2) the stamped `## [X.Y.Z]` CHANGELOG lookup now uses the plugin's version rather than the project's, so a previously-ignored matching section will now be picked up as the commit body for single-plugin marketplaces. The `marketplace.json` schema's optional top-level `version` field is not yet consumed — that is a separate follow-up.
- **Adopter-facing `LinkAuthConfig` type renamed to `LinkAuthProjectConfig` (issue #113).** Both `@vibe-agent-toolkit/utils` (engine) and `@vibe-agent-toolkit/resources` (Zod-inferred adopter shape) previously exported a type named `LinkAuthConfig`, causing IDE auto-import ambiguity in any code that touched both. The adopter type — accessible as `import type { LinkAuthProjectConfig } from '@vibe-agent-toolkit/resources/schemas/link-auth'` — is the one renamed; the engine's `LinkAuthConfig` is unchanged (more API surface depends on it). Migration: rename the import. The Zod schema's name (`LinkAuthConfigSchema`) is unchanged.
- **External-link cache directory layout adds an `auth-${osUser}/` subdirectory and an entry `version: 1` field (issue #113 §6.3).** When `vat resources validate` runs with `resources.linkAuth` configured, authenticated-fetch results land under `<cacheDir>/auth-${sanitizedOsUser}/external-links.json` rather than the shared `external-links.json` used by the anonymous `markdown-link-check` path — two users on the same host (e.g. shared CI runners) cannot read each other's authenticated cache entries. All cache entries now carry an explicit `version: 1` field; entries written under a different (or missing) version are treated as a cache miss, so any pre-existing `external-links.json` triggers a one-time re-fetch on first run after upgrade. The `version` gate is forward-compat for slice 3's content-cache shape evolution.
- **`vat claude marketplace publish` no longer pushes per-plugin `<name>-v<version>` source-repo tags.** The post-publish tagging step (introduced alongside multi-plugin versioning) is removed entirely — no tags are created or pushed, and the misleading `Repository not found` / "tag already exists at a different commit" warnings it emitted on every cross-repo publish are gone ([#121](https://github.com/jdutton/vibe-agent-toolkit/issues/121)). The tags were pushed to the marketplace remote rather than a source remote, never landed anywhere useful, and there was no opt-in demand. Which plugin moved to which version is now determined solely by the per-plugin `version` fields in the published `marketplace.json`. No config key or flag is involved; if you relied on these tags, create them in your own release workflow.
### Fixed
- **`vat build` now fails when a shipped Claude plugin skill has a broken packaged link.** `vat claude plugin build` never ran a post-assembly link check on the plugin output tree — only the pool packaging path did. A plugin skill whose shipped links were broken (e.g. relative links that assumed pool-packaging relocation but the skill was verbatim tree-copied) previously shipped silently. `vat build` now runs the existing depth-free `checkBrokenPackagedLinks` check against every shipped skill dir after the `claude` phase and fails the build with a `PACKAGED_BROKEN_LINK` error on any dead link. The check is scoped per skill dir — a skill is a self-contained portable unit, so a link that escapes its own directory (even to a sibling skill that co-ships in the same plugin) is a broken shipped link.
- **`vat claude plugin build` no longer double-produces a skill that is both pool-selected and present in the plugin's own `skills/` source tree.** Tree-copy (verbatim, unaware of packaging) and pool-import (packaged, link-rewritten) never coordinated — a skill claimed by both mechanisms shipped as two coexisting copies at different depths inside the same `skills/<name>/` directory, with the raw tree-copy carrying un-rewritten (and therefore potentially dead) relative links. The plugin's resolved pool selector is now excluded from the verbatim tree-copy before it runs, so the pool-packaged copy is the sole source for a colliding skill; the build prints a warning naming the skill and both sources. Non-colliding tree-copy and pool-import usage is unaffected.
- **`validateSkill` no longer silently reports a boundary-escaping AND missing link as a warning-only boundary notice.** `validateLocalLink`'s boundary-escape check returned before the existence check ever ran, so a link that both escaped the skill directory boundary and pointed at a non-existent file was classified `LINK_OUTSIDE_PROJECT` (warning) and never surfaced as broken — this is why `vat claude marketplace validate` could report a shipped tree with a dead, boundary-escaping link as 0 errors. Existence is now checked before boundary classification: a missing target is always `LINK_INTEGRITY_BROKEN` (error), regardless of whether it also escapes the boundary. A link that escapes the boundary but resolves to an existing file is unaffected (still a warning).
- **Skill-test eval-suite schema hardened after an adversarial review of the Postel liberalization.** Four issues the `id`/passthrough widening introduced or left open, all verified against the real `dxa` adopter suites in `app-platform`:
- **String eval ids are now validated as filesystem-safe path segments** (`[A-Za-z0-9_-]+`). A string `id` names a per-eval working directory, and the experimenter substitutes it verbatim into `<workspaces>/<id>`; an id like `year:extraction` previously passed parse, then failed on Windows (illegal filename) — surfacing as a *misleading* "escapes the eval directory" copy error. Rejected at parse with a clear message instead. dxa's hyphenated ids (`dollar-quote-recovery`) are unaffected.
- **Numeric `1` and string `"1"` no longer slip past the uniqueness check.** Ids are deduped on their stringified form, since both name the same workspace directory and would otherwise silently clobber each other's staged files.
- **A near-miss typo of the optional `files` field is now flagged** (`filez` → "did you mean files?"). Under plain `.passthrough()` such a typo was silently swallowed and the eval ran in an empty workspace. The check is a single-edit match scoped to recognized fields, so legitimate adopter extras (`name`, `category`, `notes`, `_category_note`) still pass through untouched.
- **`stageEvalWorkspaces` no longer mislabels copy failures as containment escapes.** Containment (`joinUnderRoot`) and the filesystem copy are now in separate try/catch blocks, so a permission/illegal-filename/disk error reports accurately instead of as "escapes the eval directory."
- **Skill-test `expected_output` is now optional, and is fed to the grader as context when present.** The pass/fail verdict is always decided per `expectations` entry, so `expected_output` is no longer required (per Postel's Law) — this unblocks real adopter suites (e.g. `dxa-consumption`) that grade with `expectations` alone. Previously the field was accepted but consumed by nothing; the experimenter prompt now passes it to the grader as the author's prose description of a correct result, informing judgment without becoming a checklist item. Still validated as a non-empty string when present.
- **`vat claude plugin build` now copies a tree-copied skill's `files:` artifacts into the distributed plugin (#127).** A skill that ships build-provided artifacts in its own directory via `files: [{ source, dest }]` but lives in a plugin's source tree was distributed by a verbatim tree-copy that skipped its `files:` step, so the shipped plugin was missing those artifacts. Build now applies each tree-copied skill's `files:` config into `skills/<name>/`, exactly as it already does for shared-pool skills — removing the need for an external inject-into-dist script (which VAT couldn't see, producing false `LINK_TO_GITIGNORED_FILE` and `missing-bundled-file` findings).
- **`vat verify` no longer false-flags skills in plugins distributed by verbatim tree-copy (`vat build --only claude`).** A plugin that ships its skills by copying its own `skills/` tree (`source:` set, `skills: []`) builds correctly, but two verify checks still assumed the shared-skill-pool model and failed a byte-correct artifact: `files-config-dests` looked for a skill's `files:` dests only under `dist/skills/<name>/` and missed the plugin tree where build actually wrote them, and `PUBLISHED_SKILL_NOT_IN_PLUGIN` was blind to `source:`, flagging every skill a tree-copy plugin ships. Both checks (and `vat build`) now agree on where a tree-copied skill lands, so the false failures are gone. (Whether private `.claude/skills/**` skills should count as "published" is unchanged and tracked separately.)
- **`ExternalLinkValidator.clearCache()` and `getCacheStats()` now operate on both caches (issue #113).** Slice 2 introduced a second cache instance for authenticated-link results (per-OS-user scoping); the existing `clearCache()` / `getCacheStats()` methods continued to touch only the anonymous cache, so an adopter rotating a token would see stale `401`/`403` entries until the auth cache TTL expired. Both methods now clear/sum across both caches.
- **`ExternalLinkCache` IO errors degrade to a cache miss instead of aborting validation (issue #113).** `loadCache()` previously threw on anything other than `ENOENT` / `SyntaxError` (e.g. `EACCES` on a permissions-restricted cache file, `EROFS` on a read-only filesystem); `saveCache()` had no try/catch (write errors propagated). A failed read / write on the status-cache file would abort the whole `vat resources validate` run. Both paths are now fail-soft: a read failure returns an empty in-memory cache, a write failure no-ops while the in-memory cache stays authoritative for the remainder of the run. Cost of a bad cache entry: one extra fetch. Cost of a bad cache entry under the previous behavior: the whole run.
- **Lazy-loaded embedding providers no longer mislabel model/runtime failures as "not installed" ([#118](https://github.com/jdutton/vibe-agent-toolkit/issues/118)).** `loadPipeline` in `transformers-embedding-provider.ts` wrapped both the dynamic `import('@xenova/transformers')` and the model download/inference in a single `catch` that always rethrew a fixed `@xenova/transformers is not installed` message, swallowing the real error (not even as `cause`) — so a model-download or `onnxruntime-node` native-backend failure on an installed package was reported as a missing dependency. The two failure modes are now separated: an import failure keeps the actionable install hint (now with the original error attached as `cause`), while a model/inference failure throws `Failed to load transformers model '<model>'` preserving `cause`. The sibling `onnx-embedding-provider.ts` was audited: its install-hint `catch` was already correctly scoped to the import alone, but its model download (`ensureModelFiles`) and session creation (`InferenceSession.create`) previously bubbled raw errors with no provider/model context, so they now throw `Failed to download ONNX model '<model>'` / `Failed to load ONNX model '<model>'` with `cause` preserved.
- **Transformers.js integration tests now skip on Windows CI instead of flaking.** `transformers-embedding-provider.integration.test.ts` and the Transformers.js block of `comparison.integration.test.ts` skip on Windows (in addition to skipping when the optional `@xenova/transformers` dependency is absent), matching the existing `onnx-embedding-provider` test. These tests download a model over the network and load the `onnxruntime-node` native backend — both flaky in Windows CI. Such a failure was previously mislabeled `@xenova/transformers is not installed` by an over-broad `catch` in the provider's `loadPipeline` (the package was installed; the model download/inference is what failed), which is also why an availability-only guard did not prevent it.
- **Config-first skill discovery now honors `..` in `skills.include` patterns.** `vat build`, `vat verify`, and `vat skills validate` all funnel through `discoverSkillsFromConfig`, which previously passed every include pattern to a single downward-only crawl rooted at `projectRoot` — so an include like `"../../docs/skills/*/SKILL.md"` (common in monorepos where SKILL.md sources live alongside, not inside, the package) silently matched zero skills. `vat audit` accepted the same config only because it has a separate filesystem-first walker. Each include pattern is now split into a literal base + glob remainder via `picomatch.scan`, patterns are grouped by their resolved absolute base, and the crawler runs once per base — making config-first discovery agree with audit. User-supplied excludes stay anchored to `projectRoot` so patterns like `docs/private/**` keep their original meaning, and a pattern resolving to a nonexistent base now silently produces zero matches.
- **Anchor validation no longer reports a false `LINK_BROKEN_ANCHOR` for un-indexed target files (#112).** Previously a fragment link to any file the resource registry had not parsed (e.g. a target outside the crawl) was reported as a broken anchor. Anchor checks now skip targets absent from the fragment index — affecting markdown and HTML alike — while genuinely missing fragments in indexed files are still reported.
- **`vat resources validate` no longer crashes on same-stem `.md` + `.html` sibling files (#116).** Making HTML first-class added `.html`/`.htm` to the crawl, and same-stem siblings (e.g. `index.md` + `index.html`) previously produced an uncaught `Duplicate resource ID` exception that aborted the whole command. Fixed by the extension-suffixed ids above (siblings now get distinct ids), with `DUPLICATE_RESOURCE_ID` as a graceful backstop for any genuine post-normalization collision.
- **Post-build link checks now cover bundled HTML (#116).** `checkBrokenPackagedLinks` and the unreferenced-file check previously scanned only `.md`, so a broken `<a href>`/`<img src>` inside a packaged `.html`/`.htm` file shipped with a green build. Both checks — and the reachability traversal — now extract HTML links via the same parser, so broken links in packaged HTML surface as `PACKAGED_BROKEN_LINK` (failing the build) and an HTML file referenced only by other HTML is no longer falsely flagged `PACKAGED_UNREFERENCED_FILE`.
- **Deferred-artifact existence parity in the link walker (issue #129 carry-forward).** `walk-link-graph`'s `checkDeferred` guarded only the `files:` *source* branch with `!existsSync`; the *dest* branch deferred unconditionally. An existing real file at a `files:` dest (e.g. a gitignored artifact already on disk) was therefore silently downgraded to the `LINK_DEFERRED_ARTIFACT` info code, masking a genuine `LINK_TO_GITIGNORED_FILE` / directory-target signal. Both branches now share the existence guard: a path is treated as deferred only when it does not yet exist on disk.
- **`computeDeferredPaths` resolves `files:` sources exactly as the packager does (issue #129 carry-forward).** The deferred-source set was computed with `resolve(projectRoot, source)`, which let an absolute-looking source escape the project root, while the packager copies with `resolve(join(projectRoot, source))`. The two now use the identical expression, so an absolute-looking source roots under the project root in both places and the deferred set matches what the build actually copies.
### Internal
- **Skill-test eval fixtures excluded from the remaining link/structure validators (CI hygiene, no adopter-facing change).** The intentionally-broken eval fixtures (`resources/skills/evals/**` — non-portable SKILL.md samples, a fake plugin for `vat audit`) are test input, not real docs/code. They were already excluded from the repo-root resource validation, ESLint, and repo-structure checks; now also from the `vat-development-agents` package config (so `vat verify`'s resources phase stops failing on the fixtures' deliberate `LINK_BROKEN_FILE`s) and the `project-validation` dogfooding system test (hardcoded exclude list). Every exclusion site cross-references the others.
- **Eval fixtures hold clean, realistic code — incidental smells removed.** Two fixtures carried code-quality issues unrelated to what their eval tests: the `release-notifier-plugin` notifier script (a payload that only needs to *exist* so `vat audit` can flag the skill's local-script dependency) now validates its `--changelog` path instead of opening it blind, and the `vat-knowledge-resources` starter config dropped a redundant `TODO` comment (the eval's prompt already states the task). Fixtures that are themselves the *subject under review* (e.g. the vat-agent-authoring analyzer the eval asks an agent to improve) keep their VAT-domain flaws by design.
- **Unified `resolveSkillSource` skill-source resolver (#132, foundation).** A `skill-source/` module in `@vibe-agent-toolkit/agent-skills` that materializes a typed source union (`workspace` / `npm` / `url(+sha256)` / `path` / `vendored`) to a hardened, content-addressed staged directory through a per-user, `0700`, uid-checked fetch cache. The git-URL parser moved from `@vibe-agent-toolkit/cli` to `@vibe-agent-toolkit/utils`. No user-facing CLI surface yet — this is the resolver consumed by `vat skill test`.
- **Authenticated external-URL resolution foundation (issue #113, slice 1).** A pure `link-auth/` engine in `@vibe-agent-toolkit/utils` (host-glob provider selection, ordered token sources with no shell, header rendering with `Authorization` redaction, `github`/`sharepoint` macros) plus a strict `resources.linkAuth` config schema in `@vibe-agent-toolkit/resources`. Not yet wired into validation — consumer integration and the `LINK_AUTH_*` codes land in later slices — so there is no user-facing behavior yet.
- **`corpus/seed.yaml` is now generated from the upstream Anthropic marketplaces (issue #99, slice 1b).** A committed importer (`bun run import-marketplace [--allow-shrink]`) fetches the `claude-plugins-official` and `knowledge-work-plugins` catalogs, deduplicates by `source` URL, and rewrites the seed — replacing the previously hand-maintained list. Re-import is guarded against accidental shrinkage (refuses to overwrite on a 0-plugin fetch or a >20% drop unless `--allow-shrink`); current entry counts and audit provenance live in the generated seed header.
- **Empirical compatibility harness (issue #100).** A research scaffold (`packages/dev-tools/src/compat-empirical/`) for measuring skill compatibility across `claude-code`, `claude-cowork`, and `claude-chat` — it joins VAT's static predictions with deterministic runtime observations and an LLM-judge read into a reality-vs-prediction matrix, as evidence for future detector improvements. Lives entirely in the private `dev-tools` package with no adopter-facing surface; no detector or `RUNTIME_PROFILES` changes. [Design](./docs/research/2026-05-23-compat-empirical-harness-v2-design.md).
- **Cowork driver spike.** [`docs/contributing/cowork-driver-spike.md`](docs/contributing/cowork-driver-spike.md) records a time-boxed finding that `claude-cowork` cannot currently be driven programmatically (no public API/CLI surface), so it stays on `scripted-assisted` in the compat harness. Notes the public-beta Skills API as a separate, fully-automatable runtime worth a future follow-up.
- **Subscription-only compat harness billing.** The compat harness now bills a Claude Pro/Max subscription via a shared `claude` CLI invoker (uses the operator's `CLAUDE_CODE_OAUTH_TOKEN` and strips all API credentials from the child env), instead of the API; the LLM judge migrated off `@anthropic-ai/sdk` onto the same CLI. Private `dev-tools` only — no adopter-facing surface.
- **Intent-aware skill-resource verdict engine (issue #129, slice 3).** Skill-resource validation now routes through a pure verdict engine (`packages/agent-skills/src/validators/rule-engine/`): `evaluate(ctx)` maps an intent-aware context to at most one validation code, and a single `materializeIssue` constructor sources severity/description/fix/reference from `CODE_REGISTRY` so docs, runtime, and tests cannot drift. This is a refactor of how the existing codes are produced — the built and live paths now share one engine instead of duplicated literals, with no change to which codes fire — guarded by a table-driven scenario harness that enforces one-code-per-context, registry equality, and an anti-workaround invariant on every code's `fix`.
- **Single-source rule catalog (issue #129 AC5).** `docs/validation-codes.md` gains a machine-readable skill-resource rule catalog (between `<!-- BEGIN:rule-catalog -->` markers) and a disambiguation map; a docs test enforces full cell-equality (severity/description/fix) with `CODE_REGISTRY` so the registry, docs, and runtime cannot drift.
github-actions Bot
pushed a commit
that referenced
this pull request
Jul 2, 2026
### Added
- **Dogfood eval suites for the whole `vat-development-agents` skill set, plus the fixes that dogfooding surfaced.** Every published VAT dev skill now ships a committed `vat skill test` eval suite (`evals/<skill>/`): `vat-audit`, `vat-skill-authoring`, `vat-knowledge-resources`, `vat-skill-distribution`, `vat-rag`, `vat-agent-authoring`, and `markdown-rewriting` (joining the existing `vat-skill-review` suite), wired via `skills.config.<skill>.test`. Final grades: vat-skill-distribution 25/25, vat-agent-authoring 24/24, vat-rag 22/22, vat-knowledge-resources 22/22, markdown-rewriting 18/18, vat-skill-authoring 21/22 (one capability-headroom miss), vat-audit 33/40 baseline A/B (the without-skill failures demonstrate the skill's lift on CI-gating/compat knowledge). Running the suites caught real skill/doc bugs, now fixed:
- **`markdown-rewriting` is now actually published.** It lived in the skills dir and `vat-skill-authoring` told agents to load `[[markdown-rewriting]]`, but the discovery glob (`vat-*.md`) didn't match its name, so it never shipped — a dangling skill reference. Added it to `skills.include` and `package.json` `vat.skills`; it now builds and ships.
- **`vat-skill-authoring`** gained the conservative-frontmatter-keys rule (the standard key set; stamp `version`/`team`/ownership under `metadata:` or in config.yaml, never as bare top-level keys) — the agent was inventing top-level `version:`/`team:` fields.
- **`vat-skill-review`** corrected a factual error: it claimed a `metadata:` field "will be rejected," but `metadata` is an allowed standard key (the sanctioned home for custom data per `SKILL_FRONTMATTER_EXTRA_FIELDS`).
- **`vat-rag`** removed a nonexistent `vat rag index --rebuild` flag (the real reset is `vat rag clear`; indexing is incremental) and added the missing `OnnxEmbeddingProvider` to the providers table.
- **`vat-knowledge-resources`** now states that `strict` mode only rejects extra fields when the schema sets `"additionalProperties": false`, and that collection validation defaults to `permissive`.
- **Collection-validation docs** corrected: `mode` defaults to `permissive` (matching `validateAgainstCollectionSchema`), not `strict` as previously documented.
- **Skill-test harness:** `buildForwardedEnv` now forwards `USER`/`LOGNAME` (see below) and eval fixtures (including intentionally-broken `.ts` files) are excluded from ESLint.
- **`vat skill test run` / `vat skill test configure` — behavioral skill testing in a context-isolated harness (#132).** Stage a packaged skill plus its declared dependencies into a throwaway, locked-down harness and run a canned, non-interactive evaluation that grades the skill against your `evals.json` (reusing skill-creator's grading rubric and JSON shapes) and writes `grading.json` (with a published [JSON Schema](docs/skill-test-grading-schema.md)), `friction.json`, and full transcripts you can inspect. `configure` writes a per-skill `test:` block to your config as a surgical edit — only the keys you pass change; surrounding formatting and comments are byte-preserved; a first `run` with no `evals.json` writes a template for you to fill in. Runs end-to-end against `claude` 2.x. **Security:** the harness runs the skill's own code with your account's privileges — it is *context* isolation, not an OS sandbox — so `run` requires `--i-understand-this-runs-skill-code`, enforced *before* anything runs (including the optional pre-stage build), and you should only test skills you trust. The pass/fail verdict is recomputed from the graded expectations, so a failing or empty grade is never silently reported as a pass; add `--fail-on-eval-failure` to make a failing eval exit non-zero and gate CI on it. See the new `vibe-agent-toolkit:vat-skill-testing` skill for auth modes, budget/turn/timeout caps, `--baseline` A/B runs, and exit codes.
- **Pre-stage `build:` hook + plugin-root staging.** An optional `test.build` command runs once before staging, so a skill that depends on a generated, un-committed artifact has it present (a non-zero build fails fast at preflight, before any tokens are spent). Plugin-distributed skills stage under their real plugin-root layout with `CLAUDE_PLUGIN_ROOT` set; standalone skills stage flat.
- **Declared test-env passthrough.** `passEnv` / `--pass-env` forwards host variables; `env` / `--env` injects values with `${fixturesDir}` / `${stagedSkillDir}` / `${harnessRoot}` / `${resultsDir}` interpolation. Both apply *after* the security allowlist — protected names always win, so committed test config can neither reroute your account credentials nor inject code: auth credentials, `PATH`, and credential-routing variables (`ANTHROPIC_BASE_URL` and the other endpoint/proxy overrides, `NODE_OPTIONS`, `NODE_EXTRA_CA_CERTS`) cannot be overridden. Fixtures under the skill's `evals/fixtures/` auto-stage with the eval tree.
- **Project-aware subject resolution.** Name a skill declared in `vibe-agent-toolkit.config.yaml` and `run` builds it first and tests the shipping **dist** — link-following, reference-rewriting, nav-stripping, and `files:` injection all applied — so you exercise exactly what installs, not the source tree. A path (including an already-built dist dir), or a `workspace:` / `npm:` / `url:` / `path:` / `vendored` source, is tested as-is; use `./<name>` to force a local directory over a colliding declared name. `--no-build` stages an existing dist without rebuilding (and errors if it is absent); `--dry-run` assembles the command without building and flags when the previewed dist may be stale, and — when no `evals.json` exists yet — reports where a real run *would* scaffold the template (exit 3) instead of writing it, so a dry run never touches your tree. A build failure fails fast at preflight (exit 2), before any tokens are spent.
- **Eval `files` are now provisioned.** Each eval's declared input files are staged into a per-eval working directory the executor operates on, enabling realistic "drop the agent in a project" evals. Files resolve relative to the `evals.json` directory and are materialized under `<harnessRoot>/workspaces/<id>/`; the experimenter prompt hands the executor that directory via a new `{{WORKSPACES_ROOT}}` token. A declared-but-missing input file fails fast at preflight (exit 2). Previously `files` was documented but inert.
- **Merge-readiness: liberal eval-suite schema, macOS subscription-auth fix, expanded skill, first dogfood suite.** (1) `evals.json` is adopter-authored data VAT *reads*, so its schema is now liberal per VAT's Postel's Law: `EvalSuiteSchema`/`EvalEntrySchema` are `.passthrough()` and `id` accepts a descriptive **string** or an int — only the fields VAT consumes (`prompt`, `expected_output`, `expectations`) stay required. This reverses the earlier strict-parser call that rejected real adopter suites three ways (string `id`, `category`, `_category_note`) and restores compatibility for the flagship adopter (app-platform/dxa). The persisted `test:` *config* block stays **strict** (it's VAT-produced config) — the deliberate inverse. (2) **macOS subscription-auth fix:** the harness env allowlist (`buildForwardedEnv`) dropped the POSIX `USER`/`LOGNAME` vars, so on macOS `claude auth status` could not read the login Keychain with the API key scrubbed — `--auth subscription` (and `inherit`'s subscription fallback) wrongly failed preflight, and the experimenter child could not authenticate. `USER`/`LOGNAME` are now forwarded (non-secret; already derivable from the forwarded `HOME`). (3) The `vibe-agent-toolkit:vat-skill-testing` skill gains a research-grounded "Authoring `evals.json`" section (blind realistic prompts, discriminating + negative expectations, categories, fixtures, `--baseline` skill-lift, grading) and a full flag⇄config knob table. (4) Ships the first committed VAT dogfood suite (`vat-skill-review`, 5 evals across catch-violation / no-false-positive / guidance-correctness) wired via `skills.config.vat-skill-review.test`, with eval fixtures excluded from `vat resources validate`.
- **`files:` entries now support glob sources and an optional `integrity` byte-verify.** A `source` containing glob magic (`*`, `**`, `?`, `[`) fans out into a directory `dest`, preserving the directory structure below the static base (glob is VAT's existing idiom, as in `skills.include` — no `recursive` flag). Globbed dests are late-bound, so `SKILL.md` links into them are treated as deferred artifacts at validate time (no `LINK_TO_GITIGNORED_FILE` allowlist needed). Add `integrity: true` to byte-verify the copy at build time and assert an exact dest subtree for glob entries.
- **`NON_PORTABLE_ASSET_REFERENCE` validation code (default `warning`) — a portability check family.** `vat skills validate` / `vat audit` now flag a skill document that references a bundled script/asset via a non-portable anchor, scanning the `SKILL.md` body **and every reachable bundled markdown doc** (agents copy invocations from reference files too). It's a family of sub-checks under one code — `claude-plugin-root`, `claude-project-dir`, and `absolute-script-path` — each finding names the variant and carries a tailored fix, and a single `validation.allow` entry silences the whole family for a file. These anchors don't exist when a skill is mounted standalone (claude.ai upload, API container), so the path breaks on the agent's first invocation; reference bundled files relative to the skill directory instead. See [`NON_PORTABLE_ASSET_REFERENCE`](docs/validation-codes.md#non_portable_asset_reference).
- **Skill-authoring guidance: portable bundled-script paths.** The `vibe-agent-toolkit:vat-skill-authoring` skill now documents how to reference bundled scripts/assets portably (relative to the skill directory, never `CLAUDE_PLUGIN_ROOT`/absolute/env-var anchors), and `vibe-agent-toolkit:vat-skill-review` carries the matching pre-publication checklist item.
- **Skill-review guidance: reserved words `claude`/`anthropic` in skill names.** The `vibe-agent-toolkit:vat-skill-review` skill's Naming section now carries the reserved-word rule as a canonical `[A]` item — Anthropic's authoring guidance states a skill `name` "Cannot contain reserved words: 'anthropic', 'claude'", and Claude Code refuses to load a non-certified skill named that way, so it fails at install/validation, not just review (`[RESERVED_WORD_IN_NAME]`). Surfaced by dogfooding the skill against its own eval suite (the reviewer was noting the prefix as "redundant" but missing the install-blocking consequence). The rule directs the reviewer to surface that consequence when reviewing such a name and to include the warning when advising on naming.
- **`NON_PORTABLE_COMMAND` validation code (default `warning`) — a portability check family.** `vat skills validate` / `vat audit` now flag a skill document that tells an agent to run a GNU/Linux-only shell command, scanning the `SKILL.md` body **and every reachable bundled markdown doc** (agents copy invocations from reference files too). It's a family of sub-checks under one code — `timeout`, `grep-pcre` (`grep -P`), `sed-i-no-backup` (`sed -i` with no suffix), `readlink-f`, and `date-d` (GNU `date -d`) — each finding names the variant and carries a tailored fix, and a single `validation.allow` entry silences the whole family for a file. Patterns match commands in command position only (not bare prose), so `grep -E`/`sed -i.bak` and nouns like "the request will timeout" are not flagged. Promotes a former manual `vat skill review` checklist line into an automated check. See [`NON_PORTABLE_COMMAND`](docs/validation-codes.md#non_portable_command).
- **linkAuth content-fetch primitive + content cache (issue #113, slice 3).** Ships the public `fetchAuthenticated(url, config, options) → { bytes, metadata, cached } | { outcome: 'unsupported' | 'unverified' }` primitive (`packages/resources/src/link-auth-content-fetch.ts`), per design §6.2 — *sibling to* the slice-2 health-check path, both reading from the same engine config and rewrite pipeline. No consumer wiring (asset-references, bundling) lands in this slice; the primitive ships standalone so future callers can adopt it without reworking the contract. **Two-mode headers (§6.2):** `Provider` gains an optional `fetch: { headers }` block alongside `auth: { headers }`; `resolveAuthenticatedUrl` now dual-expands both header sets against the same context (URL captures + resolved token), surfacing them on the success outcome as `headers` (auth, for health-check) and `fetchHeaders` (fetch, for content retrieval). The primitive merges `fetchHeaders` over `headers` so fetch-mode overrides on conflict — the canonical case being GitHub, where `Accept: application/vnd.github+json` returns 200 for any size but omits bytes >1 MiB (good for health-check) while `Accept: application/vnd.github.raw` streams the bytes inline (required for content). Adopter schema (`InlineProviderSchema`) gains a parallel `ProviderFetchSchema` (passthrough, like the rest of the adopter linkAuth tree per the repo Postel's Law rule), and the compile-time `_KeysAgree` drift check picks up the new field automatically. The resolved-token-wins precedence (URL-capture-named `token` cannot beat the resolved value) and the null-prototype hardening apply to `fetchHeaders` too. **`ContentCache` (§6.3, `packages/resources/src/content-cache.ts`):** new persistence class for the content-fetch primitive — distinct from slice 2's `ExternalLinkCache` (which is a status cache). Per-entry layout: `<sha256(rewrittenUrl)>.json` (metadata: status, content-type, etag, last-modified, fetchedAt, rewrittenUrl, `version: 1`) + `<sha256(rewrittenUrl)>.bin` (raw bytes), under a caller-supplied `cacheDir` (the validator-style `<cacheDir>/content/auth-${osUser}/` scoping is the caller's responsibility — the class only knows about the directory it was given, mirroring §6.3's "cross-user isolation = OS user, not cache key"). 30-minute default TTL (`§6.3`), tunable via constructor and via the threaded-through `resources.linkAuth.cache.ttlMinutes` adopter config (`buildLinkAuthEngineConfig` now copies the adopter `cache` block onto the engine `LinkAuthConfig`, which previously dropped it silently). Write order is `.bin` first, then `.json` as the commit marker — a partial-write crash leaves either no entry or `.bin` ahead of `.json` (reads as a miss), never `.json` ahead of `.bin` (which would serve stale bytes under new metadata). On-disk metadata fields are whitelisted via a single `pickMetadata()` helper used by both `set()` (strip smuggled fields before write) and `get()` (strip the on-disk `version` before return), so token-bearing fields a caller might smuggle through structural typing cannot land on disk — defense in depth on top of the closed `ContentMetadata` interface. TTL boundary is `>`, not `>=` — entries are valid at exactly the TTL, expire at TTL + 1 ms; tests pin both boundary cases. Fail-soft IO per #125 review: `EACCES` / `EROFS` / corrupted JSON degrade to a miss (read) or no-op (write), never throw. Forward-compat `version: 1` mirrors `ExternalLinkCache`. **Primitive behavior:** the four outcome branches — `unsupported` and `unverified` short-circuit with no fetch and **no cache touch** (§6.3: never cache `unverified`, since the result flips the moment a token appears); cache-hit returns `{ bytes, metadata, cached: true }` with no fetch; otherwise fetch via `authTransport` (cross-origin auth strip + 429 retry inherited from slice 2), read the body binary-clean via `arrayBuffer()`, build metadata from response headers (content-type / etag / last-modified default to `null` when absent), write through to the cache if supplied. `forceRefresh: true` bypasses cache reads but still writes through. `AbortSignal` propagates to the transport. The token value is interpolated into request headers in-memory and never flows into `ContentMetadata`; an end-to-end test reads every file in the cache directory after a fetch and asserts the literal token string is absent. **`wrapLinkAuthDepsWithMemo` lifted to its own module** (`packages/resources/src/link-auth-deps-memo.ts`) and exported from the resources barrel — slice 2 originally housed it private inside `external-link-validator.ts`, but the standalone primitive needs the same memoization, and the lift centralizes the implementation so jscpd cannot flag a clone. Validators and primitive callers iterating many URLs from the same provider wrap their `deps` once and reuse, so `gh auth token` / any `command`-source resolver runs at most once across the iteration. **File rename for slice 2's transport:** `packages/resources/src/link-auth-fetch.ts` → `link-auth-transport.ts`, and the exported function `fetchAuthenticated` → `authTransport` (with `AuthFetchOptions` → `AuthTransportOptions`) — frees the `fetchAuthenticated` name for the spec-documented primitive and aligns the filename with the symbol's role as the lower-level auth-safe HTTP wrapper. **47 new tests:** `link-auth-content-fetch` (15 covering short-circuits, header merge with fetch.headers override, cache hit/miss/forceRefresh, unverified-never-cached, binary-clean round-trip, signal pass-through, token-never-persisted), `content-cache` (14 covering round-trip, binary safety, distinct-URL isolation, overwrite, TTL boundary at `=` and `=+1ms`, version-mismatch eviction, corrupted-JSON tolerance, POSIX-skipped `EACCES` fail-soft on read and write, and the whitelist-on-write check), `link-auth-deps-memo` (5 covering single-source memo, distinct-argv independence, default-runCommand fallback, deps pass-through, undefined-deps handling), and 13 augmenting tests on the slice 2 surface for the new `provider.fetch` block (engine dual-expansion, schema acceptance/rejection, cache field propagation). The slice 3 primitive does not wire into any existing CLI command — `--refresh` / `--no-cache` ships with the first consumer slice.
- **linkAuth validator wiring + per-host outcome codes (issue #113, slice 2).** The slice-1 pure engine is now end-to-end: when an adopter sets `resources.linkAuth` in `vibe-agent-toolkit.config.yaml`, `vat resources validate` bypasses the anonymous `markdown-link-check` path for any URL whose host is claimed by a provider, issues an authenticated `fetch()` against the rewritten URL with the configured token, and classifies the response per design §7 into one of five new `CODE_REGISTRY` entries: `LINK_AUTH_DEAD` (404/410 from an honest-404 host — `error`-severity, the only such code in the slice; design §7 establishes that an authenticated 404 against e.g. SharePoint is high-confidence link rot, satisfying the rule-design corpus-evidence bar), `LINK_AUTH_DEAD_OR_UNAUTHORIZED` (404 from an ambiguous host like GitHub that masks `403`s — warn), `LINK_AUTH_FORBIDDEN` (`403` — warn), `LINK_AUTH_UNAUTHORIZED` (`401` — warn, promote to `error` on strict CI lanes), and `LINK_AUTH_UNVERIFIED` (no token resolved — warn, never cached per §6.3). New files: `packages/resources/src/link-auth-fetch.ts` (`fetchAuthenticated()` — bounded redirect loop with **cross-origin `Authorization` stripping (§8)** that is sticky across the rest of the chain to defeat token-laundering, **429/`Retry-After` honoring (§5.2)** parsing both delta-seconds and HTTP-date forms with a 60s DoS cap *and* a 250ms good-neighbor floor, all dependency-injectable via `fetchImpl`/`sleep`/`signal`); `packages/resources/src/link-auth-classify.ts` (pure `(status, providerCheck) → outcome+code` per §7's table); `packages/resources/src/link-auth-config-build.ts` (bridge from adopter config to engine — runs `expandMacro` on `{ use: <macro>, ...overrides }` entries; the adopter schemas are passthrough per the repo's Postel's Law rule, so the post-expansion `InlineProviderSchema.safeParse` catches missing required fields and wrong types on declared fields but lets unknown extras through, matching how the rest of project-config treats adopter input; a compile-time `_KeysAgree` assertion locks the schema's top-level field set to the engine's `Provider` interface). New cache architecture: a second `ExternalLinkCache` instance for auth-branch results, **keyed by the rewritten URL** (the original `blob/` URL 404s — caching it would poison results) and scoped to a per-OS-user subdirectory `cacheDir/auth-${sanitizedOsUser}/` so two users on a shared CI host never read each other's authenticated results (§6.3); the OS user resolves through `os.userInfo()` → `USER`/`USERNAME` env → `'default'` with a one-shot `console.warn` on the last fallback so the cross-user-leak risk is observable. Cache entries gain an explicit `version: 1` field; reads of any other version produce a miss, so slice 3's content-cache evolution can change the entry shape without misparsing pre-existing files. Doc-anchor coverage iterator test (`packages/agent-schema/test/docs/validation-codes.test.ts`) iterates `CODE_REGISTRY` and asserts each code has a matching `### \`CODE\`` heading in `docs/validation-codes.md` plus a convention-matching `entry.reference` — 126 assertions covering all 63 codes, future-proofs the per-code docs requirement. Five new doc sections under "Authenticated External Link Codes" in `docs/validation-codes.md`. Engine surface gains: `Provider.check` now flows through on the verified `ResolveOutcome` so the classifier can route per-provider `notFoundMeaning`; `ExternalLinkValidatorOptions` gains `linkAuthConfig`, `fetchImpl`, `linkAuthDeps`, `sleep`, and `osUser` (the first two are adopter-usable for corporate-proxy/custom-TLS injection, the rest test-only); `LinkValidationResult` gains a `code?: IssueCode` field that the `resource-registry.ts` consumer prefers over the existing status-code-to-`EXTERNAL_URL_*` mapping. The validator memoizes `runCommand` results per unique argv for the duration of a `validate()` run, so validating N URLs from the same host runs `gh auth token` (and any other command-source token) at most once. **196 new tests across `link-auth-classify` (13), `link-auth-fetch` (19), `external-link-validator-auth` (25), `link-auth-config-build` (10), `validation-codes` (+126 doc-anchor iterator, +2 LINK_AUTH_* registry), and `external-link-cache` (+1 version-gate).** Security-load-bearing tests pin the cross-origin Authorization strip (case-insensitive, sticky across chains, with userinfo/relative-Location edge cases), the path-traversal sanitizer (table-driven over 9 pathological `osUser` inputs), the unverified-no-cache invariant, the cache-hit re-classification (cache hits re-run the classifier against the current provider so a `notFoundMeaning` flip between runs surfaces the new code, not the old one), the runCommand memoization (N URLs from the same provider → 1 command invocation), and an Object.hasOwn-based prototype-pollution defense on the `{ use }` discriminator in `buildLinkAuthEngineConfig`. Slice 4 (cross-platform `.cmd`-shim system test, `VAT_LINKAUTH_ALLOW_COMMAND=0` opt-out, contributor docs) and slice 3 (content-fetch primitive + content cache) are downstream.
- **linkAuth pure engine foundation (issue #113, slice 1).** Adds a config-driven engine for authenticated external URL resolution, scoped to the pure-logic layer with no consumer wiring yet (the `ExternalLinkValidator` integration and the `LINK_AUTH_*` `CODE_REGISTRY` entries are slice 2; the content-fetch primitive is slice 3). New `link-auth/` module under `@vibe-agent-toolkit/utils` with eight files: `transforms.ts` (closed allowlist — `base64url`, `urlencode`, `lower` — with `Object.hasOwn`-based prototype-chain defense), `template.ts` (tiny `${name}` / `${transform(name)}` renderer, deliberately separate from the Handlebars renderer in `utils/template.ts`), `rewrite.ts` (ordered `when → vars → to` pipeline with fragment/query stripping per design §5.2), `build-headers.ts` (header rendering plus structural `Authorization` redaction), `select-provider.ts` (host-glob matching via picomatch with `excludeHost`), `expand-macro.ts` (YAML loader + deep-merge expander), `resolve-token.ts` (ordered env / `safeExecResult`-backed argv-command sources, first-non-empty wins, no shell), and `resolve.ts` (the public `resolveAuthenticatedUrl(url, config)` entry returning one of `{fetchUrl, headers}` / `{outcome: 'unsupported'}` / `{outcome: 'unverified', reason}`). Ships the `github` and `sharepoint` macros as a YAML data asset (`src/link-auth/macros.yaml`), with a new cross-platform `packages/dev-tools/src/copy-yaml-assets.ts` post-build step bundling `.yaml` into `dist/` — first YAML-asset shipping pattern in the utils package. Adds `yaml` as a utils dependency. Companion Zod schema in `@vibe-agent-toolkit/resources` (`src/schemas/link-auth.ts`) validates the `resources.linkAuth` config block (strict; accepts either `{ use: <macro>, ...overrides }` or full inline providers), wired as an optional field on `ResourcesConfigSchema`. 140 unit tests in utils + 29 schema tests in resources, all pure-logic with no network or filesystem dependencies; security-load-bearing tests pin the closed-allowlist guarantee, the `${__proto__}` bypass defense, the token-never-leaks-into-Authorization invariant, and `shell: false` literal-argv handling.
- **Corpus seed expanded from 9 → 237 entries via a new committed importer at `packages/dev-tools/src/import-marketplace.ts` (`bun run import-marketplace [--allow-shrink]`).** The script fetches `.claude-plugin/marketplace.json` from `anthropics/claude-plugins-official` (205 of 209 raw entries kept) and `anthropics/knowledge-work-plugins` (30 of 60 — the knowledge-work catalog turns out to be ≈50% mirror entries of the official catalog) via `gh api`, maps each upstream entry to a `PluginEntry`, deduplicates by `source` URL (preserved VAT-owned entries always win; otherwise alphabetical-first-name wins within each duplicate cluster), and rewrites `corpus/seed.yaml`. Mapping rules: `bucket: official` uniformly (both catalogs are anthropics-curated marketplaces — `bucket` is the *reporting posture* per slice 1a, not code provenance); `confidence: first-party` for catalog-internal string sources and `github.com/anthropics/...` object sources, else `curated`; the `./partner-built/` knowledge-work convention overrides to `curated`; `maturity: production` for all entries. URL composition handles all five upstream source shapes (string, `git-subdir` ± `ref`, `url` ± `path`, `github`), throwing on unknown discriminators. The seven sample entries from slice 1a are regenerated from upstream manifests on every re-import. Re-import safety: the importer refuses to overwrite `corpus/seed.yaml` if either upstream catalog returned 0 plugins or the new entry count would drop more than 20% vs. the existing seed; `--allow-shrink` bypasses both gates for the rare case where shrinkage is real. The generated `seed.yaml` header dropped its earlier per-entry `validation:` claim (the importer throws on validation blocks today) and now states explicitly that entry `source` URLs pin a fragment ref (typically the default branch), not a per-entry commit SHA — the catalog SHAs in the header are this run's audit provenance. Issue #99 slice 1b — follows the schema change from PR #111 (slice 1a).
- **Empirical compatibility harness (`packages/dev-tools/src/compat-empirical/`).** Per-#100 research scaffold for measuring skill compatibility across `claude-code`, `claude-cowork`, and `claude-chat`: a CLI (`predict`/`run`/`judge`/`report`/`all`) that joins VAT's static predictions with deterministic runtime observations and an LLM-judge semantic read into a reality-vs-prediction matrix — an evidence artifact for proposing detector improvements that each cite specific (skill, runtime) cells. Probe coverage: multi-prompt + repeat-N with adaptive N=3→N=5 extension, mandatory positive+negative prompt pairing per corpus entry, and negative-prompt agreement inversion so false-positive triggers surface as `vat-optimistic`. Evidence quality: the deterministic class is widened from 6 to 9 values (splitting `error` into `install-failed`/`runtime-error`, `not-invoked` into `not-invoked-engaged`/`not-invoked-empty`, adding `refused`), with a v2 judge prompt that adds a `refused` verdict. Report fidelity: coverage stats, per-bucket headline (own/official/community × ran/agree/optimistic/pessimistic/gray-zone), gray-zone (mixed-signal) and high-variance subsections, and per-attempt variance rendered inline (`runtime-error (2/3) / failed (3/3)`). Judge replay persists `judge-calls/<skillId>-<promptId>-<target>-<attemptIdx>.json` artifacts that a new `re-judge` subcommand re-executes against an optionally different model or freshly-edited system prompt — without re-spending operator hours on the runtime side. Also landed: `git fetch --tags --force` before named-ref fetch (annotated tag refresh) and `setup()` teardown-first idempotency for the manual driver. No detector code or `RUNTIME_PROFILES` changes; lives entirely in the private `@vibe-agent-toolkit/dev-tools` package with no adopter-facing surface. Design: [the v2 harness design](./docs/research/2026-05-23-compat-empirical-harness-v2-design.md). Corpus authoring, the first real run, and the docs deliverable are the downstream work.
- **Cowork driver spike.** Added [`docs/contributing/cowork-driver-spike.md`](docs/contributing/cowork-driver-spike.md) — a time-boxed investigation (per §4a of the harness v2 design) of whether `claude-cowork` can be driven programmatically by the empirical compat harness today. Verdict: **not feasible**; cowork is a Claude Desktop app product with no public API/CLI surface. The `claude-cowork` runtime stays on `scripted-assisted` until Anthropic ships a Cowork CLI mode, Sessions API, or documented filesystem-import path. Adjacent finding (not a cowork replacement): the public-beta Skills API (`POST /v1/skills` + `container.skills[]` on `/v1/messages`) supports a fully-automatable *new* runtime — captured in the spike doc as a potential follow-up, gated on a separate design decision.
- **Subscription-only compat harness billing.** The harness now bills a Claude Pro/Max subscription instead of the API: both token-consuming surfaces (the `claude-code` runtime driver and the LLM judge) route through one shared `claude` CLI invoker (`runtimes/shared/claude-cli.ts`) that injects the operator's `CLAUDE_CODE_OAUTH_TOKEN` and deletes every API credential from the child env, so the CLI cannot fall back to API billing. The operator's own token is sourced at preflight — env var if set, otherwise an interactive prompt — so a run only ever spends the operator's personal plan. The judge was migrated off `@anthropic-ai/sdk` (dependency removed) onto the CLI, parsing a strict JSON verdict with one retry instead of the SDK's forced-tool call (`judge-system.md` now asks for a JSON object). `RunMetadata` gains `authMode` and the report methodology discloses subscription auth + parsed-not-forced verdicts. Premise (zero API billing under the OAuth token) still pending the manual smoke test.
- **First-class local HTML resources (#112).** `.html`/`.htm` files are now discovered, parsed, link- and anchor-validated, checked for well-formedness, and link-rewritten on bundle — using the same `ParseResult` contract and validation framework as markdown. A parse5-backed parser extracts `<a href>` and `<img src>` links plus `id`/`name` fragment anchors; `ResourceRegistry` routes HTML through it and persists optional `anchors`/`parseErrors` on `ResourceMetadata`. Anchor validation now uses a format-neutral fragment index (each file's markdown heading slugs or HTML `id`/`name`, with its case-matching policy carried per entry), enabling cross-format anchor checks (md↔html) with HTML ids matched case-sensitively and markdown slugs case-insensitively. A new `MALFORMED_HTML` code (default `info`) surfaces parser well-formedness diagnostics. On bundle, `<a href>`/`<img src>` values are rewritten by offset-splicing the original source (never re-serialized), so unchanged markup round-trips byte-for-byte and original attribute quoting is preserved (a rewritten value that would be unsafe unquoted is wrapped in quotes). Scope is `<a href>` + `<img src>` only; `<link>`/`<script>`/`<iframe>`/`<source srcset>`/CSS `url(...)` are deferred (asset/machinery references, not the content link graph). `<base href>` is not honored — relative hrefs resolve against the file's own directory (see the breaking note below for the `ResourceMetadataSchema` tightening that shipped with this work).
- **`DUPLICATE_RESOURCE_ID` validation code (default `error`).** When two files resolve to the same resource id after path normalization (e.g. `My Guide.md` and `my-guide.md` both → `my-guide-md`), `vat resources validate` now reports it as an `error` issue naming both files, instead of aborting the entire run with an uncaught `Duplicate resource ID` exception. Documented under [Resource Registry Codes](./docs/validation-codes.md).
- **Live audit/validate now sees source HTML links (issue #129 AC2).** `vat audit` / `vat skills validate` previously crawled `**/*.md` only, so links inside source `.html`/`.htm` files were invisible until build time. The live crawl now includes HTML (the registry already parses it via parse5), so the link-graph walker traverses HTML references and a broken local link inside a source HTML file surfaces as `LINK_MISSING_TARGET` at validate time, at parity with the built path's `PACKAGED_BROKEN_LINK`.
- **`LINK_DEFERRED_ARTIFACT` info code (issue #127, slice 2 of #129).** A `SKILL.md` link to a `files:`-declared artifact that doesn't exist yet (a dest built later, or a not-yet-created source) is no longer reported as a broken link — it downgrades from `LINK_MISSING_TARGET` to the new [`LINK_DEFERRED_ARTIFACT`](docs/validation-codes.md#link_deferred_artifact) info code at validate time, and `vat skills build` preserves and rewrites the link to the materialized dest instead of stripping it.
### Changed (breaking, pre-1.0)
- **`computeDeferredPaths` return type changed (issue #127, slice 2 of #129).** `computeDeferredPaths(files)` now returns `{ destPaths, sourcePaths }` instead of a flat `Set<string>` — a breaking API change (pre-1.0, intentional). Both `vat skills validate` and `vat skills build` now consume the deferred-path set (previously `deferredAssets` was silently dropped), and deferred dest/source paths resolve project-root-relative so the new behavior works for skills in subdirectories, not only at the project root. Plugin-local `files:` deferred paths remain out of scope for this slice (see [AC-10d](docs/architecture/skill-packaging.md#ac-10d--plugin-local-files-deferred-paths-are-out-of-scope-for-issue-127--slice-2-of-129)).
- **Directory links are now valid targets; `LINK_TARGETS_DIRECTORY` is narrowed to typed single-file slots (issue #126, slice 1 of #129).** A navigational local link that resolves to an existing directory (e.g. `[docs/](docs/)` in a ToC, README, or SKILL.md body) is no longer an error in `vat resources validate` or the skill-bundling link walk — previously any local link to a directory was a hard error. A renamed/deleted directory still fails via the ordinary broken-link path. `LINK_TARGETS_DIRECTORY` (still `error`) now fires **only** for a packaging `files:` *source* entry that resolves to a directory (the contract demands exactly one file). GitHub-style directory-index resolution (`docs/` → `docs/README.md`) is intentionally not implemented. Known limit (tracked for #129): a no-slash link such as `[Concepts](concepts)` that resolves to a directory is still treated as a file link; the slash form is the navigational case this slice covers.
- **`ResourceMetadataSchema` is now `strict()`.** Shipped with first-class HTML support (#112): the resource-metadata schema rejects unknown top-level fields instead of silently accepting them, so a typo or stale field in code that constructs `ResourceMetadata` now fails at parse time rather than passing through. Move any extra data into a recognized field or drop it.
- **Resource ids now carry a file-extension suffix.** `generateIdFromPath` appends `-<ext>` to every resource id (e.g. `guide.md` → `guide-md`, `guide.html` → `guide-html`, `README.md` → `readme-md`). This makes a markdown file and a same-stem HTML file distinct resources instead of colliding — the prerequisite for first-class HTML resources sharing a directory with their markdown source. Resource ids are internal, path-derived identifiers (never hand-authored in config or frontmatter), but anything that referenced an id by its old bare form must use the suffixed form — most visibly `vat rag query --resource-id` filters and re-indexed chunk ids (re-index to regenerate).
- **`vat resources validate` gains per-code severity configuration, and external-URL findings no longer fail the build by default.** Resource findings now use the same configurable severity framework as `vat skills`: each is a documented code (e.g. `LINK_BROKEN_FILE`, `EXTERNAL_URL_DEAD`) with a default severity, overridable per project under `resources.validation.severity` / `resources.validation.allow`. External-URL findings now default to `warning` and no longer flip the exit code (fixing a bug where they always failed the command); set their severity to `error` to restore failing. Severity now also accepts an `info` level. The never-implemented `resources.validation.checkLinks`/`checkAnchors`/`allowExternal` keys are removed.
- **`validation.severity` / `validation.allow` keys are validated against real codes.** A mistyped code key (e.g. `LNIK_OUTSIDE_PROJECT`) is now a config-load error instead of a silent no-op.
- **Corpus seed entries now require `bucket`, `confidence`, and `maturity` metadata fields.** `PluginEntrySchema` in `vat corpus scan`'s seed loader gains three required enum fields: `bucket: 'official' | 'community'`, `confidence: 'first-party' | 'curated' | 'listed'`, and `maturity: 'production' | 'experimental' | 'example'`. The bundled `corpus/seed.yaml` is updated; downstream callers running custom seeds must add the fields to every entry. `bucket` is the load-bearing discriminator (`official` entries report named findings; `community` entries are aggregate-only in follow-up work). The other two are descriptive metadata used by triage tooling.
- **`vat claude marketplace publish` no longer reports the project root `package.json` version in the CLI banner, commit message, status YAML, or CHANGELOG section lookup.** The label is now derived from the staged `marketplace.json`. Single-plugin marketplaces use the plugin's version — banner reads `Publishing marketplace "X" v0.0.4`, commit subject reads `publish v0.0.4`. Multi-plugin marketplaces drop the `v<X>` entirely — banner reads `Publishing marketplace "X"`, commit subject reads `publish X` — since the per-plugin `version` fields in the published `marketplace.json` are the source of truth for which plugin moved to which version. Two visible side-effects follow: (1) the status YAML's `published[*].version` field is now absent for multi-plugin marketplaces (previously it carried the misleading project version) — automation should read per-plugin versions from the published `marketplace.json` instead; (2) the stamped `## [X.Y.Z]` CHANGELOG lookup now uses the plugin's version rather than the project's, so a previously-ignored matching section will now be picked up as the commit body for single-plugin marketplaces. The `marketplace.json` schema's optional top-level `version` field is not yet consumed — that is a separate follow-up.
- **Adopter-facing `LinkAuthConfig` type renamed to `LinkAuthProjectConfig` (issue #113).** Both `@vibe-agent-toolkit/utils` (engine) and `@vibe-agent-toolkit/resources` (Zod-inferred adopter shape) previously exported a type named `LinkAuthConfig`, causing IDE auto-import ambiguity in any code that touched both. The adopter type — accessible as `import type { LinkAuthProjectConfig } from '@vibe-agent-toolkit/resources/schemas/link-auth'` — is the one renamed; the engine's `LinkAuthConfig` is unchanged (more API surface depends on it). Migration: rename the import. The Zod schema's name (`LinkAuthConfigSchema`) is unchanged.
- **External-link cache directory layout adds an `auth-${osUser}/` subdirectory and an entry `version: 1` field (issue #113 §6.3).** When `vat resources validate` runs with `resources.linkAuth` configured, authenticated-fetch results land under `<cacheDir>/auth-${sanitizedOsUser}/external-links.json` rather than the shared `external-links.json` used by the anonymous `markdown-link-check` path — two users on the same host (e.g. shared CI runners) cannot read each other's authenticated cache entries. All cache entries now carry an explicit `version: 1` field; entries written under a different (or missing) version are treated as a cache miss, so any pre-existing `external-links.json` triggers a one-time re-fetch on first run after upgrade. The `version` gate is forward-compat for slice 3's content-cache shape evolution.
- **`vat claude marketplace publish` no longer pushes per-plugin `<name>-v<version>` source-repo tags.** The post-publish tagging step (introduced alongside multi-plugin versioning) is removed entirely — no tags are created or pushed, and the misleading `Repository not found` / "tag already exists at a different commit" warnings it emitted on every cross-repo publish are gone ([#121](https://github.com/jdutton/vibe-agent-toolkit/issues/121)). The tags were pushed to the marketplace remote rather than a source remote, never landed anywhere useful, and there was no opt-in demand. Which plugin moved to which version is now determined solely by the per-plugin `version` fields in the published `marketplace.json`. No config key or flag is involved; if you relied on these tags, create them in your own release workflow.
### Fixed
- **`vat resources validate` no longer emits false-positive `LINK_BROKEN_ANCHOR` errors for `#fragment` links in HTML files.** HTML fragment anchors are frequently resolved at runtime by client-side JavaScript — hash routers, SPA `#/route` links, hash-encoded query params (`#id=1&mode=x`) — rather than by a literal element `id`/`name` in the markup, and ids can also be injected dynamically at runtime. A static "id not found" is therefore not proof the link is broken. Anchor resolution is now **skipped for HTML targets by default**; markdown heading-anchor validation is unchanged and still errors on a genuine miss. A new `--check-html-anchors` flag (mirroring `--check-external-urls`) opts in to strict HTML anchor resolution for fully-static pages — and even then, structural non-anchors (`#/route`, `#k=v&…`) are skipped since they can never be element ids. This restores clean `vat verify`/`vat resources validate` runs for HTML/SPA projects, reported by an external adopter whose gating CI turned red on functional runtime deep-links.
- **`vat build` now fails when a shipped Claude plugin skill has a broken packaged link.** `vat claude plugin build` never ran a post-assembly link check on the plugin output tree — only the pool packaging path did. A plugin skill whose shipped links were broken (e.g. relative links that assumed pool-packaging relocation but the skill was verbatim tree-copied) previously shipped silently. `vat build` now runs the existing depth-free `checkBrokenPackagedLinks` check against every shipped skill dir after the `claude` phase and fails the build with a `PACKAGED_BROKEN_LINK` error on any dead link. The check is scoped per skill dir — a skill is a self-contained portable unit, so a link that escapes its own directory (even to a sibling skill that co-ships in the same plugin) is a broken shipped link.
- **`vat claude plugin build` no longer double-produces a skill that is both pool-selected and present in the plugin's own `skills/` source tree.** Tree-copy (verbatim, unaware of packaging) and pool-import (packaged, link-rewritten) never coordinated — a skill claimed by both mechanisms shipped as two coexisting copies at different depths inside the same `skills/<name>/` directory, with the raw tree-copy carrying un-rewritten (and therefore potentially dead) relative links. The plugin's resolved pool selector is now excluded from the verbatim tree-copy before it runs, so the pool-packaged copy is the sole source for a colliding skill; the build prints a warning naming the skill and both sources. Non-colliding tree-copy and pool-import usage is unaffected.
- **`validateSkill` no longer silently reports a boundary-escaping AND missing link as a warning-only boundary notice.** `validateLocalLink`'s boundary-escape check returned before the existence check ever ran, so a link that both escaped the skill directory boundary and pointed at a non-existent file was classified `LINK_OUTSIDE_PROJECT` (warning) and never surfaced as broken — this is why `vat claude marketplace validate` could report a shipped tree with a dead, boundary-escaping link as 0 errors. Existence is now checked before boundary classification: a missing target is always `LINK_INTEGRITY_BROKEN` (error), regardless of whether it also escapes the boundary. A link that escapes the boundary but resolves to an existing file is unaffected (still a warning).
- **Skill-test eval-suite schema hardened after an adversarial review of the Postel liberalization.** Four issues the `id`/passthrough widening introduced or left open, all verified against the real `dxa` adopter suites in `app-platform`:
- **String eval ids are now validated as filesystem-safe path segments** (`[A-Za-z0-9_-]+`). A string `id` names a per-eval working directory, and the experimenter substitutes it verbatim into `<workspaces>/<id>`; an id like `year:extraction` previously passed parse, then failed on Windows (illegal filename) — surfacing as a *misleading* "escapes the eval directory" copy error. Rejected at parse with a clear message instead. dxa's hyphenated ids (`dollar-quote-recovery`) are unaffected.
- **Numeric `1` and string `"1"` no longer slip past the uniqueness check.** Ids are deduped on their stringified form, since both name the same workspace directory and would otherwise silently clobber each other's staged files.
- **A near-miss typo of the optional `files` field is now flagged** (`filez` → "did you mean files?"). Under plain `.passthrough()` such a typo was silently swallowed and the eval ran in an empty workspace. The check is a single-edit match scoped to recognized fields, so legitimate adopter extras (`name`, `category`, `notes`, `_category_note`) still pass through untouched.
- **`stageEvalWorkspaces` no longer mislabels copy failures as containment escapes.** Containment (`joinUnderRoot`) and the filesystem copy are now in separate try/catch blocks, so a permission/illegal-filename/disk error reports accurately instead of as "escapes the eval directory."
- **Skill-test `expected_output` is now optional, and is fed to the grader as context when present.** The pass/fail verdict is always decided per `expectations` entry, so `expected_output` is no longer required (per Postel's Law) — this unblocks real adopter suites (e.g. `dxa-consumption`) that grade with `expectations` alone. Previously the field was accepted but consumed by nothing; the experimenter prompt now passes it to the grader as the author's prose description of a correct result, informing judgment without becoming a checklist item. Still validated as a non-empty string when present.
- **`vat claude plugin build` now copies a tree-copied skill's `files:` artifacts into the distributed plugin (#127).** A skill that ships build-provided artifacts in its own directory via `files: [{ source, dest }]` but lives in a plugin's source tree was distributed by a verbatim tree-copy that skipped its `files:` step, so the shipped plugin was missing those artifacts. Build now applies each tree-copied skill's `files:` config into `skills/<name>/`, exactly as it already does for shared-pool skills — removing the need for an external inject-into-dist script (which VAT couldn't see, producing false `LINK_TO_GITIGNORED_FILE` and `missing-bundled-file` findings).
- **`vat verify` no longer false-flags skills in plugins distributed by verbatim tree-copy (`vat build --only claude`).** A plugin that ships its skills by copying its own `skills/` tree (`source:` set, `skills: []`) builds correctly, but two verify checks still assumed the shared-skill-pool model and failed a byte-correct artifact: `files-config-dests` looked for a skill's `files:` dests only under `dist/skills/<name>/` and missed the plugin tree where build actually wrote them, and `PUBLISHED_SKILL_NOT_IN_PLUGIN` was blind to `source:`, flagging every skill a tree-copy plugin ships. Both checks (and `vat build`) now agree on where a tree-copied skill lands, so the false failures are gone. (Whether private `.claude/skills/**` skills should count as "published" is unchanged and tracked separately.)
- **`ExternalLinkValidator.clearCache()` and `getCacheStats()` now operate on both caches (issue #113).** Slice 2 introduced a second cache instance for authenticated-link results (per-OS-user scoping); the existing `clearCache()` / `getCacheStats()` methods continued to touch only the anonymous cache, so an adopter rotating a token would see stale `401`/`403` entries until the auth cache TTL expired. Both methods now clear/sum across both caches.
- **`ExternalLinkCache` IO errors degrade to a cache miss instead of aborting validation (issue #113).** `loadCache()` previously threw on anything other than `ENOENT` / `SyntaxError` (e.g. `EACCES` on a permissions-restricted cache file, `EROFS` on a read-only filesystem); `saveCache()` had no try/catch (write errors propagated). A failed read / write on the status-cache file would abort the whole `vat resources validate` run. Both paths are now fail-soft: a read failure returns an empty in-memory cache, a write failure no-ops while the in-memory cache stays authoritative for the remainder of the run. Cost of a bad cache entry: one extra fetch. Cost of a bad cache entry under the previous behavior: the whole run.
- **Lazy-loaded embedding providers no longer mislabel model/runtime failures as "not installed" ([#118](https://github.com/jdutton/vibe-agent-toolkit/issues/118)).** `loadPipeline` in `transformers-embedding-provider.ts` wrapped both the dynamic `import('@xenova/transformers')` and the model download/inference in a single `catch` that always rethrew a fixed `@xenova/transformers is not installed` message, swallowing the real error (not even as `cause`) — so a model-download or `onnxruntime-node` native-backend failure on an installed package was reported as a missing dependency. The two failure modes are now separated: an import failure keeps the actionable install hint (now with the original error attached as `cause`), while a model/inference failure throws `Failed to load transformers model '<model>'` preserving `cause`. The sibling `onnx-embedding-provider.ts` was audited: its install-hint `catch` was already correctly scoped to the import alone, but its model download (`ensureModelFiles`) and session creation (`InferenceSession.create`) previously bubbled raw errors with no provider/model context, so they now throw `Failed to download ONNX model '<model>'` / `Failed to load ONNX model '<model>'` with `cause` preserved.
- **Transformers.js integration tests now skip on Windows CI instead of flaking.** `transformers-embedding-provider.integration.test.ts` and the Transformers.js block of `comparison.integration.test.ts` skip on Windows (in addition to skipping when the optional `@xenova/transformers` dependency is absent), matching the existing `onnx-embedding-provider` test. These tests download a model over the network and load the `onnxruntime-node` native backend — both flaky in Windows CI. Such a failure was previously mislabeled `@xenova/transformers is not installed` by an over-broad `catch` in the provider's `loadPipeline` (the package was installed; the model download/inference is what failed), which is also why an availability-only guard did not prevent it.
- **Config-first skill discovery now honors `..` in `skills.include` patterns.** `vat build`, `vat verify`, and `vat skills validate` all funnel through `discoverSkillsFromConfig`, which previously passed every include pattern to a single downward-only crawl rooted at `projectRoot` — so an include like `"../../docs/skills/*/SKILL.md"` (common in monorepos where SKILL.md sources live alongside, not inside, the package) silently matched zero skills. `vat audit` accepted the same config only because it has a separate filesystem-first walker. Each include pattern is now split into a literal base + glob remainder via `picomatch.scan`, patterns are grouped by their resolved absolute base, and the crawler runs once per base — making config-first discovery agree with audit. User-supplied excludes stay anchored to `projectRoot` so patterns like `docs/private/**` keep their original meaning, and a pattern resolving to a nonexistent base now silently produces zero matches.
- **Anchor validation no longer reports a false `LINK_BROKEN_ANCHOR` for un-indexed target files (#112).** Previously a fragment link to any file the resource registry had not parsed (e.g. a target outside the crawl) was reported as a broken anchor. Anchor checks now skip targets absent from the fragment index — affecting markdown and HTML alike — while genuinely missing fragments in indexed files are still reported.
- **`vat resources validate` no longer crashes on same-stem `.md` + `.html` sibling files (#116).** Making HTML first-class added `.html`/`.htm` to the crawl, and same-stem siblings (e.g. `index.md` + `index.html`) previously produced an uncaught `Duplicate resource ID` exception that aborted the whole command. Fixed by the extension-suffixed ids above (siblings now get distinct ids), with `DUPLICATE_RESOURCE_ID` as a graceful backstop for any genuine post-normalization collision.
- **Post-build link checks now cover bundled HTML (#116).** `checkBrokenPackagedLinks` and the unreferenced-file check previously scanned only `.md`, so a broken `<a href>`/`<img src>` inside a packaged `.html`/`.htm` file shipped with a green build. Both checks — and the reachability traversal — now extract HTML links via the same parser, so broken links in packaged HTML surface as `PACKAGED_BROKEN_LINK` (failing the build) and an HTML file referenced only by other HTML is no longer falsely flagged `PACKAGED_UNREFERENCED_FILE`.
- **Deferred-artifact existence parity in the link walker (issue #129 carry-forward).** `walk-link-graph`'s `checkDeferred` guarded only the `files:` *source* branch with `!existsSync`; the *dest* branch deferred unconditionally. An existing real file at a `files:` dest (e.g. a gitignored artifact already on disk) was therefore silently downgraded to the `LINK_DEFERRED_ARTIFACT` info code, masking a genuine `LINK_TO_GITIGNORED_FILE` / directory-target signal. Both branches now share the existence guard: a path is treated as deferred only when it does not yet exist on disk.
- **`computeDeferredPaths` resolves `files:` sources exactly as the packager does (issue #129 carry-forward).** The deferred-source set was computed with `resolve(projectRoot, source)`, which let an absolute-looking source escape the project root, while the packager copies with `resolve(join(projectRoot, source))`. The two now use the identical expression, so an absolute-looking source roots under the project root in both places and the deferred set matches what the build actually copies.
### Internal
- **Skill-test eval fixtures excluded from the remaining link/structure validators (CI hygiene, no adopter-facing change).** The intentionally-broken eval fixtures (`resources/skills/evals/**` — non-portable SKILL.md samples, a fake plugin for `vat audit`) are test input, not real docs/code. They were already excluded from the repo-root resource validation, ESLint, and repo-structure checks; now also from the `vat-development-agents` package config (so `vat verify`'s resources phase stops failing on the fixtures' deliberate `LINK_BROKEN_FILE`s) and the `project-validation` dogfooding system test (hardcoded exclude list). Every exclusion site cross-references the others.
- **Eval fixtures hold clean, realistic code — incidental smells removed.** Two fixtures carried code-quality issues unrelated to what their eval tests: the `release-notifier-plugin` notifier script (a payload that only needs to *exist* so `vat audit` can flag the skill's local-script dependency) now validates its `--changelog` path instead of opening it blind, and the `vat-knowledge-resources` starter config dropped a redundant `TODO` comment (the eval's prompt already states the task). Fixtures that are themselves the *subject under review* (e.g. the vat-agent-authoring analyzer the eval asks an agent to improve) keep their VAT-domain flaws by design.
- **Unified `resolveSkillSource` skill-source resolver (#132, foundation).** A `skill-source/` module in `@vibe-agent-toolkit/agent-skills` that materializes a typed source union (`workspace` / `npm` / `url(+sha256)` / `path` / `vendored`) to a hardened, content-addressed staged directory through a per-user, `0700`, uid-checked fetch cache. The git-URL parser moved from `@vibe-agent-toolkit/cli` to `@vibe-agent-toolkit/utils`. No user-facing CLI surface yet — this is the resolver consumed by `vat skill test`.
- **Authenticated external-URL resolution foundation (issue #113, slice 1).** A pure `link-auth/` engine in `@vibe-agent-toolkit/utils` (host-glob provider selection, ordered token sources with no shell, header rendering with `Authorization` redaction, `github`/`sharepoint` macros) plus a strict `resources.linkAuth` config schema in `@vibe-agent-toolkit/resources`. Not yet wired into validation — consumer integration and the `LINK_AUTH_*` codes land in later slices — so there is no user-facing behavior yet.
- **`corpus/seed.yaml` is now generated from the upstream Anthropic marketplaces (issue #99, slice 1b).** A committed importer (`bun run import-marketplace [--allow-shrink]`) fetches the `claude-plugins-official` and `knowledge-work-plugins` catalogs, deduplicates by `source` URL, and rewrites the seed — replacing the previously hand-maintained list. Re-import is guarded against accidental shrinkage (refuses to overwrite on a 0-plugin fetch or a >20% drop unless `--allow-shrink`); current entry counts and audit provenance live in the generated seed header.
- **Empirical compatibility harness (issue #100).** A research scaffold (`packages/dev-tools/src/compat-empirical/`) for measuring skill compatibility across `claude-code`, `claude-cowork`, and `claude-chat` — it joins VAT's static predictions with deterministic runtime observations and an LLM-judge read into a reality-vs-prediction matrix, as evidence for future detector improvements. Lives entirely in the private `dev-tools` package with no adopter-facing surface; no detector or `RUNTIME_PROFILES` changes. [Design](./docs/research/2026-05-23-compat-empirical-harness-v2-design.md).
- **Cowork driver spike.** [`docs/contributing/cowork-driver-spike.md`](docs/contributing/cowork-driver-spike.md) records a time-boxed finding that `claude-cowork` cannot currently be driven programmatically (no public API/CLI surface), so it stays on `scripted-assisted` in the compat harness. Notes the public-beta Skills API as a separate, fully-automatable runtime worth a future follow-up.
- **Subscription-only compat harness billing.** The compat harness now bills a Claude Pro/Max subscription via a shared `claude` CLI invoker (uses the operator's `CLAUDE_CODE_OAUTH_TOKEN` and strips all API credentials from the child env), instead of the API; the LLM judge migrated off `@anthropic-ai/sdk` onto the same CLI. Private `dev-tools` only — no adopter-facing surface.
- **Intent-aware skill-resource verdict engine (issue #129, slice 3).** Skill-resource validation now routes through a pure verdict engine (`packages/agent-skills/src/validators/rule-engine/`): `evaluate(ctx)` maps an intent-aware context to at most one validation code, and a single `materializeIssue` constructor sources severity/description/fix/reference from `CODE_REGISTRY` so docs, runtime, and tests cannot drift. This is a refactor of how the existing codes are produced — the built and live paths now share one engine instead of duplicated literals, with no change to which codes fire — guarded by a table-driven scenario harness that enforces one-code-per-context, registry equality, and an anti-workaround invariant on every code's `fix`.
- **Single-source rule catalog (issue #129 AC5).** `docs/validation-codes.md` gains a machine-readable skill-resource rule catalog (between `<!-- BEGIN:rule-catalog -->` markers) and a disambiguation map; a docs test enforces full cell-equality (severity/description/fix) with `CODE_REGISTRY` so the registry, docs, and runtime cannot drift.
github-actions Bot
pushed a commit
that referenced
this pull request
Jul 3, 2026
### Added
- **Dogfood eval suites for the whole `vat-development-agents` skill set, plus the fixes that dogfooding surfaced.** Every published VAT dev skill now ships a committed `vat skill test` eval suite (`evals/<skill>/`): `vat-audit`, `vat-skill-authoring`, `vat-knowledge-resources`, `vat-skill-distribution`, `vat-rag`, `vat-agent-authoring`, and `markdown-rewriting` (joining the existing `vat-skill-review` suite), wired via `skills.config.<skill>.test`. Final grades: vat-skill-distribution 25/25, vat-agent-authoring 24/24, vat-rag 22/22, vat-knowledge-resources 22/22, markdown-rewriting 18/18, vat-skill-authoring 21/22 (one capability-headroom miss), vat-audit 33/40 baseline A/B (the without-skill failures demonstrate the skill's lift on CI-gating/compat knowledge). Running the suites caught real skill/doc bugs, now fixed:
- **`markdown-rewriting` is now actually published.** It lived in the skills dir and `vat-skill-authoring` told agents to load `[[markdown-rewriting]]`, but the discovery glob (`vat-*.md`) didn't match its name, so it never shipped — a dangling skill reference. Added it to `skills.include` and `package.json` `vat.skills`; it now builds and ships.
- **`vat-skill-authoring`** gained the conservative-frontmatter-keys rule (the standard key set; stamp `version`/`team`/ownership under `metadata:` or in config.yaml, never as bare top-level keys) — the agent was inventing top-level `version:`/`team:` fields.
- **`vat-skill-review`** corrected a factual error: it claimed a `metadata:` field "will be rejected," but `metadata` is an allowed standard key (the sanctioned home for custom data per `SKILL_FRONTMATTER_EXTRA_FIELDS`).
- **`vat-rag`** removed a nonexistent `vat rag index --rebuild` flag (the real reset is `vat rag clear`; indexing is incremental) and added the missing `OnnxEmbeddingProvider` to the providers table.
- **`vat-knowledge-resources`** now states that `strict` mode only rejects extra fields when the schema sets `"additionalProperties": false`, and that collection validation defaults to `permissive`.
- **Collection-validation docs** corrected: `mode` defaults to `permissive` (matching `validateAgainstCollectionSchema`), not `strict` as previously documented.
- **Skill-test harness:** `buildForwardedEnv` now forwards `USER`/`LOGNAME` (see below) and eval fixtures (including intentionally-broken `.ts` files) are excluded from ESLint.
- **`vat skill test run` / `vat skill test configure` — behavioral skill testing in a context-isolated harness (#132).** Stage a packaged skill plus its declared dependencies into a throwaway, locked-down harness and run a canned, non-interactive evaluation that grades the skill against your `evals.json` (reusing skill-creator's grading rubric and JSON shapes) and writes `grading.json` (with a published [JSON Schema](docs/skill-test-grading-schema.md)), `friction.json`, and full transcripts you can inspect. `configure` writes a per-skill `test:` block to your config as a surgical edit — only the keys you pass change; surrounding formatting and comments are byte-preserved; a first `run` with no `evals.json` writes a template for you to fill in. Runs end-to-end against `claude` 2.x. **Security:** the harness runs the skill's own code with your account's privileges — it is *context* isolation, not an OS sandbox — so `run` requires `--i-understand-this-runs-skill-code`, enforced *before* anything runs (including the optional pre-stage build), and you should only test skills you trust. The pass/fail verdict is recomputed from the graded expectations, so a failing or empty grade is never silently reported as a pass; add `--fail-on-eval-failure` to make a failing eval exit non-zero and gate CI on it. See the new `vibe-agent-toolkit:vat-skill-testing` skill for auth modes, budget/turn/timeout caps, `--baseline` A/B runs, and exit codes.
- **Pre-stage `build:` hook + plugin-root staging.** An optional `test.build` command runs once before staging, so a skill that depends on a generated, un-committed artifact has it present (a non-zero build fails fast at preflight, before any tokens are spent). Plugin-distributed skills stage under their real plugin-root layout with `CLAUDE_PLUGIN_ROOT` set; standalone skills stage flat.
- **Declared test-env passthrough.** `passEnv` / `--pass-env` forwards host variables; `env` / `--env` injects values with `${fixturesDir}` / `${stagedSkillDir}` / `${harnessRoot}` / `${resultsDir}` interpolation. Both apply *after* the security allowlist — protected names always win, so committed test config can neither reroute your account credentials nor inject code: auth credentials, `PATH`, and credential-routing variables (`ANTHROPIC_BASE_URL` and the other endpoint/proxy overrides, `NODE_OPTIONS`, `NODE_EXTRA_CA_CERTS`) cannot be overridden. Fixtures under the skill's `evals/fixtures/` auto-stage with the eval tree.
- **Project-aware subject resolution.** Name a skill declared in `vibe-agent-toolkit.config.yaml` and `run` builds it first and tests the shipping **dist** — link-following, reference-rewriting, nav-stripping, and `files:` injection all applied — so you exercise exactly what installs, not the source tree. A path (including an already-built dist dir), or a `workspace:` / `npm:` / `url:` / `path:` / `vendored` source, is tested as-is; use `./<name>` to force a local directory over a colliding declared name. `--no-build` stages an existing dist without rebuilding (and errors if it is absent); `--dry-run` assembles the command without building and flags when the previewed dist may be stale, and — when no `evals.json` exists yet — reports where a real run *would* scaffold the template (exit 3) instead of writing it, so a dry run never touches your tree. A build failure fails fast at preflight (exit 2), before any tokens are spent.
- **Eval `files` are now provisioned.** Each eval's declared input files are staged into a per-eval working directory the executor operates on, enabling realistic "drop the agent in a project" evals. Files resolve relative to the `evals.json` directory and are materialized under `<harnessRoot>/workspaces/<id>/`; the experimenter prompt hands the executor that directory via a new `{{WORKSPACES_ROOT}}` token. A declared-but-missing input file fails fast at preflight (exit 2). Previously `files` was documented but inert.
- **Merge-readiness: liberal eval-suite schema, macOS subscription-auth fix, expanded skill, first dogfood suite.** (1) `evals.json` is adopter-authored data VAT *reads*, so its schema is now liberal per VAT's Postel's Law: `EvalSuiteSchema`/`EvalEntrySchema` are `.passthrough()` and `id` accepts a descriptive **string** or an int — only the fields VAT consumes (`prompt`, `expected_output`, `expectations`) stay required. This reverses the earlier strict-parser call that rejected real adopter suites three ways (string `id`, `category`, `_category_note`) and restores compatibility for the flagship adopter (app-platform/dxa). The persisted `test:` *config* block stays **strict** (it's VAT-produced config) — the deliberate inverse. (2) **macOS subscription-auth fix:** the harness env allowlist (`buildForwardedEnv`) dropped the POSIX `USER`/`LOGNAME` vars, so on macOS `claude auth status` could not read the login Keychain with the API key scrubbed — `--auth subscription` (and `inherit`'s subscription fallback) wrongly failed preflight, and the experimenter child could not authenticate. `USER`/`LOGNAME` are now forwarded (non-secret; already derivable from the forwarded `HOME`). (3) The `vibe-agent-toolkit:vat-skill-testing` skill gains a research-grounded "Authoring `evals.json`" section (blind realistic prompts, discriminating + negative expectations, categories, fixtures, `--baseline` skill-lift, grading) and a full flag⇄config knob table. (4) Ships the first committed VAT dogfood suite (`vat-skill-review`, 5 evals across catch-violation / no-false-positive / guidance-correctness) wired via `skills.config.vat-skill-review.test`, with eval fixtures excluded from `vat resources validate`.
- **`files:` entries now support glob sources and an optional `integrity` byte-verify.** A `source` containing glob magic (`*`, `**`, `?`, `[`) fans out into a directory `dest`, preserving the directory structure below the static base (glob is VAT's existing idiom, as in `skills.include` — no `recursive` flag). Globbed dests are late-bound, so `SKILL.md` links into them are treated as deferred artifacts at validate time (no `LINK_TO_GITIGNORED_FILE` allowlist needed). Add `integrity: true` to byte-verify the copy at build time and assert an exact dest subtree for glob entries.
- **`NON_PORTABLE_ASSET_REFERENCE` validation code (default `warning`) — a portability check family.** `vat skills validate` / `vat audit` now flag a skill document that references a bundled script/asset via a non-portable anchor, scanning the `SKILL.md` body **and every reachable bundled markdown doc** (agents copy invocations from reference files too). It's a family of sub-checks under one code — `claude-plugin-root`, `claude-project-dir`, and `absolute-script-path` — each finding names the variant and carries a tailored fix, and a single `validation.allow` entry silences the whole family for a file. These anchors don't exist when a skill is mounted standalone (claude.ai upload, API container), so the path breaks on the agent's first invocation; reference bundled files relative to the skill directory instead. See [`NON_PORTABLE_ASSET_REFERENCE`](docs/validation-codes.md#non_portable_asset_reference).
- **Skill-authoring guidance: portable bundled-script paths.** The `vibe-agent-toolkit:vat-skill-authoring` skill now documents how to reference bundled scripts/assets portably (relative to the skill directory, never `CLAUDE_PLUGIN_ROOT`/absolute/env-var anchors), and `vibe-agent-toolkit:vat-skill-review` carries the matching pre-publication checklist item.
- **Skill-review guidance: reserved words `claude`/`anthropic` in skill names.** The `vibe-agent-toolkit:vat-skill-review` skill's Naming section now carries the reserved-word rule as a canonical `[A]` item — Anthropic's authoring guidance states a skill `name` "Cannot contain reserved words: 'anthropic', 'claude'", and Claude Code refuses to load a non-certified skill named that way, so it fails at install/validation, not just review (`[RESERVED_WORD_IN_NAME]`). Surfaced by dogfooding the skill against its own eval suite (the reviewer was noting the prefix as "redundant" but missing the install-blocking consequence). The rule directs the reviewer to surface that consequence when reviewing such a name and to include the warning when advising on naming.
- **`NON_PORTABLE_COMMAND` validation code (default `warning`) — a portability check family.** `vat skills validate` / `vat audit` now flag a skill document that tells an agent to run a GNU/Linux-only shell command, scanning the `SKILL.md` body **and every reachable bundled markdown doc** (agents copy invocations from reference files too). It's a family of sub-checks under one code — `timeout`, `grep-pcre` (`grep -P`), `sed-i-no-backup` (`sed -i` with no suffix), `readlink-f`, and `date-d` (GNU `date -d`) — each finding names the variant and carries a tailored fix, and a single `validation.allow` entry silences the whole family for a file. Patterns match commands in command position only (not bare prose), so `grep -E`/`sed -i.bak` and nouns like "the request will timeout" are not flagged. Promotes a former manual `vat skill review` checklist line into an automated check. See [`NON_PORTABLE_COMMAND`](docs/validation-codes.md#non_portable_command).
- **linkAuth content-fetch primitive + content cache (issue #113, slice 3).** Ships the public `fetchAuthenticated(url, config, options) → { bytes, metadata, cached } | { outcome: 'unsupported' | 'unverified' }` primitive (`packages/resources/src/link-auth-content-fetch.ts`), per design §6.2 — *sibling to* the slice-2 health-check path, both reading from the same engine config and rewrite pipeline. No consumer wiring (asset-references, bundling) lands in this slice; the primitive ships standalone so future callers can adopt it without reworking the contract. **Two-mode headers (§6.2):** `Provider` gains an optional `fetch: { headers }` block alongside `auth: { headers }`; `resolveAuthenticatedUrl` now dual-expands both header sets against the same context (URL captures + resolved token), surfacing them on the success outcome as `headers` (auth, for health-check) and `fetchHeaders` (fetch, for content retrieval). The primitive merges `fetchHeaders` over `headers` so fetch-mode overrides on conflict — the canonical case being GitHub, where `Accept: application/vnd.github+json` returns 200 for any size but omits bytes >1 MiB (good for health-check) while `Accept: application/vnd.github.raw` streams the bytes inline (required for content). Adopter schema (`InlineProviderSchema`) gains a parallel `ProviderFetchSchema` (passthrough, like the rest of the adopter linkAuth tree per the repo Postel's Law rule), and the compile-time `_KeysAgree` drift check picks up the new field automatically. The resolved-token-wins precedence (URL-capture-named `token` cannot beat the resolved value) and the null-prototype hardening apply to `fetchHeaders` too. **`ContentCache` (§6.3, `packages/resources/src/content-cache.ts`):** new persistence class for the content-fetch primitive — distinct from slice 2's `ExternalLinkCache` (which is a status cache). Per-entry layout: `<sha256(rewrittenUrl)>.json` (metadata: status, content-type, etag, last-modified, fetchedAt, rewrittenUrl, `version: 1`) + `<sha256(rewrittenUrl)>.bin` (raw bytes), under a caller-supplied `cacheDir` (the validator-style `<cacheDir>/content/auth-${osUser}/` scoping is the caller's responsibility — the class only knows about the directory it was given, mirroring §6.3's "cross-user isolation = OS user, not cache key"). 30-minute default TTL (`§6.3`), tunable via constructor and via the threaded-through `resources.linkAuth.cache.ttlMinutes` adopter config (`buildLinkAuthEngineConfig` now copies the adopter `cache` block onto the engine `LinkAuthConfig`, which previously dropped it silently). Write order is `.bin` first, then `.json` as the commit marker — a partial-write crash leaves either no entry or `.bin` ahead of `.json` (reads as a miss), never `.json` ahead of `.bin` (which would serve stale bytes under new metadata). On-disk metadata fields are whitelisted via a single `pickMetadata()` helper used by both `set()` (strip smuggled fields before write) and `get()` (strip the on-disk `version` before return), so token-bearing fields a caller might smuggle through structural typing cannot land on disk — defense in depth on top of the closed `ContentMetadata` interface. TTL boundary is `>`, not `>=` — entries are valid at exactly the TTL, expire at TTL + 1 ms; tests pin both boundary cases. Fail-soft IO per #125 review: `EACCES` / `EROFS` / corrupted JSON degrade to a miss (read) or no-op (write), never throw. Forward-compat `version: 1` mirrors `ExternalLinkCache`. **Primitive behavior:** the four outcome branches — `unsupported` and `unverified` short-circuit with no fetch and **no cache touch** (§6.3: never cache `unverified`, since the result flips the moment a token appears); cache-hit returns `{ bytes, metadata, cached: true }` with no fetch; otherwise fetch via `authTransport` (cross-origin auth strip + 429 retry inherited from slice 2), read the body binary-clean via `arrayBuffer()`, build metadata from response headers (content-type / etag / last-modified default to `null` when absent), write through to the cache if supplied. `forceRefresh: true` bypasses cache reads but still writes through. `AbortSignal` propagates to the transport. The token value is interpolated into request headers in-memory and never flows into `ContentMetadata`; an end-to-end test reads every file in the cache directory after a fetch and asserts the literal token string is absent. **`wrapLinkAuthDepsWithMemo` lifted to its own module** (`packages/resources/src/link-auth-deps-memo.ts`) and exported from the resources barrel — slice 2 originally housed it private inside `external-link-validator.ts`, but the standalone primitive needs the same memoization, and the lift centralizes the implementation so jscpd cannot flag a clone. Validators and primitive callers iterating many URLs from the same provider wrap their `deps` once and reuse, so `gh auth token` / any `command`-source resolver runs at most once across the iteration. **File rename for slice 2's transport:** `packages/resources/src/link-auth-fetch.ts` → `link-auth-transport.ts`, and the exported function `fetchAuthenticated` → `authTransport` (with `AuthFetchOptions` → `AuthTransportOptions`) — frees the `fetchAuthenticated` name for the spec-documented primitive and aligns the filename with the symbol's role as the lower-level auth-safe HTTP wrapper. **47 new tests:** `link-auth-content-fetch` (15 covering short-circuits, header merge with fetch.headers override, cache hit/miss/forceRefresh, unverified-never-cached, binary-clean round-trip, signal pass-through, token-never-persisted), `content-cache` (14 covering round-trip, binary safety, distinct-URL isolation, overwrite, TTL boundary at `=` and `=+1ms`, version-mismatch eviction, corrupted-JSON tolerance, POSIX-skipped `EACCES` fail-soft on read and write, and the whitelist-on-write check), `link-auth-deps-memo` (5 covering single-source memo, distinct-argv independence, default-runCommand fallback, deps pass-through, undefined-deps handling), and 13 augmenting tests on the slice 2 surface for the new `provider.fetch` block (engine dual-expansion, schema acceptance/rejection, cache field propagation). The slice 3 primitive does not wire into any existing CLI command — `--refresh` / `--no-cache` ships with the first consumer slice.
- **linkAuth validator wiring + per-host outcome codes (issue #113, slice 2).** The slice-1 pure engine is now end-to-end: when an adopter sets `resources.linkAuth` in `vibe-agent-toolkit.config.yaml`, `vat resources validate` bypasses the anonymous `markdown-link-check` path for any URL whose host is claimed by a provider, issues an authenticated `fetch()` against the rewritten URL with the configured token, and classifies the response per design §7 into one of five new `CODE_REGISTRY` entries: `LINK_AUTH_DEAD` (404/410 from an honest-404 host — `error`-severity, the only such code in the slice; design §7 establishes that an authenticated 404 against e.g. SharePoint is high-confidence link rot, satisfying the rule-design corpus-evidence bar), `LINK_AUTH_DEAD_OR_UNAUTHORIZED` (404 from an ambiguous host like GitHub that masks `403`s — warn), `LINK_AUTH_FORBIDDEN` (`403` — warn), `LINK_AUTH_UNAUTHORIZED` (`401` — warn, promote to `error` on strict CI lanes), and `LINK_AUTH_UNVERIFIED` (no token resolved — warn, never cached per §6.3). New files: `packages/resources/src/link-auth-fetch.ts` (`fetchAuthenticated()` — bounded redirect loop with **cross-origin `Authorization` stripping (§8)** that is sticky across the rest of the chain to defeat token-laundering, **429/`Retry-After` honoring (§5.2)** parsing both delta-seconds and HTTP-date forms with a 60s DoS cap *and* a 250ms good-neighbor floor, all dependency-injectable via `fetchImpl`/`sleep`/`signal`); `packages/resources/src/link-auth-classify.ts` (pure `(status, providerCheck) → outcome+code` per §7's table); `packages/resources/src/link-auth-config-build.ts` (bridge from adopter config to engine — runs `expandMacro` on `{ use: <macro>, ...overrides }` entries; the adopter schemas are passthrough per the repo's Postel's Law rule, so the post-expansion `InlineProviderSchema.safeParse` catches missing required fields and wrong types on declared fields but lets unknown extras through, matching how the rest of project-config treats adopter input; a compile-time `_KeysAgree` assertion locks the schema's top-level field set to the engine's `Provider` interface). New cache architecture: a second `ExternalLinkCache` instance for auth-branch results, **keyed by the rewritten URL** (the original `blob/` URL 404s — caching it would poison results) and scoped to a per-OS-user subdirectory `cacheDir/auth-${sanitizedOsUser}/` so two users on a shared CI host never read each other's authenticated results (§6.3); the OS user resolves through `os.userInfo()` → `USER`/`USERNAME` env → `'default'` with a one-shot `console.warn` on the last fallback so the cross-user-leak risk is observable. Cache entries gain an explicit `version: 1` field; reads of any other version produce a miss, so slice 3's content-cache evolution can change the entry shape without misparsing pre-existing files. Doc-anchor coverage iterator test (`packages/agent-schema/test/docs/validation-codes.test.ts`) iterates `CODE_REGISTRY` and asserts each code has a matching `### \`CODE\`` heading in `docs/validation-codes.md` plus a convention-matching `entry.reference` — 126 assertions covering all 63 codes, future-proofs the per-code docs requirement. Five new doc sections under "Authenticated External Link Codes" in `docs/validation-codes.md`. Engine surface gains: `Provider.check` now flows through on the verified `ResolveOutcome` so the classifier can route per-provider `notFoundMeaning`; `ExternalLinkValidatorOptions` gains `linkAuthConfig`, `fetchImpl`, `linkAuthDeps`, `sleep`, and `osUser` (the first two are adopter-usable for corporate-proxy/custom-TLS injection, the rest test-only); `LinkValidationResult` gains a `code?: IssueCode` field that the `resource-registry.ts` consumer prefers over the existing status-code-to-`EXTERNAL_URL_*` mapping. The validator memoizes `runCommand` results per unique argv for the duration of a `validate()` run, so validating N URLs from the same host runs `gh auth token` (and any other command-source token) at most once. **196 new tests across `link-auth-classify` (13), `link-auth-fetch` (19), `external-link-validator-auth` (25), `link-auth-config-build` (10), `validation-codes` (+126 doc-anchor iterator, +2 LINK_AUTH_* registry), and `external-link-cache` (+1 version-gate).** Security-load-bearing tests pin the cross-origin Authorization strip (case-insensitive, sticky across chains, with userinfo/relative-Location edge cases), the path-traversal sanitizer (table-driven over 9 pathological `osUser` inputs), the unverified-no-cache invariant, the cache-hit re-classification (cache hits re-run the classifier against the current provider so a `notFoundMeaning` flip between runs surfaces the new code, not the old one), the runCommand memoization (N URLs from the same provider → 1 command invocation), and an Object.hasOwn-based prototype-pollution defense on the `{ use }` discriminator in `buildLinkAuthEngineConfig`. Slice 4 (cross-platform `.cmd`-shim system test, `VAT_LINKAUTH_ALLOW_COMMAND=0` opt-out, contributor docs) and slice 3 (content-fetch primitive + content cache) are downstream.
- **linkAuth pure engine foundation (issue #113, slice 1).** Adds a config-driven engine for authenticated external URL resolution, scoped to the pure-logic layer with no consumer wiring yet (the `ExternalLinkValidator` integration and the `LINK_AUTH_*` `CODE_REGISTRY` entries are slice 2; the content-fetch primitive is slice 3). New `link-auth/` module under `@vibe-agent-toolkit/utils` with eight files: `transforms.ts` (closed allowlist — `base64url`, `urlencode`, `lower` — with `Object.hasOwn`-based prototype-chain defense), `template.ts` (tiny `${name}` / `${transform(name)}` renderer, deliberately separate from the Handlebars renderer in `utils/template.ts`), `rewrite.ts` (ordered `when → vars → to` pipeline with fragment/query stripping per design §5.2), `build-headers.ts` (header rendering plus structural `Authorization` redaction), `select-provider.ts` (host-glob matching via picomatch with `excludeHost`), `expand-macro.ts` (YAML loader + deep-merge expander), `resolve-token.ts` (ordered env / `safeExecResult`-backed argv-command sources, first-non-empty wins, no shell), and `resolve.ts` (the public `resolveAuthenticatedUrl(url, config)` entry returning one of `{fetchUrl, headers}` / `{outcome: 'unsupported'}` / `{outcome: 'unverified', reason}`). Ships the `github` and `sharepoint` macros as a YAML data asset (`src/link-auth/macros.yaml`), with a new cross-platform `packages/dev-tools/src/copy-yaml-assets.ts` post-build step bundling `.yaml` into `dist/` — first YAML-asset shipping pattern in the utils package. Adds `yaml` as a utils dependency. Companion Zod schema in `@vibe-agent-toolkit/resources` (`src/schemas/link-auth.ts`) validates the `resources.linkAuth` config block (strict; accepts either `{ use: <macro>, ...overrides }` or full inline providers), wired as an optional field on `ResourcesConfigSchema`. 140 unit tests in utils + 29 schema tests in resources, all pure-logic with no network or filesystem dependencies; security-load-bearing tests pin the closed-allowlist guarantee, the `${__proto__}` bypass defense, the token-never-leaks-into-Authorization invariant, and `shell: false` literal-argv handling.
- **Corpus seed expanded from 9 → 237 entries via a new committed importer at `packages/dev-tools/src/import-marketplace.ts` (`bun run import-marketplace [--allow-shrink]`).** The script fetches `.claude-plugin/marketplace.json` from `anthropics/claude-plugins-official` (205 of 209 raw entries kept) and `anthropics/knowledge-work-plugins` (30 of 60 — the knowledge-work catalog turns out to be ≈50% mirror entries of the official catalog) via `gh api`, maps each upstream entry to a `PluginEntry`, deduplicates by `source` URL (preserved VAT-owned entries always win; otherwise alphabetical-first-name wins within each duplicate cluster), and rewrites `corpus/seed.yaml`. Mapping rules: `bucket: official` uniformly (both catalogs are anthropics-curated marketplaces — `bucket` is the *reporting posture* per slice 1a, not code provenance); `confidence: first-party` for catalog-internal string sources and `github.com/anthropics/...` object sources, else `curated`; the `./partner-built/` knowledge-work convention overrides to `curated`; `maturity: production` for all entries. URL composition handles all five upstream source shapes (string, `git-subdir` ± `ref`, `url` ± `path`, `github`), throwing on unknown discriminators. The seven sample entries from slice 1a are regenerated from upstream manifests on every re-import. Re-import safety: the importer refuses to overwrite `corpus/seed.yaml` if either upstream catalog returned 0 plugins or the new entry count would drop more than 20% vs. the existing seed; `--allow-shrink` bypasses both gates for the rare case where shrinkage is real. The generated `seed.yaml` header dropped its earlier per-entry `validation:` claim (the importer throws on validation blocks today) and now states explicitly that entry `source` URLs pin a fragment ref (typically the default branch), not a per-entry commit SHA — the catalog SHAs in the header are this run's audit provenance. Issue #99 slice 1b — follows the schema change from PR #111 (slice 1a).
- **Empirical compatibility harness (`packages/dev-tools/src/compat-empirical/`).** Per-#100 research scaffold for measuring skill compatibility across `claude-code`, `claude-cowork`, and `claude-chat`: a CLI (`predict`/`run`/`judge`/`report`/`all`) that joins VAT's static predictions with deterministic runtime observations and an LLM-judge semantic read into a reality-vs-prediction matrix — an evidence artifact for proposing detector improvements that each cite specific (skill, runtime) cells. Probe coverage: multi-prompt + repeat-N with adaptive N=3→N=5 extension, mandatory positive+negative prompt pairing per corpus entry, and negative-prompt agreement inversion so false-positive triggers surface as `vat-optimistic`. Evidence quality: the deterministic class is widened from 6 to 9 values (splitting `error` into `install-failed`/`runtime-error`, `not-invoked` into `not-invoked-engaged`/`not-invoked-empty`, adding `refused`), with a v2 judge prompt that adds a `refused` verdict. Report fidelity: coverage stats, per-bucket headline (own/official/community × ran/agree/optimistic/pessimistic/gray-zone), gray-zone (mixed-signal) and high-variance subsections, and per-attempt variance rendered inline (`runtime-error (2/3) / failed (3/3)`). Judge replay persists `judge-calls/<skillId>-<promptId>-<target>-<attemptIdx>.json` artifacts that a new `re-judge` subcommand re-executes against an optionally different model or freshly-edited system prompt — without re-spending operator hours on the runtime side. Also landed: `git fetch --tags --force` before named-ref fetch (annotated tag refresh) and `setup()` teardown-first idempotency for the manual driver. No detector code or `RUNTIME_PROFILES` changes; lives entirely in the private `@vibe-agent-toolkit/dev-tools` package with no adopter-facing surface. Design: [the v2 harness design](./docs/research/2026-05-23-compat-empirical-harness-v2-design.md). Corpus authoring, the first real run, and the docs deliverable are the downstream work.
- **Cowork driver spike.** Added [`docs/contributing/cowork-driver-spike.md`](docs/contributing/cowork-driver-spike.md) — a time-boxed investigation (per §4a of the harness v2 design) of whether `claude-cowork` can be driven programmatically by the empirical compat harness today. Verdict: **not feasible**; cowork is a Claude Desktop app product with no public API/CLI surface. The `claude-cowork` runtime stays on `scripted-assisted` until Anthropic ships a Cowork CLI mode, Sessions API, or documented filesystem-import path. Adjacent finding (not a cowork replacement): the public-beta Skills API (`POST /v1/skills` + `container.skills[]` on `/v1/messages`) supports a fully-automatable *new* runtime — captured in the spike doc as a potential follow-up, gated on a separate design decision.
- **Subscription-only compat harness billing.** The harness now bills a Claude Pro/Max subscription instead of the API: both token-consuming surfaces (the `claude-code` runtime driver and the LLM judge) route through one shared `claude` CLI invoker (`runtimes/shared/claude-cli.ts`) that injects the operator's `CLAUDE_CODE_OAUTH_TOKEN` and deletes every API credential from the child env, so the CLI cannot fall back to API billing. The operator's own token is sourced at preflight — env var if set, otherwise an interactive prompt — so a run only ever spends the operator's personal plan. The judge was migrated off `@anthropic-ai/sdk` (dependency removed) onto the CLI, parsing a strict JSON verdict with one retry instead of the SDK's forced-tool call (`judge-system.md` now asks for a JSON object). `RunMetadata` gains `authMode` and the report methodology discloses subscription auth + parsed-not-forced verdicts. Premise (zero API billing under the OAuth token) still pending the manual smoke test.
- **First-class local HTML resources (#112).** `.html`/`.htm` files are now discovered, parsed, link- and anchor-validated, checked for well-formedness, and link-rewritten on bundle — using the same `ParseResult` contract and validation framework as markdown. A parse5-backed parser extracts `<a href>` and `<img src>` links plus `id`/`name` fragment anchors; `ResourceRegistry` routes HTML through it and persists optional `anchors`/`parseErrors` on `ResourceMetadata`. Anchor validation now uses a format-neutral fragment index (each file's markdown heading slugs or HTML `id`/`name`, with its case-matching policy carried per entry), enabling cross-format anchor checks (md↔html) with HTML ids matched case-sensitively and markdown slugs case-insensitively. A new `MALFORMED_HTML` code (default `info`) surfaces parser well-formedness diagnostics. On bundle, `<a href>`/`<img src>` values are rewritten by offset-splicing the original source (never re-serialized), so unchanged markup round-trips byte-for-byte and original attribute quoting is preserved (a rewritten value that would be unsafe unquoted is wrapped in quotes). Scope is `<a href>` + `<img src>` only; `<link>`/`<script>`/`<iframe>`/`<source srcset>`/CSS `url(...)` are deferred (asset/machinery references, not the content link graph). `<base href>` is not honored — relative hrefs resolve against the file's own directory (see the breaking note below for the `ResourceMetadataSchema` tightening that shipped with this work).
- **`DUPLICATE_RESOURCE_ID` validation code (default `error`).** When two files resolve to the same resource id after path normalization (e.g. `My Guide.md` and `my-guide.md` both → `my-guide-md`), `vat resources validate` now reports it as an `error` issue naming both files, instead of aborting the entire run with an uncaught `Duplicate resource ID` exception. Documented under [Resource Registry Codes](./docs/validation-codes.md).
- **Live audit/validate now sees source HTML links (issue #129 AC2).** `vat audit` / `vat skills validate` previously crawled `**/*.md` only, so links inside source `.html`/`.htm` files were invisible until build time. The live crawl now includes HTML (the registry already parses it via parse5), so the link-graph walker traverses HTML references and a broken local link inside a source HTML file surfaces as `LINK_MISSING_TARGET` at validate time, at parity with the built path's `PACKAGED_BROKEN_LINK`.
- **`LINK_DEFERRED_ARTIFACT` info code (issue #127, slice 2 of #129).** A `SKILL.md` link to a `files:`-declared artifact that doesn't exist yet (a dest built later, or a not-yet-created source) is no longer reported as a broken link — it downgrades from `LINK_MISSING_TARGET` to the new [`LINK_DEFERRED_ARTIFACT`](docs/validation-codes.md#link_deferred_artifact) info code at validate time, and `vat skills build` preserves and rewrites the link to the materialized dest instead of stripping it.
### Changed (breaking, pre-1.0)
- **`computeDeferredPaths` return type changed (issue #127, slice 2 of #129).** `computeDeferredPaths(files)` now returns `{ destPaths, sourcePaths }` instead of a flat `Set<string>` — a breaking API change (pre-1.0, intentional). Both `vat skills validate` and `vat skills build` now consume the deferred-path set (previously `deferredAssets` was silently dropped), and deferred dest/source paths resolve project-root-relative so the new behavior works for skills in subdirectories, not only at the project root. Plugin-local `files:` deferred paths remain out of scope for this slice (see [AC-10d](docs/architecture/skill-packaging.md#ac-10d--plugin-local-files-deferred-paths-are-out-of-scope-for-issue-127--slice-2-of-129)).
- **Directory links are now valid targets; `LINK_TARGETS_DIRECTORY` is narrowed to typed single-file slots (issue #126, slice 1 of #129).** A navigational local link that resolves to an existing directory (e.g. `[docs/](docs/)` in a ToC, README, or SKILL.md body) is no longer an error in `vat resources validate` or the skill-bundling link walk — previously any local link to a directory was a hard error. A renamed/deleted directory still fails via the ordinary broken-link path. `LINK_TARGETS_DIRECTORY` (still `error`) now fires **only** for a packaging `files:` *source* entry that resolves to a directory (the contract demands exactly one file). GitHub-style directory-index resolution (`docs/` → `docs/README.md`) is intentionally not implemented. Known limit (tracked for #129): a no-slash link such as `[Concepts](concepts)` that resolves to a directory is still treated as a file link; the slash form is the navigational case this slice covers.
- **`ResourceMetadataSchema` is now `strict()`.** Shipped with first-class HTML support (#112): the resource-metadata schema rejects unknown top-level fields instead of silently accepting them, so a typo or stale field in code that constructs `ResourceMetadata` now fails at parse time rather than passing through. Move any extra data into a recognized field or drop it.
- **Resource ids now carry a file-extension suffix.** `generateIdFromPath` appends `-<ext>` to every resource id (e.g. `guide.md` → `guide-md`, `guide.html` → `guide-html`, `README.md` → `readme-md`). This makes a markdown file and a same-stem HTML file distinct resources instead of colliding — the prerequisite for first-class HTML resources sharing a directory with their markdown source. Resource ids are internal, path-derived identifiers (never hand-authored in config or frontmatter), but anything that referenced an id by its old bare form must use the suffixed form — most visibly `vat rag query --resource-id` filters and re-indexed chunk ids (re-index to regenerate).
- **`vat resources validate` gains per-code severity configuration, and external-URL findings no longer fail the build by default.** Resource findings now use the same configurable severity framework as `vat skills`: each is a documented code (e.g. `LINK_BROKEN_FILE`, `EXTERNAL_URL_DEAD`) with a default severity, overridable per project under `resources.validation.severity` / `resources.validation.allow`. External-URL findings now default to `warning` and no longer flip the exit code (fixing a bug where they always failed the command); set their severity to `error` to restore failing. Severity now also accepts an `info` level. The never-implemented `resources.validation.checkLinks`/`checkAnchors`/`allowExternal` keys are removed.
- **`validation.severity` / `validation.allow` keys are validated against real codes.** A mistyped code key (e.g. `LNIK_OUTSIDE_PROJECT`) is now a config-load error instead of a silent no-op.
- **Corpus seed entries now require `bucket`, `confidence`, and `maturity` metadata fields.** `PluginEntrySchema` in `vat corpus scan`'s seed loader gains three required enum fields: `bucket: 'official' | 'community'`, `confidence: 'first-party' | 'curated' | 'listed'`, and `maturity: 'production' | 'experimental' | 'example'`. The bundled `corpus/seed.yaml` is updated; downstream callers running custom seeds must add the fields to every entry. `bucket` is the load-bearing discriminator (`official` entries report named findings; `community` entries are aggregate-only in follow-up work). The other two are descriptive metadata used by triage tooling.
- **`vat claude marketplace publish` no longer reports the project root `package.json` version in the CLI banner, commit message, status YAML, or CHANGELOG section lookup.** The label is now derived from the staged `marketplace.json`. Single-plugin marketplaces use the plugin's version — banner reads `Publishing marketplace "X" v0.0.4`, commit subject reads `publish v0.0.4`. Multi-plugin marketplaces drop the `v<X>` entirely — banner reads `Publishing marketplace "X"`, commit subject reads `publish X` — since the per-plugin `version` fields in the published `marketplace.json` are the source of truth for which plugin moved to which version. Two visible side-effects follow: (1) the status YAML's `published[*].version` field is now absent for multi-plugin marketplaces (previously it carried the misleading project version) — automation should read per-plugin versions from the published `marketplace.json` instead; (2) the stamped `## [X.Y.Z]` CHANGELOG lookup now uses the plugin's version rather than the project's, so a previously-ignored matching section will now be picked up as the commit body for single-plugin marketplaces. The `marketplace.json` schema's optional top-level `version` field is not yet consumed — that is a separate follow-up.
- **Adopter-facing `LinkAuthConfig` type renamed to `LinkAuthProjectConfig` (issue #113).** Both `@vibe-agent-toolkit/utils` (engine) and `@vibe-agent-toolkit/resources` (Zod-inferred adopter shape) previously exported a type named `LinkAuthConfig`, causing IDE auto-import ambiguity in any code that touched both. The adopter type — accessible as `import type { LinkAuthProjectConfig } from '@vibe-agent-toolkit/resources/schemas/link-auth'` — is the one renamed; the engine's `LinkAuthConfig` is unchanged (more API surface depends on it). Migration: rename the import. The Zod schema's name (`LinkAuthConfigSchema`) is unchanged.
- **External-link cache directory layout adds an `auth-${osUser}/` subdirectory and an entry `version: 1` field (issue #113 §6.3).** When `vat resources validate` runs with `resources.linkAuth` configured, authenticated-fetch results land under `<cacheDir>/auth-${sanitizedOsUser}/external-links.json` rather than the shared `external-links.json` used by the anonymous `markdown-link-check` path — two users on the same host (e.g. shared CI runners) cannot read each other's authenticated cache entries. All cache entries now carry an explicit `version: 1` field; entries written under a different (or missing) version are treated as a cache miss, so any pre-existing `external-links.json` triggers a one-time re-fetch on first run after upgrade. The `version` gate is forward-compat for slice 3's content-cache shape evolution.
- **`vat claude marketplace publish` no longer pushes per-plugin `<name>-v<version>` source-repo tags.** The post-publish tagging step (introduced alongside multi-plugin versioning) is removed entirely — no tags are created or pushed, and the misleading `Repository not found` / "tag already exists at a different commit" warnings it emitted on every cross-repo publish are gone ([#121](https://github.com/jdutton/vibe-agent-toolkit/issues/121)). The tags were pushed to the marketplace remote rather than a source remote, never landed anywhere useful, and there was no opt-in demand. Which plugin moved to which version is now determined solely by the per-plugin `version` fields in the published `marketplace.json`. No config key or flag is involved; if you relied on these tags, create them in your own release workflow.
### Fixed
- **`vat resources validate` no longer flags inline `data:`/`blob:` resources as `LINK_UNKNOWN` warnings.** A `data:` URI embeds its own payload and a `blob:` URL references an in-memory object — neither has a target to fetch or an anchor to resolve, so there is nothing to validate. They previously fell into the "unknown link type" catch-all (any href containing `:` that wasn't `http(s)`/`mailto`) and surfaced as warnings, which is noise for the extremely common inline-image pattern (`<img src="data:image/svg+xml,…">`). A new `embedded` link type classifies them and skips validation, mirroring how `external`/`email` links are already skipped. Genuinely unrecognized schemes (`javascript:`, `tel:`, `ftp:`) still classify as `unknown`.
- **`vat resources validate` no longer emits false-positive `LINK_BROKEN_ANCHOR` errors for `#fragment` links in HTML files.** HTML fragment anchors are frequently resolved at runtime by client-side JavaScript — hash routers, SPA `#/route` links, hash-encoded query params (`#id=1&mode=x`) — rather than by a literal element `id`/`name` in the markup, and ids can also be injected dynamically at runtime. A static "id not found" is therefore not proof the link is broken. Anchor resolution is now **skipped for HTML targets by default**; markdown heading-anchor validation is unchanged and still errors on a genuine miss. A new `--check-html-anchors` flag (mirroring `--check-external-urls`) opts in to strict HTML anchor resolution for fully-static pages — and even then, structural non-anchors (`#/route`, `#k=v&…`) are skipped since they can never be element ids. This restores clean `vat verify`/`vat resources validate` runs for HTML/SPA projects, reported by an external adopter whose gating CI turned red on functional runtime deep-links.
- **`vat build` now fails when a shipped Claude plugin skill has a broken packaged link.** `vat claude plugin build` never ran a post-assembly link check on the plugin output tree — only the pool packaging path did. A plugin skill whose shipped links were broken (e.g. relative links that assumed pool-packaging relocation but the skill was verbatim tree-copied) previously shipped silently. `vat build` now runs the existing depth-free `checkBrokenPackagedLinks` check against every shipped skill dir after the `claude` phase and fails the build with a `PACKAGED_BROKEN_LINK` error on any dead link. The check is scoped per skill dir — a skill is a self-contained portable unit, so a link that escapes its own directory (even to a sibling skill that co-ships in the same plugin) is a broken shipped link.
- **`vat claude plugin build` no longer double-produces a skill that is both pool-selected and present in the plugin's own `skills/` source tree.** Tree-copy (verbatim, unaware of packaging) and pool-import (packaged, link-rewritten) never coordinated — a skill claimed by both mechanisms shipped as two coexisting copies at different depths inside the same `skills/<name>/` directory, with the raw tree-copy carrying un-rewritten (and therefore potentially dead) relative links. The plugin's resolved pool selector is now excluded from the verbatim tree-copy before it runs, so the pool-packaged copy is the sole source for a colliding skill; the build prints a warning naming the skill and both sources. Non-colliding tree-copy and pool-import usage is unaffected.
- **`validateSkill` no longer silently reports a boundary-escaping AND missing link as a warning-only boundary notice.** `validateLocalLink`'s boundary-escape check returned before the existence check ever ran, so a link that both escaped the skill directory boundary and pointed at a non-existent file was classified `LINK_OUTSIDE_PROJECT` (warning) and never surfaced as broken — this is why `vat claude marketplace validate` could report a shipped tree with a dead, boundary-escaping link as 0 errors. Existence is now checked before boundary classification: a missing target is always `LINK_INTEGRITY_BROKEN` (error), regardless of whether it also escapes the boundary. A link that escapes the boundary but resolves to an existing file is unaffected (still a warning).
- **Skill-test eval-suite schema hardened after an adversarial review of the Postel liberalization.** Four issues the `id`/passthrough widening introduced or left open, all verified against the real `dxa` adopter suites in `app-platform`:
- **String eval ids are now validated as filesystem-safe path segments** (`[A-Za-z0-9_-]+`). A string `id` names a per-eval working directory, and the experimenter substitutes it verbatim into `<workspaces>/<id>`; an id like `year:extraction` previously passed parse, then failed on Windows (illegal filename) — surfacing as a *misleading* "escapes the eval directory" copy error. Rejected at parse with a clear message instead. dxa's hyphenated ids (`dollar-quote-recovery`) are unaffected.
- **Numeric `1` and string `"1"` no longer slip past the uniqueness check.** Ids are deduped on their stringified form, since both name the same workspace directory and would otherwise silently clobber each other's staged files.
- **A near-miss typo of the optional `files` field is now flagged** (`filez` → "did you mean files?"). Under plain `.passthrough()` such a typo was silently swallowed and the eval ran in an empty workspace. The check is a single-edit match scoped to recognized fields, so legitimate adopter extras (`name`, `category`, `notes`, `_category_note`) still pass through untouched.
- **`stageEvalWorkspaces` no longer mislabels copy failures as containment escapes.** Containment (`joinUnderRoot`) and the filesystem copy are now in separate try/catch blocks, so a permission/illegal-filename/disk error reports accurately instead of as "escapes the eval directory."
- **Skill-test `expected_output` is now optional, and is fed to the grader as context when present.** The pass/fail verdict is always decided per `expectations` entry, so `expected_output` is no longer required (per Postel's Law) — this unblocks real adopter suites (e.g. `dxa-consumption`) that grade with `expectations` alone. Previously the field was accepted but consumed by nothing; the experimenter prompt now passes it to the grader as the author's prose description of a correct result, informing judgment without becoming a checklist item. Still validated as a non-empty string when present.
- **`vat claude plugin build` now copies a tree-copied skill's `files:` artifacts into the distributed plugin (#127).** A skill that ships build-provided artifacts in its own directory via `files: [{ source, dest }]` but lives in a plugin's source tree was distributed by a verbatim tree-copy that skipped its `files:` step, so the shipped plugin was missing those artifacts. Build now applies each tree-copied skill's `files:` config into `skills/<name>/`, exactly as it already does for shared-pool skills — removing the need for an external inject-into-dist script (which VAT couldn't see, producing false `LINK_TO_GITIGNORED_FILE` and `missing-bundled-file` findings).
- **`vat verify` no longer false-flags skills in plugins distributed by verbatim tree-copy (`vat build --only claude`).** A plugin that ships its skills by copying its own `skills/` tree (`source:` set, `skills: []`) builds correctly, but two verify checks still assumed the shared-skill-pool model and failed a byte-correct artifact: `files-config-dests` looked for a skill's `files:` dests only under `dist/skills/<name>/` and missed the plugin tree where build actually wrote them, and `PUBLISHED_SKILL_NOT_IN_PLUGIN` was blind to `source:`, flagging every skill a tree-copy plugin ships. Both checks (and `vat build`) now agree on where a tree-copied skill lands, so the false failures are gone. (Whether private `.claude/skills/**` skills should count as "published" is unchanged and tracked separately.)
- **`ExternalLinkValidator.clearCache()` and `getCacheStats()` now operate on both caches (issue #113).** Slice 2 introduced a second cache instance for authenticated-link results (per-OS-user scoping); the existing `clearCache()` / `getCacheStats()` methods continued to touch only the anonymous cache, so an adopter rotating a token would see stale `401`/`403` entries until the auth cache TTL expired. Both methods now clear/sum across both caches.
- **`ExternalLinkCache` IO errors degrade to a cache miss instead of aborting validation (issue #113).** `loadCache()` previously threw on anything other than `ENOENT` / `SyntaxError` (e.g. `EACCES` on a permissions-restricted cache file, `EROFS` on a read-only filesystem); `saveCache()` had no try/catch (write errors propagated). A failed read / write on the status-cache file would abort the whole `vat resources validate` run. Both paths are now fail-soft: a read failure returns an empty in-memory cache, a write failure no-ops while the in-memory cache stays authoritative for the remainder of the run. Cost of a bad cache entry: one extra fetch. Cost of a bad cache entry under the previous behavior: the whole run.
- **Lazy-loaded embedding providers no longer mislabel model/runtime failures as "not installed" ([#118](https://github.com/jdutton/vibe-agent-toolkit/issues/118)).** `loadPipeline` in `transformers-embedding-provider.ts` wrapped both the dynamic `import('@xenova/transformers')` and the model download/inference in a single `catch` that always rethrew a fixed `@xenova/transformers is not installed` message, swallowing the real error (not even as `cause`) — so a model-download or `onnxruntime-node` native-backend failure on an installed package was reported as a missing dependency. The two failure modes are now separated: an import failure keeps the actionable install hint (now with the original error attached as `cause`), while a model/inference failure throws `Failed to load transformers model '<model>'` preserving `cause`. The sibling `onnx-embedding-provider.ts` was audited: its install-hint `catch` was already correctly scoped to the import alone, but its model download (`ensureModelFiles`) and session creation (`InferenceSession.create`) previously bubbled raw errors with no provider/model context, so they now throw `Failed to download ONNX model '<model>'` / `Failed to load ONNX model '<model>'` with `cause` preserved.
- **Transformers.js integration tests now skip on Windows CI instead of flaking.** `transformers-embedding-provider.integration.test.ts` and the Transformers.js block of `comparison.integration.test.ts` skip on Windows (in addition to skipping when the optional `@xenova/transformers` dependency is absent), matching the existing `onnx-embedding-provider` test. These tests download a model over the network and load the `onnxruntime-node` native backend — both flaky in Windows CI. Such a failure was previously mislabeled `@xenova/transformers is not installed` by an over-broad `catch` in the provider's `loadPipeline` (the package was installed; the model download/inference is what failed), which is also why an availability-only guard did not prevent it.
- **Config-first skill discovery now honors `..` in `skills.include` patterns.** `vat build`, `vat verify`, and `vat skills validate` all funnel through `discoverSkillsFromConfig`, which previously passed every include pattern to a single downward-only crawl rooted at `projectRoot` — so an include like `"../../docs/skills/*/SKILL.md"` (common in monorepos where SKILL.md sources live alongside, not inside, the package) silently matched zero skills. `vat audit` accepted the same config only because it has a separate filesystem-first walker. Each include pattern is now split into a literal base + glob remainder via `picomatch.scan`, patterns are grouped by their resolved absolute base, and the crawler runs once per base — making config-first discovery agree with audit. User-supplied excludes stay anchored to `projectRoot` so patterns like `docs/private/**` keep their original meaning, and a pattern resolving to a nonexistent base now silently produces zero matches.
- **Anchor validation no longer reports a false `LINK_BROKEN_ANCHOR` for un-indexed target files (#112).** Previously a fragment link to any file the resource registry had not parsed (e.g. a target outside the crawl) was reported as a broken anchor. Anchor checks now skip targets absent from the fragment index — affecting markdown and HTML alike — while genuinely missing fragments in indexed files are still reported.
- **`vat resources validate` no longer crashes on same-stem `.md` + `.html` sibling files (#116).** Making HTML first-class added `.html`/`.htm` to the crawl, and same-stem siblings (e.g. `index.md` + `index.html`) previously produced an uncaught `Duplicate resource ID` exception that aborted the whole command. Fixed by the extension-suffixed ids above (siblings now get distinct ids), with `DUPLICATE_RESOURCE_ID` as a graceful backstop for any genuine post-normalization collision.
- **Post-build link checks now cover bundled HTML (#116).** `checkBrokenPackagedLinks` and the unreferenced-file check previously scanned only `.md`, so a broken `<a href>`/`<img src>` inside a packaged `.html`/`.htm` file shipped with a green build. Both checks — and the reachability traversal — now extract HTML links via the same parser, so broken links in packaged HTML surface as `PACKAGED_BROKEN_LINK` (failing the build) and an HTML file referenced only by other HTML is no longer falsely flagged `PACKAGED_UNREFERENCED_FILE`.
- **Deferred-artifact existence parity in the link walker (issue #129 carry-forward).** `walk-link-graph`'s `checkDeferred` guarded only the `files:` *source* branch with `!existsSync`; the *dest* branch deferred unconditionally. An existing real file at a `files:` dest (e.g. a gitignored artifact already on disk) was therefore silently downgraded to the `LINK_DEFERRED_ARTIFACT` info code, masking a genuine `LINK_TO_GITIGNORED_FILE` / directory-target signal. Both branches now share the existence guard: a path is treated as deferred only when it does not yet exist on disk.
- **`computeDeferredPaths` resolves `files:` sources exactly as the packager does (issue #129 carry-forward).** The deferred-source set was computed with `resolve(projectRoot, source)`, which let an absolute-looking source escape the project root, while the packager copies with `resolve(join(projectRoot, source))`. The two now use the identical expression, so an absolute-looking source roots under the project root in both places and the deferred set matches what the build actually copies.
### Internal
- **Skill-test eval fixtures excluded from the remaining link/structure validators (CI hygiene, no adopter-facing change).** The intentionally-broken eval fixtures (`resources/skills/evals/**` — non-portable SKILL.md samples, a fake plugin for `vat audit`) are test input, not real docs/code. They were already excluded from the repo-root resource validation, ESLint, and repo-structure checks; now also from the `vat-development-agents` package config (so `vat verify`'s resources phase stops failing on the fixtures' deliberate `LINK_BROKEN_FILE`s) and the `project-validation` dogfooding system test (hardcoded exclude list). Every exclusion site cross-references the others.
- **Eval fixtures hold clean, realistic code — incidental smells removed.** Two fixtures carried code-quality issues unrelated to what their eval tests: the `release-notifier-plugin` notifier script (a payload that only needs to *exist* so `vat audit` can flag the skill's local-script dependency) now validates its `--changelog` path instead of opening it blind, and the `vat-knowledge-resources` starter config dropped a redundant `TODO` comment (the eval's prompt already states the task). Fixtures that are themselves the *subject under review* (e.g. the vat-agent-authoring analyzer the eval asks an agent to improve) keep their VAT-domain flaws by design.
- **Unified `resolveSkillSource` skill-source resolver (#132, foundation).** A `skill-source/` module in `@vibe-agent-toolkit/agent-skills` that materializes a typed source union (`workspace` / `npm` / `url(+sha256)` / `path` / `vendored`) to a hardened, content-addressed staged directory through a per-user, `0700`, uid-checked fetch cache. The git-URL parser moved from `@vibe-agent-toolkit/cli` to `@vibe-agent-toolkit/utils`. No user-facing CLI surface yet — this is the resolver consumed by `vat skill test`.
- **Authenticated external-URL resolution foundation (issue #113, slice 1).** A pure `link-auth/` engine in `@vibe-agent-toolkit/utils` (host-glob provider selection, ordered token sources with no shell, header rendering with `Authorization` redaction, `github`/`sharepoint` macros) plus a strict `resources.linkAuth` config schema in `@vibe-agent-toolkit/resources`. Not yet wired into validation — consumer integration and the `LINK_AUTH_*` codes land in later slices — so there is no user-facing behavior yet.
- **`corpus/seed.yaml` is now generated from the upstream Anthropic marketplaces (issue #99, slice 1b).** A committed importer (`bun run import-marketplace [--allow-shrink]`) fetches the `claude-plugins-official` and `knowledge-work-plugins` catalogs, deduplicates by `source` URL, and rewrites the seed — replacing the previously hand-maintained list. Re-import is guarded against accidental shrinkage (refuses to overwrite on a 0-plugin fetch or a >20% drop unless `--allow-shrink`); current entry counts and audit provenance live in the generated seed header.
- **Empirical compatibility harness (issue #100).** A research scaffold (`packages/dev-tools/src/compat-empirical/`) for measuring skill compatibility across `claude-code`, `claude-cowork`, and `claude-chat` — it joins VAT's static predictions with deterministic runtime observations and an LLM-judge read into a reality-vs-prediction matrix, as evidence for future detector improvements. Lives entirely in the private `dev-tools` package with no adopter-facing surface; no detector or `RUNTIME_PROFILES` changes. [Design](./docs/research/2026-05-23-compat-empirical-harness-v2-design.md).
- **Cowork driver spike.** [`docs/contributing/cowork-driver-spike.md`](docs/contributing/cowork-driver-spike.md) records a time-boxed finding that `claude-cowork` cannot currently be driven programmatically (no public API/CLI surface), so it stays on `scripted-assisted` in the compat harness. Notes the public-beta Skills API as a separate, fully-automatable runtime worth a future follow-up.
- **Subscription-only compat harness billing.** The compat harness now bills a Claude Pro/Max subscription via a shared `claude` CLI invoker (uses the operator's `CLAUDE_CODE_OAUTH_TOKEN` and strips all API credentials from the child env), instead of the API; the LLM judge migrated off `@anthropic-ai/sdk` onto the same CLI. Private `dev-tools` only — no adopter-facing surface.
- **Intent-aware skill-resource verdict engine (issue #129, slice 3).** Skill-resource validation now routes through a pure verdict engine (`packages/agent-skills/src/validators/rule-engine/`): `evaluate(ctx)` maps an intent-aware context to at most one validation code, and a single `materializeIssue` constructor sources severity/description/fix/reference from `CODE_REGISTRY` so docs, runtime, and tests cannot drift. This is a refactor of how the existing codes are produced — the built and live paths now share one engine instead of duplicated literals, with no change to which codes fire — guarded by a table-driven scenario harness that enforces one-code-per-context, registry equality, and an anti-workaround invariant on every code's `fix`.
- **Single-source rule catalog (issue #129 AC5).** `docs/validation-codes.md` gains a machine-readable skill-resource rule catalog (between `<!-- BEGIN:rule-catalog -->` markers) and a disambiguation map; a docs test enforces full cell-equality (severity/description/fix) with `CODE_REGISTRY` so the registry, docs, and runtime cannot drift.
github-actions Bot
pushed a commit
that referenced
this pull request
Jul 3, 2026
### Added
- **Dogfood eval suites for the whole `vat-development-agents` skill set, plus the fixes that dogfooding surfaced.** Every published VAT dev skill now ships a committed `vat skill test` eval suite (`evals/<skill>/`): `vat-audit`, `vat-skill-authoring`, `vat-knowledge-resources`, `vat-skill-distribution`, `vat-rag`, `vat-agent-authoring`, and `markdown-rewriting` (joining the existing `vat-skill-review` suite), wired via `skills.config.<skill>.test`. Final grades: vat-skill-distribution 25/25, vat-agent-authoring 24/24, vat-rag 22/22, vat-knowledge-resources 22/22, markdown-rewriting 18/18, vat-skill-authoring 21/22 (one capability-headroom miss), vat-audit 33/40 baseline A/B (the without-skill failures demonstrate the skill's lift on CI-gating/compat knowledge). Running the suites caught real skill/doc bugs, now fixed:
- **`markdown-rewriting` is now actually published.** It lived in the skills dir and `vat-skill-authoring` told agents to load `[[markdown-rewriting]]`, but the discovery glob (`vat-*.md`) didn't match its name, so it never shipped — a dangling skill reference. Added it to `skills.include` and `package.json` `vat.skills`; it now builds and ships.
- **`vat-skill-authoring`** gained the conservative-frontmatter-keys rule (the standard key set; stamp `version`/`team`/ownership under `metadata:` or in config.yaml, never as bare top-level keys) — the agent was inventing top-level `version:`/`team:` fields.
- **`vat-skill-review`** corrected a factual error: it claimed a `metadata:` field "will be rejected," but `metadata` is an allowed standard key (the sanctioned home for custom data per `SKILL_FRONTMATTER_EXTRA_FIELDS`).
- **`vat-rag`** removed a nonexistent `vat rag index --rebuild` flag (the real reset is `vat rag clear`; indexing is incremental) and added the missing `OnnxEmbeddingProvider` to the providers table.
- **`vat-knowledge-resources`** now states that `strict` mode only rejects extra fields when the schema sets `"additionalProperties": false`, and that collection validation defaults to `permissive`.
- **Collection-validation docs** corrected: `mode` defaults to `permissive` (matching `validateAgainstCollectionSchema`), not `strict` as previously documented.
- **Skill-test harness:** `buildForwardedEnv` now forwards `USER`/`LOGNAME` (see below) and eval fixtures (including intentionally-broken `.ts` files) are excluded from ESLint.
- **`vat skill test run` / `vat skill test configure` — behavioral skill testing in a context-isolated harness (#132).** Stage a packaged skill plus its declared dependencies into a throwaway, locked-down harness and run a canned, non-interactive evaluation that grades the skill against your `evals.json` (reusing skill-creator's grading rubric and JSON shapes) and writes `grading.json` (with a published [JSON Schema](docs/skill-test-grading-schema.md)), `friction.json`, and full transcripts you can inspect. `configure` writes a per-skill `test:` block to your config as a surgical edit — only the keys you pass change; surrounding formatting and comments are byte-preserved; a first `run` with no `evals.json` writes a template for you to fill in. Runs end-to-end against `claude` 2.x. **Security:** the harness runs the skill's own code with your account's privileges — it is *context* isolation, not an OS sandbox — so `run` requires `--i-understand-this-runs-skill-code`, enforced *before* anything runs (including the optional pre-stage build), and you should only test skills you trust. The pass/fail verdict is recomputed from the graded expectations, so a failing or empty grade is never silently reported as a pass; add `--fail-on-eval-failure` to make a failing eval exit non-zero and gate CI on it. See the new `vibe-agent-toolkit:vat-skill-testing` skill for auth modes, budget/turn/timeout caps, `--baseline` A/B runs, and exit codes.
- **Pre-stage `build:` hook + plugin-root staging.** An optional `test.build` command runs once before staging, so a skill that depends on a generated, un-committed artifact has it present (a non-zero build fails fast at preflight, before any tokens are spent). Plugin-distributed skills stage under their real plugin-root layout with `CLAUDE_PLUGIN_ROOT` set; standalone skills stage flat.
- **Declared test-env passthrough.** `passEnv` / `--pass-env` forwards host variables; `env` / `--env` injects values with `${fixturesDir}` / `${stagedSkillDir}` / `${harnessRoot}` / `${resultsDir}` interpolation. Both apply *after* the security allowlist — protected names always win, so committed test config can neither reroute your account credentials nor inject code: auth credentials, `PATH`, and credential-routing variables (`ANTHROPIC_BASE_URL` and the other endpoint/proxy overrides, `NODE_OPTIONS`, `NODE_EXTRA_CA_CERTS`) cannot be overridden. Fixtures under the skill's `evals/fixtures/` auto-stage with the eval tree.
- **Project-aware subject resolution.** Name a skill declared in `vibe-agent-toolkit.config.yaml` and `run` builds it first and tests the shipping **dist** — link-following, reference-rewriting, nav-stripping, and `files:` injection all applied — so you exercise exactly what installs, not the source tree. A path (including an already-built dist dir), or a `workspace:` / `npm:` / `url:` / `path:` / `vendored` source, is tested as-is; use `./<name>` to force a local directory over a colliding declared name. `--no-build` stages an existing dist without rebuilding (and errors if it is absent); `--dry-run` assembles the command without building and flags when the previewed dist may be stale, and — when no `evals.json` exists yet — reports where a real run *would* scaffold the template (exit 3) instead of writing it, so a dry run never touches your tree. A build failure fails fast at preflight (exit 2), before any tokens are spent.
- **Eval `files` are now provisioned.** Each eval's declared input files are staged into a per-eval working directory the executor operates on, enabling realistic "drop the agent in a project" evals. Files resolve relative to the `evals.json` directory and are materialized under `<harnessRoot>/workspaces/<id>/`; the experimenter prompt hands the executor that directory via a new `{{WORKSPACES_ROOT}}` token. A declared-but-missing input file fails fast at preflight (exit 2). Previously `files` was documented but inert.
- **Merge-readiness: liberal eval-suite schema, macOS subscription-auth fix, expanded skill, first dogfood suite.** (1) `evals.json` is adopter-authored data VAT *reads*, so its schema is now liberal per VAT's Postel's Law: `EvalSuiteSchema`/`EvalEntrySchema` are `.passthrough()` and `id` accepts a descriptive **string** or an int — only the fields VAT consumes (`prompt`, `expected_output`, `expectations`) stay required. This reverses the earlier strict-parser call that rejected real adopter suites three ways (string `id`, `category`, `_category_note`) and restores compatibility for the flagship adopter (app-platform/dxa). The persisted `test:` *config* block stays **strict** (it's VAT-produced config) — the deliberate inverse. (2) **macOS subscription-auth fix:** the harness env allowlist (`buildForwardedEnv`) dropped the POSIX `USER`/`LOGNAME` vars, so on macOS `claude auth status` could not read the login Keychain with the API key scrubbed — `--auth subscription` (and `inherit`'s subscription fallback) wrongly failed preflight, and the experimenter child could not authenticate. `USER`/`LOGNAME` are now forwarded (non-secret; already derivable from the forwarded `HOME`). (3) The `vibe-agent-toolkit:vat-skill-testing` skill gains a research-grounded "Authoring `evals.json`" section (blind realistic prompts, discriminating + negative expectations, categories, fixtures, `--baseline` skill-lift, grading) and a full flag⇄config knob table. (4) Ships the first committed VAT dogfood suite (`vat-skill-review`, 5 evals across catch-violation / no-false-positive / guidance-correctness) wired via `skills.config.vat-skill-review.test`, with eval fixtures excluded from `vat resources validate`.
- **`files:` entries now support glob sources and an optional `integrity` byte-verify.** A `source` containing glob magic (`*`, `**`, `?`, `[`) fans out into a directory `dest`, preserving the directory structure below the static base (glob is VAT's existing idiom, as in `skills.include` — no `recursive` flag). Globbed dests are late-bound, so `SKILL.md` links into them are treated as deferred artifacts at validate time (no `LINK_TO_GITIGNORED_FILE` allowlist needed). Add `integrity: true` to byte-verify the copy at build time and assert an exact dest subtree for glob entries.
- **`NON_PORTABLE_ASSET_REFERENCE` validation code (default `warning`) — a portability check family.** `vat skills validate` / `vat audit` now flag a skill document that references a bundled script/asset via a non-portable anchor, scanning the `SKILL.md` body **and every reachable bundled markdown doc** (agents copy invocations from reference files too). It's a family of sub-checks under one code — `claude-plugin-root`, `claude-project-dir`, and `absolute-script-path` — each finding names the variant and carries a tailored fix, and a single `validation.allow` entry silences the whole family for a file. These anchors don't exist when a skill is mounted standalone (claude.ai upload, API container), so the path breaks on the agent's first invocation; reference bundled files relative to the skill directory instead. See [`NON_PORTABLE_ASSET_REFERENCE`](docs/validation-codes.md#non_portable_asset_reference).
- **Skill-authoring guidance: portable bundled-script paths.** The `vibe-agent-toolkit:vat-skill-authoring` skill now documents how to reference bundled scripts/assets portably (relative to the skill directory, never `CLAUDE_PLUGIN_ROOT`/absolute/env-var anchors), and `vibe-agent-toolkit:vat-skill-review` carries the matching pre-publication checklist item.
- **Skill-review guidance: reserved words `claude`/`anthropic` in skill names.** The `vibe-agent-toolkit:vat-skill-review` skill's Naming section now carries the reserved-word rule as a canonical `[A]` item — Anthropic's authoring guidance states a skill `name` "Cannot contain reserved words: 'anthropic', 'claude'", and Claude Code refuses to load a non-certified skill named that way, so it fails at install/validation, not just review (`[RESERVED_WORD_IN_NAME]`). Surfaced by dogfooding the skill against its own eval suite (the reviewer was noting the prefix as "redundant" but missing the install-blocking consequence). The rule directs the reviewer to surface that consequence when reviewing such a name and to include the warning when advising on naming.
- **`NON_PORTABLE_COMMAND` validation code (default `warning`) — a portability check family.** `vat skills validate` / `vat audit` now flag a skill document that tells an agent to run a GNU/Linux-only shell command, scanning the `SKILL.md` body **and every reachable bundled markdown doc** (agents copy invocations from reference files too). It's a family of sub-checks under one code — `timeout`, `grep-pcre` (`grep -P`), `sed-i-no-backup` (`sed -i` with no suffix), `readlink-f`, and `date-d` (GNU `date -d`) — each finding names the variant and carries a tailored fix, and a single `validation.allow` entry silences the whole family for a file. Patterns match commands in command position only (not bare prose), so `grep -E`/`sed -i.bak` and nouns like "the request will timeout" are not flagged. Promotes a former manual `vat skill review` checklist line into an automated check. See [`NON_PORTABLE_COMMAND`](docs/validation-codes.md#non_portable_command).
- **linkAuth content-fetch primitive + content cache (issue #113, slice 3).** Ships the public `fetchAuthenticated(url, config, options) → { bytes, metadata, cached } | { outcome: 'unsupported' | 'unverified' }` primitive (`packages/resources/src/link-auth-content-fetch.ts`), per design §6.2 — *sibling to* the slice-2 health-check path, both reading from the same engine config and rewrite pipeline. No consumer wiring (asset-references, bundling) lands in this slice; the primitive ships standalone so future callers can adopt it without reworking the contract. **Two-mode headers (§6.2):** `Provider` gains an optional `fetch: { headers }` block alongside `auth: { headers }`; `resolveAuthenticatedUrl` now dual-expands both header sets against the same context (URL captures + resolved token), surfacing them on the success outcome as `headers` (auth, for health-check) and `fetchHeaders` (fetch, for content retrieval). The primitive merges `fetchHeaders` over `headers` so fetch-mode overrides on conflict — the canonical case being GitHub, where `Accept: application/vnd.github+json` returns 200 for any size but omits bytes >1 MiB (good for health-check) while `Accept: application/vnd.github.raw` streams the bytes inline (required for content). Adopter schema (`InlineProviderSchema`) gains a parallel `ProviderFetchSchema` (passthrough, like the rest of the adopter linkAuth tree per the repo Postel's Law rule), and the compile-time `_KeysAgree` drift check picks up the new field automatically. The resolved-token-wins precedence (URL-capture-named `token` cannot beat the resolved value) and the null-prototype hardening apply to `fetchHeaders` too. **`ContentCache` (§6.3, `packages/resources/src/content-cache.ts`):** new persistence class for the content-fetch primitive — distinct from slice 2's `ExternalLinkCache` (which is a status cache). Per-entry layout: `<sha256(rewrittenUrl)>.json` (metadata: status, content-type, etag, last-modified, fetchedAt, rewrittenUrl, `version: 1`) + `<sha256(rewrittenUrl)>.bin` (raw bytes), under a caller-supplied `cacheDir` (the validator-style `<cacheDir>/content/auth-${osUser}/` scoping is the caller's responsibility — the class only knows about the directory it was given, mirroring §6.3's "cross-user isolation = OS user, not cache key"). 30-minute default TTL (`§6.3`), tunable via constructor and via the threaded-through `resources.linkAuth.cache.ttlMinutes` adopter config (`buildLinkAuthEngineConfig` now copies the adopter `cache` block onto the engine `LinkAuthConfig`, which previously dropped it silently). Write order is `.bin` first, then `.json` as the commit marker — a partial-write crash leaves either no entry or `.bin` ahead of `.json` (reads as a miss), never `.json` ahead of `.bin` (which would serve stale bytes under new metadata). On-disk metadata fields are whitelisted via a single `pickMetadata()` helper used by both `set()` (strip smuggled fields before write) and `get()` (strip the on-disk `version` before return), so token-bearing fields a caller might smuggle through structural typing cannot land on disk — defense in depth on top of the closed `ContentMetadata` interface. TTL boundary is `>`, not `>=` — entries are valid at exactly the TTL, expire at TTL + 1 ms; tests pin both boundary cases. Fail-soft IO per #125 review: `EACCES` / `EROFS` / corrupted JSON degrade to a miss (read) or no-op (write), never throw. Forward-compat `version: 1` mirrors `ExternalLinkCache`. **Primitive behavior:** the four outcome branches — `unsupported` and `unverified` short-circuit with no fetch and **no cache touch** (§6.3: never cache `unverified`, since the result flips the moment a token appears); cache-hit returns `{ bytes, metadata, cached: true }` with no fetch; otherwise fetch via `authTransport` (cross-origin auth strip + 429 retry inherited from slice 2), read the body binary-clean via `arrayBuffer()`, build metadata from response headers (content-type / etag / last-modified default to `null` when absent), write through to the cache if supplied. `forceRefresh: true` bypasses cache reads but still writes through. `AbortSignal` propagates to the transport. The token value is interpolated into request headers in-memory and never flows into `ContentMetadata`; an end-to-end test reads every file in the cache directory after a fetch and asserts the literal token string is absent. **`wrapLinkAuthDepsWithMemo` lifted to its own module** (`packages/resources/src/link-auth-deps-memo.ts`) and exported from the resources barrel — slice 2 originally housed it private inside `external-link-validator.ts`, but the standalone primitive needs the same memoization, and the lift centralizes the implementation so jscpd cannot flag a clone. Validators and primitive callers iterating many URLs from the same provider wrap their `deps` once and reuse, so `gh auth token` / any `command`-source resolver runs at most once across the iteration. **File rename for slice 2's transport:** `packages/resources/src/link-auth-fetch.ts` → `link-auth-transport.ts`, and the exported function `fetchAuthenticated` → `authTransport` (with `AuthFetchOptions` → `AuthTransportOptions`) — frees the `fetchAuthenticated` name for the spec-documented primitive and aligns the filename with the symbol's role as the lower-level auth-safe HTTP wrapper. **47 new tests:** `link-auth-content-fetch` (15 covering short-circuits, header merge with fetch.headers override, cache hit/miss/forceRefresh, unverified-never-cached, binary-clean round-trip, signal pass-through, token-never-persisted), `content-cache` (14 covering round-trip, binary safety, distinct-URL isolation, overwrite, TTL boundary at `=` and `=+1ms`, version-mismatch eviction, corrupted-JSON tolerance, POSIX-skipped `EACCES` fail-soft on read and write, and the whitelist-on-write check), `link-auth-deps-memo` (5 covering single-source memo, distinct-argv independence, default-runCommand fallback, deps pass-through, undefined-deps handling), and 13 augmenting tests on the slice 2 surface for the new `provider.fetch` block (engine dual-expansion, schema acceptance/rejection, cache field propagation). The slice 3 primitive does not wire into any existing CLI command — `--refresh` / `--no-cache` ships with the first consumer slice.
- **linkAuth validator wiring + per-host outcome codes (issue #113, slice 2).** The slice-1 pure engine is now end-to-end: when an adopter sets `resources.linkAuth` in `vibe-agent-toolkit.config.yaml`, `vat resources validate` bypasses the anonymous `markdown-link-check` path for any URL whose host is claimed by a provider, issues an authenticated `fetch()` against the rewritten URL with the configured token, and classifies the response per design §7 into one of five new `CODE_REGISTRY` entries: `LINK_AUTH_DEAD` (404/410 from an honest-404 host — `error`-severity, the only such code in the slice; design §7 establishes that an authenticated 404 against e.g. SharePoint is high-confidence link rot, satisfying the rule-design corpus-evidence bar), `LINK_AUTH_DEAD_OR_UNAUTHORIZED` (404 from an ambiguous host like GitHub that masks `403`s — warn), `LINK_AUTH_FORBIDDEN` (`403` — warn), `LINK_AUTH_UNAUTHORIZED` (`401` — warn, promote to `error` on strict CI lanes), and `LINK_AUTH_UNVERIFIED` (no token resolved — warn, never cached per §6.3). New files: `packages/resources/src/link-auth-fetch.ts` (`fetchAuthenticated()` — bounded redirect loop with **cross-origin `Authorization` stripping (§8)** that is sticky across the rest of the chain to defeat token-laundering, **429/`Retry-After` honoring (§5.2)** parsing both delta-seconds and HTTP-date forms with a 60s DoS cap *and* a 250ms good-neighbor floor, all dependency-injectable via `fetchImpl`/`sleep`/`signal`); `packages/resources/src/link-auth-classify.ts` (pure `(status, providerCheck) → outcome+code` per §7's table); `packages/resources/src/link-auth-config-build.ts` (bridge from adopter config to engine — runs `expandMacro` on `{ use: <macro>, ...overrides }` entries; the adopter schemas are passthrough per the repo's Postel's Law rule, so the post-expansion `InlineProviderSchema.safeParse` catches missing required fields and wrong types on declared fields but lets unknown extras through, matching how the rest of project-config treats adopter input; a compile-time `_KeysAgree` assertion locks the schema's top-level field set to the engine's `Provider` interface). New cache architecture: a second `ExternalLinkCache` instance for auth-branch results, **keyed by the rewritten URL** (the original `blob/` URL 404s — caching it would poison results) and scoped to a per-OS-user subdirectory `cacheDir/auth-${sanitizedOsUser}/` so two users on a shared CI host never read each other's authenticated results (§6.3); the OS user resolves through `os.userInfo()` → `USER`/`USERNAME` env → `'default'` with a one-shot `console.warn` on the last fallback so the cross-user-leak risk is observable. Cache entries gain an explicit `version: 1` field; reads of any other version produce a miss, so slice 3's content-cache evolution can change the entry shape without misparsing pre-existing files. Doc-anchor coverage iterator test (`packages/agent-schema/test/docs/validation-codes.test.ts`) iterates `CODE_REGISTRY` and asserts each code has a matching `### \`CODE\`` heading in `docs/validation-codes.md` plus a convention-matching `entry.reference` — 126 assertions covering all 63 codes, future-proofs the per-code docs requirement. Five new doc sections under "Authenticated External Link Codes" in `docs/validation-codes.md`. Engine surface gains: `Provider.check` now flows through on the verified `ResolveOutcome` so the classifier can route per-provider `notFoundMeaning`; `ExternalLinkValidatorOptions` gains `linkAuthConfig`, `fetchImpl`, `linkAuthDeps`, `sleep`, and `osUser` (the first two are adopter-usable for corporate-proxy/custom-TLS injection, the rest test-only); `LinkValidationResult` gains a `code?: IssueCode` field that the `resource-registry.ts` consumer prefers over the existing status-code-to-`EXTERNAL_URL_*` mapping. The validator memoizes `runCommand` results per unique argv for the duration of a `validate()` run, so validating N URLs from the same host runs `gh auth token` (and any other command-source token) at most once. **196 new tests across `link-auth-classify` (13), `link-auth-fetch` (19), `external-link-validator-auth` (25), `link-auth-config-build` (10), `validation-codes` (+126 doc-anchor iterator, +2 LINK_AUTH_* registry), and `external-link-cache` (+1 version-gate).** Security-load-bearing tests pin the cross-origin Authorization strip (case-insensitive, sticky across chains, with userinfo/relative-Location edge cases), the path-traversal sanitizer (table-driven over 9 pathological `osUser` inputs), the unverified-no-cache invariant, the cache-hit re-classification (cache hits re-run the classifier against the current provider so a `notFoundMeaning` flip between runs surfaces the new code, not the old one), the runCommand memoization (N URLs from the same provider → 1 command invocation), and an Object.hasOwn-based prototype-pollution defense on the `{ use }` discriminator in `buildLinkAuthEngineConfig`. Slice 4 (cross-platform `.cmd`-shim system test, `VAT_LINKAUTH_ALLOW_COMMAND=0` opt-out, contributor docs) and slice 3 (content-fetch primitive + content cache) are downstream.
- **linkAuth pure engine foundation (issue #113, slice 1).** Adds a config-driven engine for authenticated external URL resolution, scoped to the pure-logic layer with no consumer wiring yet (the `ExternalLinkValidator` integration and the `LINK_AUTH_*` `CODE_REGISTRY` entries are slice 2; the content-fetch primitive is slice 3). New `link-auth/` module under `@vibe-agent-toolkit/utils` with eight files: `transforms.ts` (closed allowlist — `base64url`, `urlencode`, `lower` — with `Object.hasOwn`-based prototype-chain defense), `template.ts` (tiny `${name}` / `${transform(name)}` renderer, deliberately separate from the Handlebars renderer in `utils/template.ts`), `rewrite.ts` (ordered `when → vars → to` pipeline with fragment/query stripping per design §5.2), `build-headers.ts` (header rendering plus structural `Authorization` redaction), `select-provider.ts` (host-glob matching via picomatch with `excludeHost`), `expand-macro.ts` (YAML loader + deep-merge expander), `resolve-token.ts` (ordered env / `safeExecResult`-backed argv-command sources, first-non-empty wins, no shell), and `resolve.ts` (the public `resolveAuthenticatedUrl(url, config)` entry returning one of `{fetchUrl, headers}` / `{outcome: 'unsupported'}` / `{outcome: 'unverified', reason}`). Ships the `github` and `sharepoint` macros as a YAML data asset (`src/link-auth/macros.yaml`), with a new cross-platform `packages/dev-tools/src/copy-yaml-assets.ts` post-build step bundling `.yaml` into `dist/` — first YAML-asset shipping pattern in the utils package. Adds `yaml` as a utils dependency. Companion Zod schema in `@vibe-agent-toolkit/resources` (`src/schemas/link-auth.ts`) validates the `resources.linkAuth` config block (strict; accepts either `{ use: <macro>, ...overrides }` or full inline providers), wired as an optional field on `ResourcesConfigSchema`. 140 unit tests in utils + 29 schema tests in resources, all pure-logic with no network or filesystem dependencies; security-load-bearing tests pin the closed-allowlist guarantee, the `${__proto__}` bypass defense, the token-never-leaks-into-Authorization invariant, and `shell: false` literal-argv handling.
- **Corpus seed expanded from 9 → 237 entries via a new committed importer at `packages/dev-tools/src/import-marketplace.ts` (`bun run import-marketplace [--allow-shrink]`).** The script fetches `.claude-plugin/marketplace.json` from `anthropics/claude-plugins-official` (205 of 209 raw entries kept) and `anthropics/knowledge-work-plugins` (30 of 60 — the knowledge-work catalog turns out to be ≈50% mirror entries of the official catalog) via `gh api`, maps each upstream entry to a `PluginEntry`, deduplicates by `source` URL (preserved VAT-owned entries always win; otherwise alphabetical-first-name wins within each duplicate cluster), and rewrites `corpus/seed.yaml`. Mapping rules: `bucket: official` uniformly (both catalogs are anthropics-curated marketplaces — `bucket` is the *reporting posture* per slice 1a, not code provenance); `confidence: first-party` for catalog-internal string sources and `github.com/anthropics/...` object sources, else `curated`; the `./partner-built/` knowledge-work convention overrides to `curated`; `maturity: production` for all entries. URL composition handles all five upstream source shapes (string, `git-subdir` ± `ref`, `url` ± `path`, `github`), throwing on unknown discriminators. The seven sample entries from slice 1a are regenerated from upstream manifests on every re-import. Re-import safety: the importer refuses to overwrite `corpus/seed.yaml` if either upstream catalog returned 0 plugins or the new entry count would drop more than 20% vs. the existing seed; `--allow-shrink` bypasses both gates for the rare case where shrinkage is real. The generated `seed.yaml` header dropped its earlier per-entry `validation:` claim (the importer throws on validation blocks today) and now states explicitly that entry `source` URLs pin a fragment ref (typically the default branch), not a per-entry commit SHA — the catalog SHAs in the header are this run's audit provenance. Issue #99 slice 1b — follows the schema change from PR #111 (slice 1a).
- **Empirical compatibility harness (`packages/dev-tools/src/compat-empirical/`).** Per-#100 research scaffold for measuring skill compatibility across `claude-code`, `claude-cowork`, and `claude-chat`: a CLI (`predict`/`run`/`judge`/`report`/`all`) that joins VAT's static predictions with deterministic runtime observations and an LLM-judge semantic read into a reality-vs-prediction matrix — an evidence artifact for proposing detector improvements that each cite specific (skill, runtime) cells. Probe coverage: multi-prompt + repeat-N with adaptive N=3→N=5 extension, mandatory positive+negative prompt pairing per corpus entry, and negative-prompt agreement inversion so false-positive triggers surface as `vat-optimistic`. Evidence quality: the deterministic class is widened from 6 to 9 values (splitting `error` into `install-failed`/`runtime-error`, `not-invoked` into `not-invoked-engaged`/`not-invoked-empty`, adding `refused`), with a v2 judge prompt that adds a `refused` verdict. Report fidelity: coverage stats, per-bucket headline (own/official/community × ran/agree/optimistic/pessimistic/gray-zone), gray-zone (mixed-signal) and high-variance subsections, and per-attempt variance rendered inline (`runtime-error (2/3) / failed (3/3)`). Judge replay persists `judge-calls/<skillId>-<promptId>-<target>-<attemptIdx>.json` artifacts that a new `re-judge` subcommand re-executes against an optionally different model or freshly-edited system prompt — without re-spending operator hours on the runtime side. Also landed: `git fetch --tags --force` before named-ref fetch (annotated tag refresh) and `setup()` teardown-first idempotency for the manual driver. No detector code or `RUNTIME_PROFILES` changes; lives entirely in the private `@vibe-agent-toolkit/dev-tools` package with no adopter-facing surface. Design: [the v2 harness design](./docs/research/2026-05-23-compat-empirical-harness-v2-design.md). Corpus authoring, the first real run, and the docs deliverable are the downstream work.
- **Cowork driver spike.** Added [`docs/contributing/cowork-driver-spike.md`](docs/contributing/cowork-driver-spike.md) — a time-boxed investigation (per §4a of the harness v2 design) of whether `claude-cowork` can be driven programmatically by the empirical compat harness today. Verdict: **not feasible**; cowork is a Claude Desktop app product with no public API/CLI surface. The `claude-cowork` runtime stays on `scripted-assisted` until Anthropic ships a Cowork CLI mode, Sessions API, or documented filesystem-import path. Adjacent finding (not a cowork replacement): the public-beta Skills API (`POST /v1/skills` + `container.skills[]` on `/v1/messages`) supports a fully-automatable *new* runtime — captured in the spike doc as a potential follow-up, gated on a separate design decision.
- **Subscription-only compat harness billing.** The harness now bills a Claude Pro/Max subscription instead of the API: both token-consuming surfaces (the `claude-code` runtime driver and the LLM judge) route through one shared `claude` CLI invoker (`runtimes/shared/claude-cli.ts`) that injects the operator's `CLAUDE_CODE_OAUTH_TOKEN` and deletes every API credential from the child env, so the CLI cannot fall back to API billing. The operator's own token is sourced at preflight — env var if set, otherwise an interactive prompt — so a run only ever spends the operator's personal plan. The judge was migrated off `@anthropic-ai/sdk` (dependency removed) onto the CLI, parsing a strict JSON verdict with one retry instead of the SDK's forced-tool call (`judge-system.md` now asks for a JSON object). `RunMetadata` gains `authMode` and the report methodology discloses subscription auth + parsed-not-forced verdicts. Premise (zero API billing under the OAuth token) still pending the manual smoke test.
- **First-class local HTML resources (#112).** `.html`/`.htm` files are now discovered, parsed, link- and anchor-validated, checked for well-formedness, and link-rewritten on bundle — using the same `ParseResult` contract and validation framework as markdown. A parse5-backed parser extracts `<a href>` and `<img src>` links plus `id`/`name` fragment anchors; `ResourceRegistry` routes HTML through it and persists optional `anchors`/`parseErrors` on `ResourceMetadata`. Anchor validation now uses a format-neutral fragment index (each file's markdown heading slugs or HTML `id`/`name`, with its case-matching policy carried per entry), enabling cross-format anchor checks (md↔html) with HTML ids matched case-sensitively and markdown slugs case-insensitively. A new `MALFORMED_HTML` code (default `info`) surfaces parser well-formedness diagnostics. On bundle, `<a href>`/`<img src>` values are rewritten by offset-splicing the original source (never re-serialized), so unchanged markup round-trips byte-for-byte and original attribute quoting is preserved (a rewritten value that would be unsafe unquoted is wrapped in quotes). Scope is `<a href>` + `<img src>` only; `<link>`/`<script>`/`<iframe>`/`<source srcset>`/CSS `url(...)` are deferred (asset/machinery references, not the content link graph). `<base href>` is not honored — relative hrefs resolve against the file's own directory (see the breaking note below for the `ResourceMetadataSchema` tightening that shipped with this work).
- **`DUPLICATE_RESOURCE_ID` validation code (default `error`).** When two files resolve to the same resource id after path normalization (e.g. `My Guide.md` and `my-guide.md` both → `my-guide-md`), `vat resources validate` now reports it as an `error` issue naming both files, instead of aborting the entire run with an uncaught `Duplicate resource ID` exception. Documented under [Resource Registry Codes](./docs/validation-codes.md).
- **Live audit/validate now sees source HTML links (issue #129 AC2).** `vat audit` / `vat skills validate` previously crawled `**/*.md` only, so links inside source `.html`/`.htm` files were invisible until build time. The live crawl now includes HTML (the registry already parses it via parse5), so the link-graph walker traverses HTML references and a broken local link inside a source HTML file surfaces as `LINK_MISSING_TARGET` at validate time, at parity with the built path's `PACKAGED_BROKEN_LINK`.
- **`LINK_DEFERRED_ARTIFACT` info code (issue #127, slice 2 of #129).** A `SKILL.md` link to a `files:`-declared artifact that doesn't exist yet (a dest built later, or a not-yet-created source) is no longer reported as a broken link — it downgrades from `LINK_MISSING_TARGET` to the new [`LINK_DEFERRED_ARTIFACT`](docs/validation-codes.md#link_deferred_artifact) info code at validate time, and `vat skills build` preserves and rewrites the link to the materialized dest instead of stripping it.
### Changed (breaking, pre-1.0)
- **`computeDeferredPaths` return type changed (issue #127, slice 2 of #129).** `computeDeferredPaths(files)` now returns `{ destPaths, sourcePaths }` instead of a flat `Set<string>` — a breaking API change (pre-1.0, intentional). Both `vat skills validate` and `vat skills build` now consume the deferred-path set (previously `deferredAssets` was silently dropped), and deferred dest/source paths resolve project-root-relative so the new behavior works for skills in subdirectories, not only at the project root. Plugin-local `files:` deferred paths remain out of scope for this slice (see [AC-10d](docs/architecture/skill-packaging.md#ac-10d--plugin-local-files-deferred-paths-are-out-of-scope-for-issue-127--slice-2-of-129)).
- **Directory links are now valid targets; `LINK_TARGETS_DIRECTORY` is narrowed to typed single-file slots (issue #126, slice 1 of #129).** A navigational local link that resolves to an existing directory (e.g. `[docs/](docs/)` in a ToC, README, or SKILL.md body) is no longer an error in `vat resources validate` or the skill-bundling link walk — previously any local link to a directory was a hard error. A renamed/deleted directory still fails via the ordinary broken-link path. `LINK_TARGETS_DIRECTORY` (still `error`) now fires **only** for a packaging `files:` *source* entry that resolves to a directory (the contract demands exactly one file). GitHub-style directory-index resolution (`docs/` → `docs/README.md`) is intentionally not implemented. Known limit (tracked for #129): a no-slash link such as `[Concepts](concepts)` that resolves to a directory is still treated as a file link; the slash form is the navigational case this slice covers.
- **`ResourceMetadataSchema` is now `strict()`.** Shipped with first-class HTML support (#112): the resource-metadata schema rejects unknown top-level fields instead of silently accepting them, so a typo or stale field in code that constructs `ResourceMetadata` now fails at parse time rather than passing through. Move any extra data into a recognized field or drop it.
- **Resource ids now carry a file-extension suffix.** `generateIdFromPath` appends `-<ext>` to every resource id (e.g. `guide.md` → `guide-md`, `guide.html` → `guide-html`, `README.md` → `readme-md`). This makes a markdown file and a same-stem HTML file distinct resources instead of colliding — the prerequisite for first-class HTML resources sharing a directory with their markdown source. Resource ids are internal, path-derived identifiers (never hand-authored in config or frontmatter), but anything that referenced an id by its old bare form must use the suffixed form — most visibly `vat rag query --resource-id` filters and re-indexed chunk ids (re-index to regenerate).
- **`vat resources validate` gains per-code severity configuration, and external-URL findings no longer fail the build by default.** Resource findings now use the same configurable severity framework as `vat skills`: each is a documented code (e.g. `LINK_BROKEN_FILE`, `EXTERNAL_URL_DEAD`) with a default severity, overridable per project under `resources.validation.severity` / `resources.validation.allow`. External-URL findings now default to `warning` and no longer flip the exit code (fixing a bug where they always failed the command); set their severity to `error` to restore failing. Severity now also accepts an `info` level. The never-implemented `resources.validation.checkLinks`/`checkAnchors`/`allowExternal` keys are removed.
- **`validation.severity` / `validation.allow` keys are validated against real codes.** A mistyped code key (e.g. `LNIK_OUTSIDE_PROJECT`) is now a config-load error instead of a silent no-op.
- **Corpus seed entries now require `bucket`, `confidence`, and `maturity` metadata fields.** `PluginEntrySchema` in `vat corpus scan`'s seed loader gains three required enum fields: `bucket: 'official' | 'community'`, `confidence: 'first-party' | 'curated' | 'listed'`, and `maturity: 'production' | 'experimental' | 'example'`. The bundled `corpus/seed.yaml` is updated; downstream callers running custom seeds must add the fields to every entry. `bucket` is the load-bearing discriminator (`official` entries report named findings; `community` entries are aggregate-only in follow-up work). The other two are descriptive metadata used by triage tooling.
- **`vat claude marketplace publish` no longer reports the project root `package.json` version in the CLI banner, commit message, status YAML, or CHANGELOG section lookup.** The label is now derived from the staged `marketplace.json`. Single-plugin marketplaces use the plugin's version — banner reads `Publishing marketplace "X" v0.0.4`, commit subject reads `publish v0.0.4`. Multi-plugin marketplaces drop the `v<X>` entirely — banner reads `Publishing marketplace "X"`, commit subject reads `publish X` — since the per-plugin `version` fields in the published `marketplace.json` are the source of truth for which plugin moved to which version. Two visible side-effects follow: (1) the status YAML's `published[*].version` field is now absent for multi-plugin marketplaces (previously it carried the misleading project version) — automation should read per-plugin versions from the published `marketplace.json` instead; (2) the stamped `## [X.Y.Z]` CHANGELOG lookup now uses the plugin's version rather than the project's, so a previously-ignored matching section will now be picked up as the commit body for single-plugin marketplaces. The `marketplace.json` schema's optional top-level `version` field is not yet consumed — that is a separate follow-up.
- **Adopter-facing `LinkAuthConfig` type renamed to `LinkAuthProjectConfig` (issue #113).** Both `@vibe-agent-toolkit/utils` (engine) and `@vibe-agent-toolkit/resources` (Zod-inferred adopter shape) previously exported a type named `LinkAuthConfig`, causing IDE auto-import ambiguity in any code that touched both. The adopter type — accessible as `import type { LinkAuthProjectConfig } from '@vibe-agent-toolkit/resources/schemas/link-auth'` — is the one renamed; the engine's `LinkAuthConfig` is unchanged (more API surface depends on it). Migration: rename the import. The Zod schema's name (`LinkAuthConfigSchema`) is unchanged.
- **External-link cache directory layout adds an `auth-${osUser}/` subdirectory and an entry `version: 1` field (issue #113 §6.3).** When `vat resources validate` runs with `resources.linkAuth` configured, authenticated-fetch results land under `<cacheDir>/auth-${sanitizedOsUser}/external-links.json` rather than the shared `external-links.json` used by the anonymous `markdown-link-check` path — two users on the same host (e.g. shared CI runners) cannot read each other's authenticated cache entries. All cache entries now carry an explicit `version: 1` field; entries written under a different (or missing) version are treated as a cache miss, so any pre-existing `external-links.json` triggers a one-time re-fetch on first run after upgrade. The `version` gate is forward-compat for slice 3's content-cache shape evolution.
- **`vat claude marketplace publish` no longer pushes per-plugin `<name>-v<version>` source-repo tags.** The post-publish tagging step (introduced alongside multi-plugin versioning) is removed entirely — no tags are created or pushed, and the misleading `Repository not found` / "tag already exists at a different commit" warnings it emitted on every cross-repo publish are gone ([#121](https://github.com/jdutton/vibe-agent-toolkit/issues/121)). The tags were pushed to the marketplace remote rather than a source remote, never landed anywhere useful, and there was no opt-in demand. Which plugin moved to which version is now determined solely by the per-plugin `version` fields in the published `marketplace.json`. No config key or flag is involved; if you relied on these tags, create them in your own release workflow.
### Fixed
- **`vat skill test` now verifies a per-run integrity nonce on `grading.json`, so untrusted skill code can no longer forge a passing result.** The experimenter writes `grading.json` into the harness results dir, which is inside the skill-writable sandbox (`--add-dir`, `bypassPermissions`) — an adversarial skill could therefore write or rewrite a passing `grading.json` that the harness would trust. The harness now stamps a secret per-run nonce into the experimenter prompt and requires `grading.json` to echo it in a top-level `runNonce`; a missing or mismatched nonce is rejected (exit 1). The nonce is delivered to the experimenter **only via stdin** — the prompt is no longer written to disk (`spawnHeadlessClaude` takes the prompt in memory), and the persisted `experimenter-prompt.txt` audit copy is redacted — so skill code cannot read the nonce back to forge a match. Surfaced by adversarial review; the harness still runs behind the `--i-understand-this-runs-skill-code` ack (this is defense-in-depth, not an OS sandbox).
- **A committed `test.*` config can no longer RAISE `vat skill test`'s built-in cost/runtime caps — only lower them.** `test.maxBudgetUsd` / `test.maxTurns` / `test.timeout` fed the same precedence slot as the CLI flags, so a subject repo you were merely testing could commit `maxBudgetUsd: 100` and silently bill a run far above the built-in $5 / 50-turn / 5-minute ceilings. A config-sourced value is now clamped to the built-in cap (with a one-line stderr note when clamped); a CLI flag, being explicit operator intent, may still exceed it. Surfaced by adversarial review.
- **`vat skill test` no longer cross-wires two injected plugins that share a directory basename.** The staged plugin-root dir was keyed on `basename(pluginDir)` alone, so two different `--with` plugins at e.g. `…/a/my-plugin` and `…/b/my-plugin` collided onto one staged root — the second silently inherited the first's `CLAUDE_PLUGIN_ROOT` and `.claude-plugin/` manifest, producing a misleading result. The staged segment is now keyed on the full resolved plugin path (basename kept as the readable slug, disambiguated by a hash of the full path).
- **A `files:` entry's `integrity: true` byte check is no longer silently skipped when the file was already link-bundled.** In `applyFilesConfig`, a non-glob entry whose source had already been materialized by link traversal short-circuited past the integrity verification — so a requested byte check simply didn't run for that file. The byte check now runs against the link-bundled dest (which lands at `entry.dest`) on the skip path, exactly as it would on the copy path.
- **A broken `vibe-agent-toolkit.config.yaml` is no longer silently ignored by `vat skill review` / `vat skill test` (regression fix).** The shared config walk-up (`loadConfigCached`, via `resolveSkillPackagingConfig`) swallowed a *present-but-broken* config to `undefined` — indistinguishable from "no config." That silently downgraded `vat skill review` (which previously errored on a bad config through the throwing `loadConfig`) and would let `vat skill test` apply defaults / stage the wrong subject against a config the author clearly intended. A broken config now raises a typed `ConfigLoadError` that skill-resolving commands surface (review reports it; `vat skill test` exits 2 with a clean message), while `vat audit` — a bulk linter that must keep scanning — explicitly catches it and falls back to config-free validation. An *absent* config still resolves to `undefined` as before. The error is cached so a broken config re-throws without re-parsing across a multi-skill scan.
- **`vat skill test run` now rejects a bad usage flag with a clean message and preflight exit code (2) instead of a raw stack trace.** Flag validation and config loading (`--auth`/`--require-auth` values, numeric `--max-turns`/`--max-budget-usd`/`--timeout`/`--stall`, the persisted test config) ran *before* the command's first `try`, so a malformed flag surfaced as an unhandled promise rejection (stack dump, exit 1). They now run inside a preflight guard: an unrecognized value prints `Error: --auth must be one of: …` and exits 2 without ever reaching the harness. `--auth`/`--require-auth` are validated on the run path too (previously only `configure` checked them), via a shared `auth-flags` helper so the two commands cannot drift. The `--dry-run` help no longer claims to print "the exact assembled command" (it shows the model flag; budget/turns/permission flags are added at spawn time).
- **`vat skill test`'s scrubbed-env deny-list now blocks the OS-linker and Node module-resolution code-injection vars.** A skill-under-test's committed config can forward named host env vars into the headless `claude` child via `test.passEnv`/`test.env`. The deny-list already refused `NODE_OPTIONS`/`NODE_EXTRA_CA_CERTS` (code injection before any userland code runs) but not their exact siblings — `LD_PRELOAD`, `LD_LIBRARY_PATH`, `DYLD_INSERT_LIBRARIES`, `DYLD_LIBRARY_PATH` (native `.so`/`.dylib` injection), `NODE_PATH` (module-resolution hijack), and `GIT_SSH_COMMAND` (arbitrary command on a `git:` source clone). These are now deny-only: a config naming one is ignored with a warning, the protected value wins. (The feature already runs behind an explicit `--i-understand-this-runs-skill-code` ack and a loud security warning; this closes a defense-in-depth gap surfaced by adversarial review.)
- **`vat resources validate` no longer flags inline `data:`/`blob:` resources as `LINK_UNKNOWN` warnings.** A `data:` URI embeds its own payload and a `blob:` URL references an in-memory object — neither has a target to fetch or an anchor to resolve, so there is nothing to validate. They previously fell into the "unknown link type" catch-all (any href containing `:` that wasn't `http(s)`/`mailto`) and surfaced as warnings, which is noise for the extremely common inline-image pattern (`<img src="data:image/svg+xml,…">`). A new `embedded` link type classifies them and skips validation, mirroring how `external`/`email` links are already skipped. Genuinely unrecognized schemes (`javascript:`, `tel:`, `ftp:`) still classify as `unknown`.
- **`vat resources validate` no longer emits false-positive `LINK_BROKEN_ANCHOR` errors for `#fragment` links in HTML files.** HTML fragment anchors are frequently resolved at runtime by client-side JavaScript — hash routers, SPA `#/route` links, hash-encoded query params (`#id=1&mode=x`) — rather than by a literal element `id`/`name` in the markup, and ids can also be injected dynamically at runtime. A static "id not found" is therefore not proof the link is broken. Anchor resolution is now **skipped for HTML targets by default**; markdown heading-anchor validation is unchanged and still errors on a genuine miss. A new `--check-html-anchors` flag (mirroring `--check-external-urls`) opts in to strict HTML anchor resolution for fully-static pages — and even then, structural non-anchors (`#/route`, `#k=v&…`) are skipped since they can never be element ids. This restores clean `vat verify`/`vat resources validate` runs for HTML/SPA projects, reported by an external adopter whose gating CI turned red on functional runtime deep-links.
- **`vat build` now fails when a shipped Claude plugin skill has a broken packaged link.** `vat claude plugin build` never ran a post-assembly link check on the plugin output tree — only the pool packaging path did. A plugin skill whose shipped links were broken (e.g. relative links that assumed pool-packaging relocation but the skill was verbatim tree-copied) previously shipped silently. `vat build` now runs the existing depth-free `checkBrokenPackagedLinks` check against every shipped skill dir after the `claude` phase and fails the build with a `PACKAGED_BROKEN_LINK` error on any dead link. The check is scoped per skill dir — a skill is a self-contained portable unit, so a link that escapes its own directory (even to a sibling skill that co-ships in the same plugin) is a broken shipped link.
- **`vat claude plugin build` no longer double-produces a skill that is both pool-selected and present in the plugin's own `skills/` source tree.** Tree-copy (verbatim, unaware of packaging) and pool-import (packaged, link-rewritten) never coordinated — a skill claimed by both mechanisms shipped as two coexisting copies at different depths inside the same `skills/<name>/` directory, with the raw tree-copy carrying un-rewritten (and therefore potentially dead) relative links. The plugin's resolved pool selector is now excluded from the verbatim tree-copy before it runs, so the pool-packaged copy is the sole source for a colliding skill; the build prints a warning naming the skill and both sources. Non-colliding tree-copy and pool-import usage is unaffected.
- **`validateSkill` no longer silently reports a boundary-escaping AND missing link as a warning-only boundary notice.** `validateLocalLink`'s boundary-escape check returned before the existence check ever ran, so a link that both escaped the skill directory boundary and pointed at a non-existent file was classified `LINK_OUTSIDE_PROJECT` (warning) and never surfaced as broken — this is why `vat claude marketplace validate` could report a shipped tree with a dead, boundary-escaping link as 0 errors. Existence is now checked before boundary classification: a missing target is always `LINK_INTEGRITY_BROKEN` (error), regardless of whether it also escapes the boundary. A link that escapes the boundary but resolves to an existing file is unaffected (still a warning).
- **Skill-test eval-suite schema hardened after an adversarial review of the Postel liberalization.** Four issues the `id`/passthrough widening introduced or left open, all verified against the real `dxa` adopter suites in `app-platform`:
- **String eval ids are now validated as filesystem-safe path segments** (`[A-Za-z0-9_-]+`). A string `id` names a per-eval working directory, and the experimenter substitutes it verbatim into `<workspaces>/<id>`; an id like `year:extraction` previously passed parse, then failed on Windows (illegal filename) — surfacing as a *misleading* "escapes the eval directory" copy error. Rejected at parse with a clear message instead. dxa's hyphenated ids (`dollar-quote-recovery`) are unaffected.
- **Numeric `1` and string `"1"` no longer slip past the uniqueness check.** Ids are deduped on their stringified form, since both name the same workspace directory and would otherwise silently clobber each other's staged files.
- **A near-miss typo of the optional `files` field is now flagged** (`filez` → "did you mean files?"). Under plain `.passthrough()` such a typo was silently swallowed and the eval ran in an empty workspace. The check is a single-edit match scoped to recognized fields, so legitimate adopter extras (`name`, `category`, `notes`, `_category_note`) still pass through untouched.
- **`stageEvalWorkspaces` no longer mislabels copy failures as containment escapes.** Containment (`joinUnderRoot`) and the filesystem copy are now in separate try/catch blocks, so a permission/illegal-filename/disk error reports accurately instead of as "escapes the eval directory."
- **Skill-test `expected_output` is now optional, and is fed to the grader as context when present.** The pass/fail verdict is always decided per `expectations` entry, so `expected_output` is no longer required (per Postel's Law) — this unblocks real adopter suites (e.g. `dxa-consumption`) that grade with `expectations` alone. Previously the field was accepted but consumed by nothing; the experimenter prompt now passes it to the grader as the author's prose description of a correct result, informing judgment without becoming a checklist item. Still validated as a non-empty string when present.
- **`vat claude plugin build` now copies a tree-copied skill's `files:` artifacts into the distributed plugin (#127).** A skill that ships build-provided artifacts in its own directory via `files: [{ source, dest }]` but lives in a plugin's source tree was distributed by a verbatim tree-copy that skipped its `files:` step, so the shipped plugin was missing those artifacts. Build now applies each tree-copied skill's `files:` config into `skills/<name>/`, exactly as it already does for shared-pool skills — removing the need for an external inject-into-dist script (which VAT couldn't see, producing false `LINK_TO_GITIGNORED_FILE` and `missing-bundled-file` findings).
- **`vat verify` no longer false-flags skills in plugins distributed by verbatim tree-copy (`vat build --only claude`).** A plugin that ships its skills by copying its own `skills/` tree (`source:` set, `skills: []`) builds correctly, but two verify checks still assumed the shared-skill-pool model and failed a byte-correct artifact: `files-config-dests` looked for a skill's `files:` dests only under `dist/skills/<name>/` and missed the plugin tree where build actually wrote them, and `PUBLISHED_SKILL_NOT_IN_PLUGIN` was blind to `source:`, flagging every skill a tree-copy plugin ships. Both checks (and `vat build`) now agree on where a tree-copied skill lands, so the false failures are gone. (Whether private `.claude/skills/**` skills should count as "published" is unchanged and tracked separately.)
- **`ExternalLinkValidator.clearCache()` and `getCacheStats()` now operate on both caches (issue #113).** Slice 2 introduced a second cache instance for authenticated-link results (per-OS-user scoping); the existing `clearCache()` / `getCacheStats()` methods continued to touch only the anonymous cache, so an adopter rotating a token would see stale `401`/`403` entries until the auth cache TTL expired. Both methods now clear/sum across both caches.
- **`ExternalLinkCache` IO errors degrade to a cache miss instead of aborting validation (issue #113).** `loadCache()` previously threw on anything other than `ENOENT` / `SyntaxError` (e.g. `EACCES` on a permissions-restricted cache file, `EROFS` on a read-only filesystem); `saveCache()` had no try/catch (write errors propagated). A failed read / write on the status-cache file would abort the whole `vat resources validate` run. Both paths are now fail-soft: a read failure returns an empty in-memory cache, a write failure no-ops while the in-memory cache stays authoritative for the remainder of the run. Cost of a bad cache entry: one extra fetch. Cost of a bad cache entry under the previous behavior: the whole run.
- **Lazy-loaded embedding providers no longer mislabel model/runtime failures as "not installed" ([#118](https://github.com/jdutton/vibe-agent-toolkit/issues/118)).** `loadPipeline` in `transformers-embedding-provider.ts` wrapped both the dynamic `import('@xenova/transformers')` and the model download/inference in a single `catch` that always rethrew a fixed `@xenova/transformers is not installed` message, swallowing the real error (not even as `cause`) — so a model-download or `onnxruntime-node` native-backend failure on an installed package was reported as a missing dependency. The two failure modes are now separated: an import failure keeps the actionable install hint (now with the original error attached as `cause`), while a model/inference failure throws `Failed to load transformers model '<model>'` preserving `cause`. The sibling `onnx-embedding-provider.ts` was audited: its install-hint `catch` was already correctly scoped to the import alone, but its model download (`ensureModelFiles`) and session creation (`InferenceSession.create`) previously bubbled raw errors with no provider/model context, so they now throw `Failed to download ONNX model '<model>'` / `Failed to load ONNX model '<model>'` with `cause` preserved.
- **Transformers.js integration tests now skip on Windows CI instead of flaking.** `transformers-embedding-provider.integration.test.ts` and the Transformers.js block of `comparison.integration.test.ts` skip on Windows (in addition to skipping when the optional `@xenova/transformers` dependency is absent), matching the existing `onnx-embedding-provider` test. These tests download a model over the network and load the `onnxruntime-node` native backend — both flaky in Windows CI. Such a failure was previously mislabeled `@xenova/transformers is not installed` by an over-broad `catch` in the provider's `loadPipeline` (the package was installed; the model download/inference is what failed), which is also why an availability-only guard did not prevent it.
- **Config-first skill discovery now honors `..` in `skills.include` patterns.** `vat build`, `vat verify`, and `vat skills validate` all funnel through `discoverSkillsFromConfig`, which previously passed every include pattern to a single downward-only crawl rooted at `projectRoot` — so an include like `"../../docs/skills/*/SKILL.md"` (common in monorepos where SKILL.md sources live alongside, not inside, the package) silently matched zero skills. `vat audit` accepted the same config only because it has a separate filesystem-first walker. Each include pattern is now split into a literal base + glob remainder via `picomatch.scan`, patterns are grouped by their resolved absolute base, and the crawler runs once per base — making config-first discovery agree with audit. User-supplied excludes stay anchored to `projectRoot` so patterns like `docs/private/**` keep their original meaning, and a pattern resolving to a nonexistent base now silently produces zero matches.
- **Anchor validation no longer reports a false `LINK_BROKEN_ANCHOR` for un-indexed target files (#112).** Previously a fragment link to any file the resource registry had not parsed (e.g. a target outside the crawl) was reported as a broken anchor. Anchor checks now skip targets absent from the fragment index — affecting markdown and HTML alike — while genuinely missing fragments in indexed files are still reported.
- **`vat resources validate` no longer crashes on same-stem `.md` + `.html` sibling files (#116).** Making HTML first-class added `.html`/`.htm` to the crawl, and same-stem siblings (e.g. `index.md` + `index.html`) previously produced an uncaught `Duplicate resource ID` exception that aborted the whole command. Fixed by the extension-suffixed ids above (siblings now get distinct ids), with `DUPLICATE_RESOURCE_ID` as a graceful backstop for any genuine post-normalization collision.
- **Post-build link checks now cover bundled HTML (#116).** `checkBrokenPackagedLinks` and the unreferenced-file check previously scanned only `.md`, so a broken `<a href>`/`<img src>` inside a packaged `.html`/`.htm` file shipped with a green build. Both checks — and the reachability traversal — now extract HTML links via the same parser, so broken links in packaged HTML surface as `PACKAGED_BROKEN_LINK` (failing the build) and an HTML file referenced only by other HTML is no longer falsely flagged `PACKAGED_UNREFERENCED_FILE`.
- **Deferred-artifact existence parity in the link walker (issue #129 carry-forward).** `walk-link-graph`'s `checkDeferred` guarded only the `files:` *source* branch with `!existsSync`; the *dest* branch deferred unconditionally. An existing real file at a `files:` dest (e.g. a gitignored artifact already on disk) was therefore silently downgraded to the `LINK_DEFERRED_ARTIFACT` info code, masking a genuine `LINK_TO_GITIGNORED_FILE` / directory-target signal. Both branches now share the existence guard: a path is treated as deferred only when it does not yet exist on disk.
- **`computeDeferredPaths` resolves `files:` sources exactly as the packager does (issue #129 carry-forward).** The deferred-source set was computed with `resolve(projectRoot, source)`, which let an absolute-looking source escape the project root, while the packager copies with `resolve(join(projectRoot, source))`. The two now use the identical expression, so an absolute-looking source roots under the project root in both places and the deferred set matches what the build actually copies.
### Internal
- **Skill-test eval fixtures excluded from the remaining link/structure validators (CI hygiene, no adopter-facing change).** The intentionally-broken eval fixtures (`resources/skills/evals/**` — non-portable SKILL.md samples, a fake plugin for `vat audit`) are test input, not real docs/code. They were already excluded from the repo-root resource validation, ESLint, and repo-structure checks; now also from the `vat-development-agents` package config (so `vat verify`'s resources phase stops failing on the fixtures' deliberate `LINK_BROKEN_FILE`s) and the `project-validation` dogfooding system test (hardcoded exclude list). Every exclusion site cross-references the others.
- **Eval fixtures hold clean, realistic code — incidental smells removed.** Two fixtures carried code-quality issues unrelated to what their eval tests: the `release-notifier-plugin` notifier script (a payload that only needs to *exist* so `vat audit` can flag the skill's local-script dependency) now validates its `--changelog` path instead of opening it blind, and the `vat-knowledge-resources` starter config dropped a redundant `TODO` comment (the eval's prompt already states the task). Fixtures that are themselves the *subject under review* (e.g. the vat-agent-authoring analyzer the eval asks an agent to improve) keep their VAT-domain flaws by design.
- **Unified `resolveSkillSource` skill-source resolver (#132, foundation).** A `skill-source/` module in `@vibe-agent-toolkit/agent-skills` that materializes a typed source union (`workspace` / `npm` / `url(+sha256)` / `path` / `vendored`) to a hardened, content-addressed staged directory through a per-user, `0700`, uid-checked fetch cache. The git-URL parser moved from `@vibe-agent-toolkit/cli` to `@vibe-agent-toolkit/utils`. No user-facing CLI surface yet — this is the resolver consumed by `vat skill test`.
- **Authenticated external-URL resolution foundation (issue #113, slice 1).** A pure `link-auth/` engine in `@vibe-agent-toolkit/utils` (host-glob provider selection, ordered token sources with no shell, header rendering with `Authorization` redaction, `github`/`sharepoint` macros) plus a strict `resources.linkAuth` config schema in `@vibe-agent-toolkit/resources`. Not yet wired into validation — consumer integration and the `LINK_AUTH_*` codes land in later slices — so there is no user-facing behavior yet.
- **`corpus/seed.yaml` is now generated from the upstream Anthropic marketplaces (issue #99, slice 1b).** A committed importer (`bun run import-marketplace [--allow-shrink]`) fetches the `claude-plugins-official` and `knowledge-work-plugins` catalogs, deduplicates by `source` URL, and rewrites the seed — replacing the previously hand-maintained list. Re-import is guarded against accidental shrinkage (refuses to overwrite on a 0-plugin fetch or a >20% drop unless `--allow-shrink`); current entry counts and audit provenance live in the generated seed header.
- **Empirical compatibility harness (issue #100).** A research scaffold (`packages/dev-tools/src/compat-empirical/`) for measuring skill compatibility across `claude-code`, `claude-cowork`, and `claude-chat` — it joins VAT's static predictions with deterministic runtime observations and an LLM-judge read into a reality-vs-prediction matrix, as evidence for future detector improvements. Lives entirely in the private `dev-tools` package with n…
github-actions Bot
pushed a commit
that referenced
this pull request
Jul 3, 2026
### Added
- **Dogfood eval suites for the whole `vat-development-agents` skill set, plus the fixes that dogfooding surfaced.** Every published VAT dev skill now ships a committed `vat skill test` eval suite (`evals/<skill>/`): `vat-audit`, `vat-skill-authoring`, `vat-knowledge-resources`, `vat-skill-distribution`, `vat-rag`, `vat-agent-authoring`, and `markdown-rewriting` (joining the existing `vat-skill-review` suite), wired via `skills.config.<skill>.test`. Final grades: vat-skill-distribution 25/25, vat-agent-authoring 24/24, vat-rag 22/22, vat-knowledge-resources 22/22, markdown-rewriting 18/18, vat-skill-authoring 21/22 (one capability-headroom miss), vat-audit 33/40 baseline A/B (the without-skill failures demonstrate the skill's lift on CI-gating/compat knowledge). Running the suites caught real skill/doc bugs, now fixed:
- **`markdown-rewriting` is now actually published.** It lived in the skills dir and `vat-skill-authoring` told agents to load `[[markdown-rewriting]]`, but the discovery glob (`vat-*.md`) didn't match its name, so it never shipped — a dangling skill reference. Added it to `skills.include` and `package.json` `vat.skills`; it now builds and ships.
- **`vat-skill-authoring`** gained the conservative-frontmatter-keys rule (the standard key set; stamp `version`/`team`/ownership under `metadata:` or in config.yaml, never as bare top-level keys) — the agent was inventing top-level `version:`/`team:` fields.
- **`vat-skill-review`** corrected a factual error: it claimed a `metadata:` field "will be rejected," but `metadata` is an allowed standard key (the sanctioned home for custom data per `SKILL_FRONTMATTER_EXTRA_FIELDS`).
- **`vat-rag`** removed a nonexistent `vat rag index --rebuild` flag (the real reset is `vat rag clear`; indexing is incremental) and added the missing `OnnxEmbeddingProvider` to the providers table.
- **`vat-knowledge-resources`** now states that `strict` mode only rejects extra fields when the schema sets `"additionalProperties": false`, and that collection validation defaults to `permissive`.
- **Collection-validation docs** corrected: `mode` defaults to `permissive` (matching `validateAgainstCollectionSchema`), not `strict` as previously documented.
- **Skill-test harness:** `buildForwardedEnv` now forwards `USER`/`LOGNAME` (see below) and eval fixtures (including intentionally-broken `.ts` files) are excluded from ESLint.
- **`vat skill test run` / `vat skill test configure` — behavioral skill testing in a context-isolated harness (#132).** Stage a packaged skill plus its declared dependencies into a throwaway, locked-down harness and run a canned, non-interactive evaluation that grades the skill against your `evals.json` (reusing skill-creator's grading rubric and JSON shapes) and writes `grading.json` (with a published [JSON Schema](docs/skill-test-grading-schema.md)), `friction.json`, and full transcripts you can inspect. `configure` writes a per-skill `test:` block to your config as a surgical edit — only the keys you pass change; surrounding formatting and comments are byte-preserved; a first `run` with no `evals.json` writes a template for you to fill in. Runs end-to-end against `claude` 2.x. **Security:** the harness runs the skill's own code with your account's privileges — it is *context* isolation, not an OS sandbox — so `run` requires `--i-understand-this-runs-skill-code`, enforced *before* anything runs (including the optional pre-stage build), and you should only test skills you trust. The pass/fail verdict is recomputed from the graded expectations, so a failing or empty grade is never silently reported as a pass; add `--fail-on-eval-failure` to make a failing eval exit non-zero and gate CI on it. See the new `vibe-agent-toolkit:vat-skill-testing` skill for auth modes, budget/turn/timeout caps, `--baseline` A/B runs, and exit codes.
- **Pre-stage `build:` hook + plugin-root staging.** An optional `test.build` command runs once before staging, so a skill that depends on a generated, un-committed artifact has it present (a non-zero build fails fast at preflight, before any tokens are spent). Plugin-distributed skills stage under their real plugin-root layout with `CLAUDE_PLUGIN_ROOT` set; standalone skills stage flat.
- **Declared test-env passthrough.** `passEnv` / `--pass-env` forwards host variables; `env` / `--env` injects values with `${fixturesDir}` / `${stagedSkillDir}` / `${harnessRoot}` / `${resultsDir}` interpolation. Both apply *after* the security allowlist — protected names always win, so committed test config can neither reroute your account credentials nor inject code: auth credentials, `PATH`, and credential-routing variables (`ANTHROPIC_BASE_URL` and the other endpoint/proxy overrides, `NODE_OPTIONS`, `NODE_EXTRA_CA_CERTS`) cannot be overridden. Fixtures under the skill's `evals/fixtures/` auto-stage with the eval tree.
- **Project-aware subject resolution.** Name a skill declared in `vibe-agent-toolkit.config.yaml` and `run` builds it first and tests the shipping **dist** — link-following, reference-rewriting, nav-stripping, and `files:` injection all applied — so you exercise exactly what installs, not the source tree. A path (including an already-built dist dir), or a `workspace:` / `npm:` / `url:` / `path:` / `vendored` source, is tested as-is; use `./<name>` to force a local directory over a colliding declared name. `--no-build` stages an existing dist without rebuilding (and errors if it is absent); `--dry-run` assembles the command without building and flags when the previewed dist may be stale, and — when no `evals.json` exists yet — reports where a real run *would* scaffold the template (exit 3) instead of writing it, so a dry run never touches your tree. A build failure fails fast at preflight (exit 2), before any tokens are spent.
- **Eval `files` are now provisioned.** Each eval's declared input files are staged into a per-eval working directory the executor operates on, enabling realistic "drop the agent in a project" evals. Files resolve relative to the `evals.json` directory and are materialized under `<harnessRoot>/workspaces/<id>/`; the experimenter prompt hands the executor that directory via a new `{{WORKSPACES_ROOT}}` token. A declared-but-missing input file fails fast at preflight (exit 2). Previously `files` was documented but inert.
- **Merge-readiness: liberal eval-suite schema, macOS subscription-auth fix, expanded skill, first dogfood suite.** (1) `evals.json` is adopter-authored data VAT *reads*, so its schema is now liberal per VAT's Postel's Law: `EvalSuiteSchema`/`EvalEntrySchema` are `.passthrough()` and `id` accepts a descriptive **string** or an int — only the fields VAT consumes (`prompt`, `expected_output`, `expectations`) stay required. This reverses the earlier strict-parser call that rejected real adopter suites three ways (string `id`, `category`, `_category_note`) and restores compatibility for the flagship adopter (app-platform/dxa). The persisted `test:` *config* block stays **strict** (it's VAT-produced config) — the deliberate inverse. (2) **macOS subscription-auth fix:** the harness env allowlist (`buildForwardedEnv`) dropped the POSIX `USER`/`LOGNAME` vars, so on macOS `claude auth status` could not read the login Keychain with the API key scrubbed — `--auth subscription` (and `inherit`'s subscription fallback) wrongly failed preflight, and the experimenter child could not authenticate. `USER`/`LOGNAME` are now forwarded (non-secret; already derivable from the forwarded `HOME`). (3) The `vibe-agent-toolkit:vat-skill-testing` skill gains a research-grounded "Authoring `evals.json`" section (blind realistic prompts, discriminating + negative expectations, categories, fixtures, `--baseline` skill-lift, grading) and a full flag⇄config knob table. (4) Ships the first committed VAT dogfood suite (`vat-skill-review`, 5 evals across catch-violation / no-false-positive / guidance-correctness) wired via `skills.config.vat-skill-review.test`, with eval fixtures excluded from `vat resources validate`.
- **`files:` entries now support glob sources and an optional `integrity` byte-verify.** A `source` containing glob magic (`*`, `**`, `?`, `[`) fans out into a directory `dest`, preserving the directory structure below the static base (glob is VAT's existing idiom, as in `skills.include` — no `recursive` flag). Globbed dests are late-bound, so `SKILL.md` links into them are treated as deferred artifacts at validate time (no `LINK_TO_GITIGNORED_FILE` allowlist needed). Add `integrity: true` to byte-verify the copy at build time and assert an exact dest subtree for glob entries.
- **`NON_PORTABLE_ASSET_REFERENCE` validation code (default `warning`) — a portability check family.** `vat skills validate` / `vat audit` now flag a skill document that references a bundled script/asset via a non-portable anchor, scanning the `SKILL.md` body **and every reachable bundled markdown doc** (agents copy invocations from reference files too). It's a family of sub-checks under one code — `claude-plugin-root`, `claude-project-dir`, and `absolute-script-path` — each finding names the variant and carries a tailored fix, and a single `validation.allow` entry silences the whole family for a file. These anchors don't exist when a skill is mounted standalone (claude.ai upload, API container), so the path breaks on the agent's first invocation; reference bundled files relative to the skill directory instead. See [`NON_PORTABLE_ASSET_REFERENCE`](docs/validation-codes.md#non_portable_asset_reference).
- **Skill-authoring guidance: portable bundled-script paths.** The `vibe-agent-toolkit:vat-skill-authoring` skill now documents how to reference bundled scripts/assets portably (relative to the skill directory, never `CLAUDE_PLUGIN_ROOT`/absolute/env-var anchors), and `vibe-agent-toolkit:vat-skill-review` carries the matching pre-publication checklist item.
- **Skill-review guidance: reserved words `claude`/`anthropic` in skill names.** The `vibe-agent-toolkit:vat-skill-review` skill's Naming section now carries the reserved-word rule as a canonical `[A]` item — Anthropic's authoring guidance states a skill `name` "Cannot contain reserved words: 'anthropic', 'claude'", and Claude Code refuses to load a non-certified skill named that way, so it fails at install/validation, not just review (`[RESERVED_WORD_IN_NAME]`). Surfaced by dogfooding the skill against its own eval suite (the reviewer was noting the prefix as "redundant" but missing the install-blocking consequence). The rule directs the reviewer to surface that consequence when reviewing such a name and to include the warning when advising on naming.
- **`NON_PORTABLE_COMMAND` validation code (default `warning`) — a portability check family.** `vat skills validate` / `vat audit` now flag a skill document that tells an agent to run a GNU/Linux-only shell command, scanning the `SKILL.md` body **and every reachable bundled markdown doc** (agents copy invocations from reference files too). It's a family of sub-checks under one code — `timeout`, `grep-pcre` (`grep -P`), `sed-i-no-backup` (`sed -i` with no suffix), `readlink-f`, and `date-d` (GNU `date -d`) — each finding names the variant and carries a tailored fix, and a single `validation.allow` entry silences the whole family for a file. Patterns match commands in command position only (not bare prose), so `grep -E`/`sed -i.bak` and nouns like "the request will timeout" are not flagged. Promotes a former manual `vat skill review` checklist line into an automated check. See [`NON_PORTABLE_COMMAND`](docs/validation-codes.md#non_portable_command).
- **linkAuth content-fetch primitive + content cache (issue #113, slice 3).** Ships the public `fetchAuthenticated(url, config, options) → { bytes, metadata, cached } | { outcome: 'unsupported' | 'unverified' }` primitive (`packages/resources/src/link-auth-content-fetch.ts`), per design §6.2 — *sibling to* the slice-2 health-check path, both reading from the same engine config and rewrite pipeline. No consumer wiring (asset-references, bundling) lands in this slice; the primitive ships standalone so future callers can adopt it without reworking the contract. **Two-mode headers (§6.2):** `Provider` gains an optional `fetch: { headers }` block alongside `auth: { headers }`; `resolveAuthenticatedUrl` now dual-expands both header sets against the same context (URL captures + resolved token), surfacing them on the success outcome as `headers` (auth, for health-check) and `fetchHeaders` (fetch, for content retrieval). The primitive merges `fetchHeaders` over `headers` so fetch-mode overrides on conflict — the canonical case being GitHub, where `Accept: application/vnd.github+json` returns 200 for any size but omits bytes >1 MiB (good for health-check) while `Accept: application/vnd.github.raw` streams the bytes inline (required for content). Adopter schema (`InlineProviderSchema`) gains a parallel `ProviderFetchSchema` (passthrough, like the rest of the adopter linkAuth tree per the repo Postel's Law rule), and the compile-time `_KeysAgree` drift check picks up the new field automatically. The resolved-token-wins precedence (URL-capture-named `token` cannot beat the resolved value) and the null-prototype hardening apply to `fetchHeaders` too. **`ContentCache` (§6.3, `packages/resources/src/content-cache.ts`):** new persistence class for the content-fetch primitive — distinct from slice 2's `ExternalLinkCache` (which is a status cache). Per-entry layout: `<sha256(rewrittenUrl)>.json` (metadata: status, content-type, etag, last-modified, fetchedAt, rewrittenUrl, `version: 1`) + `<sha256(rewrittenUrl)>.bin` (raw bytes), under a caller-supplied `cacheDir` (the validator-style `<cacheDir>/content/auth-${osUser}/` scoping is the caller's responsibility — the class only knows about the directory it was given, mirroring §6.3's "cross-user isolation = OS user, not cache key"). 30-minute default TTL (`§6.3`), tunable via constructor and via the threaded-through `resources.linkAuth.cache.ttlMinutes` adopter config (`buildLinkAuthEngineConfig` now copies the adopter `cache` block onto the engine `LinkAuthConfig`, which previously dropped it silently). Write order is `.bin` first, then `.json` as the commit marker — a partial-write crash leaves either no entry or `.bin` ahead of `.json` (reads as a miss), never `.json` ahead of `.bin` (which would serve stale bytes under new metadata). On-disk metadata fields are whitelisted via a single `pickMetadata()` helper used by both `set()` (strip smuggled fields before write) and `get()` (strip the on-disk `version` before return), so token-bearing fields a caller might smuggle through structural typing cannot land on disk — defense in depth on top of the closed `ContentMetadata` interface. TTL boundary is `>`, not `>=` — entries are valid at exactly the TTL, expire at TTL + 1 ms; tests pin both boundary cases. Fail-soft IO per #125 review: `EACCES` / `EROFS` / corrupted JSON degrade to a miss (read) or no-op (write), never throw. Forward-compat `version: 1` mirrors `ExternalLinkCache`. **Primitive behavior:** the four outcome branches — `unsupported` and `unverified` short-circuit with no fetch and **no cache touch** (§6.3: never cache `unverified`, since the result flips the moment a token appears); cache-hit returns `{ bytes, metadata, cached: true }` with no fetch; otherwise fetch via `authTransport` (cross-origin auth strip + 429 retry inherited from slice 2), read the body binary-clean via `arrayBuffer()`, build metadata from response headers (content-type / etag / last-modified default to `null` when absent), write through to the cache if supplied. `forceRefresh: true` bypasses cache reads but still writes through. `AbortSignal` propagates to the transport. The token value is interpolated into request headers in-memory and never flows into `ContentMetadata`; an end-to-end test reads every file in the cache directory after a fetch and asserts the literal token string is absent. **`wrapLinkAuthDepsWithMemo` lifted to its own module** (`packages/resources/src/link-auth-deps-memo.ts`) and exported from the resources barrel — slice 2 originally housed it private inside `external-link-validator.ts`, but the standalone primitive needs the same memoization, and the lift centralizes the implementation so jscpd cannot flag a clone. Validators and primitive callers iterating many URLs from the same provider wrap their `deps` once and reuse, so `gh auth token` / any `command`-source resolver runs at most once across the iteration. **File rename for slice 2's transport:** `packages/resources/src/link-auth-fetch.ts` → `link-auth-transport.ts`, and the exported function `fetchAuthenticated` → `authTransport` (with `AuthFetchOptions` → `AuthTransportOptions`) — frees the `fetchAuthenticated` name for the spec-documented primitive and aligns the filename with the symbol's role as the lower-level auth-safe HTTP wrapper. **47 new tests:** `link-auth-content-fetch` (15 covering short-circuits, header merge with fetch.headers override, cache hit/miss/forceRefresh, unverified-never-cached, binary-clean round-trip, signal pass-through, token-never-persisted), `content-cache` (14 covering round-trip, binary safety, distinct-URL isolation, overwrite, TTL boundary at `=` and `=+1ms`, version-mismatch eviction, corrupted-JSON tolerance, POSIX-skipped `EACCES` fail-soft on read and write, and the whitelist-on-write check), `link-auth-deps-memo` (5 covering single-source memo, distinct-argv independence, default-runCommand fallback, deps pass-through, undefined-deps handling), and 13 augmenting tests on the slice 2 surface for the new `provider.fetch` block (engine dual-expansion, schema acceptance/rejection, cache field propagation). The slice 3 primitive does not wire into any existing CLI command — `--refresh` / `--no-cache` ships with the first consumer slice.
- **linkAuth validator wiring + per-host outcome codes (issue #113, slice 2).** The slice-1 pure engine is now end-to-end: when an adopter sets `resources.linkAuth` in `vibe-agent-toolkit.config.yaml`, `vat resources validate` bypasses the anonymous `markdown-link-check` path for any URL whose host is claimed by a provider, issues an authenticated `fetch()` against the rewritten URL with the configured token, and classifies the response per design §7 into one of five new `CODE_REGISTRY` entries: `LINK_AUTH_DEAD` (404/410 from an honest-404 host — `error`-severity, the only such code in the slice; design §7 establishes that an authenticated 404 against e.g. SharePoint is high-confidence link rot, satisfying the rule-design corpus-evidence bar), `LINK_AUTH_DEAD_OR_UNAUTHORIZED` (404 from an ambiguous host like GitHub that masks `403`s — warn), `LINK_AUTH_FORBIDDEN` (`403` — warn), `LINK_AUTH_UNAUTHORIZED` (`401` — warn, promote to `error` on strict CI lanes), and `LINK_AUTH_UNVERIFIED` (no token resolved — warn, never cached per §6.3). New files: `packages/resources/src/link-auth-fetch.ts` (`fetchAuthenticated()` — bounded redirect loop with **cross-origin `Authorization` stripping (§8)** that is sticky across the rest of the chain to defeat token-laundering, **429/`Retry-After` honoring (§5.2)** parsing both delta-seconds and HTTP-date forms with a 60s DoS cap *and* a 250ms good-neighbor floor, all dependency-injectable via `fetchImpl`/`sleep`/`signal`); `packages/resources/src/link-auth-classify.ts` (pure `(status, providerCheck) → outcome+code` per §7's table); `packages/resources/src/link-auth-config-build.ts` (bridge from adopter config to engine — runs `expandMacro` on `{ use: <macro>, ...overrides }` entries; the adopter schemas are passthrough per the repo's Postel's Law rule, so the post-expansion `InlineProviderSchema.safeParse` catches missing required fields and wrong types on declared fields but lets unknown extras through, matching how the rest of project-config treats adopter input; a compile-time `_KeysAgree` assertion locks the schema's top-level field set to the engine's `Provider` interface). New cache architecture: a second `ExternalLinkCache` instance for auth-branch results, **keyed by the rewritten URL** (the original `blob/` URL 404s — caching it would poison results) and scoped to a per-OS-user subdirectory `cacheDir/auth-${sanitizedOsUser}/` so two users on a shared CI host never read each other's authenticated results (§6.3); the OS user resolves through `os.userInfo()` → `USER`/`USERNAME` env → `'default'` with a one-shot `console.warn` on the last fallback so the cross-user-leak risk is observable. Cache entries gain an explicit `version: 1` field; reads of any other version produce a miss, so slice 3's content-cache evolution can change the entry shape without misparsing pre-existing files. Doc-anchor coverage iterator test (`packages/agent-schema/test/docs/validation-codes.test.ts`) iterates `CODE_REGISTRY` and asserts each code has a matching `### \`CODE\`` heading in `docs/validation-codes.md` plus a convention-matching `entry.reference` — 126 assertions covering all 63 codes, future-proofs the per-code docs requirement. Five new doc sections under "Authenticated External Link Codes" in `docs/validation-codes.md`. Engine surface gains: `Provider.check` now flows through on the verified `ResolveOutcome` so the classifier can route per-provider `notFoundMeaning`; `ExternalLinkValidatorOptions` gains `linkAuthConfig`, `fetchImpl`, `linkAuthDeps`, `sleep`, and `osUser` (the first two are adopter-usable for corporate-proxy/custom-TLS injection, the rest test-only); `LinkValidationResult` gains a `code?: IssueCode` field that the `resource-registry.ts` consumer prefers over the existing status-code-to-`EXTERNAL_URL_*` mapping. The validator memoizes `runCommand` results per unique argv for the duration of a `validate()` run, so validating N URLs from the same host runs `gh auth token` (and any other command-source token) at most once. **196 new tests across `link-auth-classify` (13), `link-auth-fetch` (19), `external-link-validator-auth` (25), `link-auth-config-build` (10), `validation-codes` (+126 doc-anchor iterator, +2 LINK_AUTH_* registry), and `external-link-cache` (+1 version-gate).** Security-load-bearing tests pin the cross-origin Authorization strip (case-insensitive, sticky across chains, with userinfo/relative-Location edge cases), the path-traversal sanitizer (table-driven over 9 pathological `osUser` inputs), the unverified-no-cache invariant, the cache-hit re-classification (cache hits re-run the classifier against the current provider so a `notFoundMeaning` flip between runs surfaces the new code, not the old one), the runCommand memoization (N URLs from the same provider → 1 command invocation), and an Object.hasOwn-based prototype-pollution defense on the `{ use }` discriminator in `buildLinkAuthEngineConfig`. Slice 4 (cross-platform `.cmd`-shim system test, `VAT_LINKAUTH_ALLOW_COMMAND=0` opt-out, contributor docs) and slice 3 (content-fetch primitive + content cache) are downstream.
- **linkAuth pure engine foundation (issue #113, slice 1).** Adds a config-driven engine for authenticated external URL resolution, scoped to the pure-logic layer with no consumer wiring yet (the `ExternalLinkValidator` integration and the `LINK_AUTH_*` `CODE_REGISTRY` entries are slice 2; the content-fetch primitive is slice 3). New `link-auth/` module under `@vibe-agent-toolkit/utils` with eight files: `transforms.ts` (closed allowlist — `base64url`, `urlencode`, `lower` — with `Object.hasOwn`-based prototype-chain defense), `template.ts` (tiny `${name}` / `${transform(name)}` renderer, deliberately separate from the Handlebars renderer in `utils/template.ts`), `rewrite.ts` (ordered `when → vars → to` pipeline with fragment/query stripping per design §5.2), `build-headers.ts` (header rendering plus structural `Authorization` redaction), `select-provider.ts` (host-glob matching via picomatch with `excludeHost`), `expand-macro.ts` (YAML loader + deep-merge expander), `resolve-token.ts` (ordered env / `safeExecResult`-backed argv-command sources, first-non-empty wins, no shell), and `resolve.ts` (the public `resolveAuthenticatedUrl(url, config)` entry returning one of `{fetchUrl, headers}` / `{outcome: 'unsupported'}` / `{outcome: 'unverified', reason}`). Ships the `github` and `sharepoint` macros as a YAML data asset (`src/link-auth/macros.yaml`), with a new cross-platform `packages/dev-tools/src/copy-yaml-assets.ts` post-build step bundling `.yaml` into `dist/` — first YAML-asset shipping pattern in the utils package. Adds `yaml` as a utils dependency. Companion Zod schema in `@vibe-agent-toolkit/resources` (`src/schemas/link-auth.ts`) validates the `resources.linkAuth` config block (strict; accepts either `{ use: <macro>, ...overrides }` or full inline providers), wired as an optional field on `ResourcesConfigSchema`. 140 unit tests in utils + 29 schema tests in resources, all pure-logic with no network or filesystem dependencies; security-load-bearing tests pin the closed-allowlist guarantee, the `${__proto__}` bypass defense, the token-never-leaks-into-Authorization invariant, and `shell: false` literal-argv handling.
- **Corpus seed expanded from 9 → 237 entries via a new committed importer at `packages/dev-tools/src/import-marketplace.ts` (`bun run import-marketplace [--allow-shrink]`).** The script fetches `.claude-plugin/marketplace.json` from `anthropics/claude-plugins-official` (205 of 209 raw entries kept) and `anthropics/knowledge-work-plugins` (30 of 60 — the knowledge-work catalog turns out to be ≈50% mirror entries of the official catalog) via `gh api`, maps each upstream entry to a `PluginEntry`, deduplicates by `source` URL (preserved VAT-owned entries always win; otherwise alphabetical-first-name wins within each duplicate cluster), and rewrites `corpus/seed.yaml`. Mapping rules: `bucket: official` uniformly (both catalogs are anthropics-curated marketplaces — `bucket` is the *reporting posture* per slice 1a, not code provenance); `confidence: first-party` for catalog-internal string sources and `github.com/anthropics/...` object sources, else `curated`; the `./partner-built/` knowledge-work convention overrides to `curated`; `maturity: production` for all entries. URL composition handles all five upstream source shapes (string, `git-subdir` ± `ref`, `url` ± `path`, `github`), throwing on unknown discriminators. The seven sample entries from slice 1a are regenerated from upstream manifests on every re-import. Re-import safety: the importer refuses to overwrite `corpus/seed.yaml` if either upstream catalog returned 0 plugins or the new entry count would drop more than 20% vs. the existing seed; `--allow-shrink` bypasses both gates for the rare case where shrinkage is real. The generated `seed.yaml` header dropped its earlier per-entry `validation:` claim (the importer throws on validation blocks today) and now states explicitly that entry `source` URLs pin a fragment ref (typically the default branch), not a per-entry commit SHA — the catalog SHAs in the header are this run's audit provenance. Issue #99 slice 1b — follows the schema change from PR #111 (slice 1a).
- **Empirical compatibility harness (`packages/dev-tools/src/compat-empirical/`).** Per-#100 research scaffold for measuring skill compatibility across `claude-code`, `claude-cowork`, and `claude-chat`: a CLI (`predict`/`run`/`judge`/`report`/`all`) that joins VAT's static predictions with deterministic runtime observations and an LLM-judge semantic read into a reality-vs-prediction matrix — an evidence artifact for proposing detector improvements that each cite specific (skill, runtime) cells. Probe coverage: multi-prompt + repeat-N with adaptive N=3→N=5 extension, mandatory positive+negative prompt pairing per corpus entry, and negative-prompt agreement inversion so false-positive triggers surface as `vat-optimistic`. Evidence quality: the deterministic class is widened from 6 to 9 values (splitting `error` into `install-failed`/`runtime-error`, `not-invoked` into `not-invoked-engaged`/`not-invoked-empty`, adding `refused`), with a v2 judge prompt that adds a `refused` verdict. Report fidelity: coverage stats, per-bucket headline (own/official/community × ran/agree/optimistic/pessimistic/gray-zone), gray-zone (mixed-signal) and high-variance subsections, and per-attempt variance rendered inline (`runtime-error (2/3) / failed (3/3)`). Judge replay persists `judge-calls/<skillId>-<promptId>-<target>-<attemptIdx>.json` artifacts that a new `re-judge` subcommand re-executes against an optionally different model or freshly-edited system prompt — without re-spending operator hours on the runtime side. Also landed: `git fetch --tags --force` before named-ref fetch (annotated tag refresh) and `setup()` teardown-first idempotency for the manual driver. No detector code or `RUNTIME_PROFILES` changes; lives entirely in the private `@vibe-agent-toolkit/dev-tools` package with no adopter-facing surface. Design: [the v2 harness design](./docs/research/2026-05-23-compat-empirical-harness-v2-design.md). Corpus authoring, the first real run, and the docs deliverable are the downstream work.
- **Cowork driver spike.** Added [`docs/contributing/cowork-driver-spike.md`](docs/contributing/cowork-driver-spike.md) — a time-boxed investigation (per §4a of the harness v2 design) of whether `claude-cowork` can be driven programmatically by the empirical compat harness today. Verdict: **not feasible**; cowork is a Claude Desktop app product with no public API/CLI surface. The `claude-cowork` runtime stays on `scripted-assisted` until Anthropic ships a Cowork CLI mode, Sessions API, or documented filesystem-import path. Adjacent finding (not a cowork replacement): the public-beta Skills API (`POST /v1/skills` + `container.skills[]` on `/v1/messages`) supports a fully-automatable *new* runtime — captured in the spike doc as a potential follow-up, gated on a separate design decision.
- **Subscription-only compat harness billing.** The harness now bills a Claude Pro/Max subscription instead of the API: both token-consuming surfaces (the `claude-code` runtime driver and the LLM judge) route through one shared `claude` CLI invoker (`runtimes/shared/claude-cli.ts`) that injects the operator's `CLAUDE_CODE_OAUTH_TOKEN` and deletes every API credential from the child env, so the CLI cannot fall back to API billing. The operator's own token is sourced at preflight — env var if set, otherwise an interactive prompt — so a run only ever spends the operator's personal plan. The judge was migrated off `@anthropic-ai/sdk` (dependency removed) onto the CLI, parsing a strict JSON verdict with one retry instead of the SDK's forced-tool call (`judge-system.md` now asks for a JSON object). `RunMetadata` gains `authMode` and the report methodology discloses subscription auth + parsed-not-forced verdicts. Premise (zero API billing under the OAuth token) still pending the manual smoke test.
- **First-class local HTML resources (#112).** `.html`/`.htm` files are now discovered, parsed, link- and anchor-validated, checked for well-formedness, and link-rewritten on bundle — using the same `ParseResult` contract and validation framework as markdown. A parse5-backed parser extracts `<a href>` and `<img src>` links plus `id`/`name` fragment anchors; `ResourceRegistry` routes HTML through it and persists optional `anchors`/`parseErrors` on `ResourceMetadata`. Anchor validation now uses a format-neutral fragment index (each file's markdown heading slugs or HTML `id`/`name`, with its case-matching policy carried per entry), enabling cross-format anchor checks (md↔html) with HTML ids matched case-sensitively and markdown slugs case-insensitively. A new `MALFORMED_HTML` code (default `info`) surfaces parser well-formedness diagnostics. On bundle, `<a href>`/`<img src>` values are rewritten by offset-splicing the original source (never re-serialized), so unchanged markup round-trips byte-for-byte and original attribute quoting is preserved (a rewritten value that would be unsafe unquoted is wrapped in quotes). Scope is `<a href>` + `<img src>` only; `<link>`/`<script>`/`<iframe>`/`<source srcset>`/CSS `url(...)` are deferred (asset/machinery references, not the content link graph). `<base href>` is not honored — relative hrefs resolve against the file's own directory (see the breaking note below for the `ResourceMetadataSchema` tightening that shipped with this work).
- **`DUPLICATE_RESOURCE_ID` validation code (default `error`).** When two files resolve to the same resource id after path normalization (e.g. `My Guide.md` and `my-guide.md` both → `my-guide-md`), `vat resources validate` now reports it as an `error` issue naming both files, instead of aborting the entire run with an uncaught `Duplicate resource ID` exception. Documented under [Resource Registry Codes](./docs/validation-codes.md).
- **Live audit/validate now sees source HTML links (issue #129 AC2).** `vat audit` / `vat skills validate` previously crawled `**/*.md` only, so links inside source `.html`/`.htm` files were invisible until build time. The live crawl now includes HTML (the registry already parses it via parse5), so the link-graph walker traverses HTML references and a broken local link inside a source HTML file surfaces as `LINK_MISSING_TARGET` at validate time, at parity with the built path's `PACKAGED_BROKEN_LINK`.
- **`LINK_DEFERRED_ARTIFACT` info code (issue #127, slice 2 of #129).** A `SKILL.md` link to a `files:`-declared artifact that doesn't exist yet (a dest built later, or a not-yet-created source) is no longer reported as a broken link — it downgrades from `LINK_MISSING_TARGET` to the new [`LINK_DEFERRED_ARTIFACT`](docs/validation-codes.md#link_deferred_artifact) info code at validate time, and `vat skills build` preserves and rewrites the link to the materialized dest instead of stripping it.
### Changed (breaking, pre-1.0)
- **`computeDeferredPaths` return type changed (issue #127, slice 2 of #129).** `computeDeferredPaths(files)` now returns `{ destPaths, sourcePaths }` instead of a flat `Set<string>` — a breaking API change (pre-1.0, intentional). Both `vat skills validate` and `vat skills build` now consume the deferred-path set (previously `deferredAssets` was silently dropped), and deferred dest/source paths resolve project-root-relative so the new behavior works for skills in subdirectories, not only at the project root. Plugin-local `files:` deferred paths remain out of scope for this slice (see [AC-10d](docs/architecture/skill-packaging.md#ac-10d--plugin-local-files-deferred-paths-are-out-of-scope-for-issue-127--slice-2-of-129)).
- **Directory links are now valid targets; `LINK_TARGETS_DIRECTORY` is narrowed to typed single-file slots (issue #126, slice 1 of #129).** A navigational local link that resolves to an existing directory (e.g. `[docs/](docs/)` in a ToC, README, or SKILL.md body) is no longer an error in `vat resources validate` or the skill-bundling link walk — previously any local link to a directory was a hard error. A renamed/deleted directory still fails via the ordinary broken-link path. `LINK_TARGETS_DIRECTORY` (still `error`) now fires **only** for a packaging `files:` *source* entry that resolves to a directory (the contract demands exactly one file). GitHub-style directory-index resolution (`docs/` → `docs/README.md`) is intentionally not implemented. Known limit (tracked for #129): a no-slash link such as `[Concepts](concepts)` that resolves to a directory is still treated as a file link; the slash form is the navigational case this slice covers.
- **`ResourceMetadataSchema` is now `strict()`.** Shipped with first-class HTML support (#112): the resource-metadata schema rejects unknown top-level fields instead of silently accepting them, so a typo or stale field in code that constructs `ResourceMetadata` now fails at parse time rather than passing through. Move any extra data into a recognized field or drop it.
- **Resource ids now carry a file-extension suffix.** `generateIdFromPath` appends `-<ext>` to every resource id (e.g. `guide.md` → `guide-md`, `guide.html` → `guide-html`, `README.md` → `readme-md`). This makes a markdown file and a same-stem HTML file distinct resources instead of colliding — the prerequisite for first-class HTML resources sharing a directory with their markdown source. Resource ids are internal, path-derived identifiers (never hand-authored in config or frontmatter), but anything that referenced an id by its old bare form must use the suffixed form — most visibly `vat rag query --resource-id` filters and re-indexed chunk ids (re-index to regenerate).
- **`vat resources validate` gains per-code severity configuration, and external-URL findings no longer fail the build by default.** Resource findings now use the same configurable severity framework as `vat skills`: each is a documented code (e.g. `LINK_BROKEN_FILE`, `EXTERNAL_URL_DEAD`) with a default severity, overridable per project under `resources.validation.severity` / `resources.validation.allow`. External-URL findings now default to `warning` and no longer flip the exit code (fixing a bug where they always failed the command); set their severity to `error` to restore failing. Severity now also accepts an `info` level. The never-implemented `resources.validation.checkLinks`/`checkAnchors`/`allowExternal` keys are removed.
- **`validation.severity` / `validation.allow` keys are validated against real codes.** A mistyped code key (e.g. `LNIK_OUTSIDE_PROJECT`) is now a config-load error instead of a silent no-op.
- **Corpus seed entries now require `bucket`, `confidence`, and `maturity` metadata fields.** `PluginEntrySchema` in `vat corpus scan`'s seed loader gains three required enum fields: `bucket: 'official' | 'community'`, `confidence: 'first-party' | 'curated' | 'listed'`, and `maturity: 'production' | 'experimental' | 'example'`. The bundled `corpus/seed.yaml` is updated; downstream callers running custom seeds must add the fields to every entry. `bucket` is the load-bearing discriminator (`official` entries report named findings; `community` entries are aggregate-only in follow-up work). The other two are descriptive metadata used by triage tooling.
- **`vat claude marketplace publish` no longer reports the project root `package.json` version in the CLI banner, commit message, status YAML, or CHANGELOG section lookup.** The label is now derived from the staged `marketplace.json`. Single-plugin marketplaces use the plugin's version — banner reads `Publishing marketplace "X" v0.0.4`, commit subject reads `publish v0.0.4`. Multi-plugin marketplaces drop the `v<X>` entirely — banner reads `Publishing marketplace "X"`, commit subject reads `publish X` — since the per-plugin `version` fields in the published `marketplace.json` are the source of truth for which plugin moved to which version. Two visible side-effects follow: (1) the status YAML's `published[*].version` field is now absent for multi-plugin marketplaces (previously it carried the misleading project version) — automation should read per-plugin versions from the published `marketplace.json` instead; (2) the stamped `## [X.Y.Z]` CHANGELOG lookup now uses the plugin's version rather than the project's, so a previously-ignored matching section will now be picked up as the commit body for single-plugin marketplaces. The `marketplace.json` schema's optional top-level `version` field is not yet consumed — that is a separate follow-up.
- **Adopter-facing `LinkAuthConfig` type renamed to `LinkAuthProjectConfig` (issue #113).** Both `@vibe-agent-toolkit/utils` (engine) and `@vibe-agent-toolkit/resources` (Zod-inferred adopter shape) previously exported a type named `LinkAuthConfig`, causing IDE auto-import ambiguity in any code that touched both. The adopter type — accessible as `import type { LinkAuthProjectConfig } from '@vibe-agent-toolkit/resources/schemas/link-auth'` — is the one renamed; the engine's `LinkAuthConfig` is unchanged (more API surface depends on it). Migration: rename the import. The Zod schema's name (`LinkAuthConfigSchema`) is unchanged.
- **External-link cache directory layout adds an `auth-${osUser}/` subdirectory and an entry `version: 1` field (issue #113 §6.3).** When `vat resources validate` runs with `resources.linkAuth` configured, authenticated-fetch results land under `<cacheDir>/auth-${sanitizedOsUser}/external-links.json` rather than the shared `external-links.json` used by the anonymous `markdown-link-check` path — two users on the same host (e.g. shared CI runners) cannot read each other's authenticated cache entries. All cache entries now carry an explicit `version: 1` field; entries written under a different (or missing) version are treated as a cache miss, so any pre-existing `external-links.json` triggers a one-time re-fetch on first run after upgrade. The `version` gate is forward-compat for slice 3's content-cache shape evolution.
- **`vat claude marketplace publish` no longer pushes per-plugin `<name>-v<version>` source-repo tags.** The post-publish tagging step (introduced alongside multi-plugin versioning) is removed entirely — no tags are created or pushed, and the misleading `Repository not found` / "tag already exists at a different commit" warnings it emitted on every cross-repo publish are gone ([#121](https://github.com/jdutton/vibe-agent-toolkit/issues/121)). The tags were pushed to the marketplace remote rather than a source remote, never landed anywhere useful, and there was no opt-in demand. Which plugin moved to which version is now determined solely by the per-plugin `version` fields in the published `marketplace.json`. No config key or flag is involved; if you relied on these tags, create them in your own release workflow.
### Fixed
- **`vat skill test` now verifies a per-run integrity nonce on `grading.json`, so untrusted skill code can no longer forge a passing result.** The experimenter writes `grading.json` into the harness results dir, which is inside the skill-writable sandbox (`--add-dir`, `bypassPermissions`) — an adversarial skill could therefore write or rewrite a passing `grading.json` that the harness would trust. The harness now stamps a secret per-run nonce into the experimenter prompt and requires `grading.json` to echo it in a top-level `runNonce`; a missing or mismatched nonce is rejected (exit 1). The nonce is delivered to the experimenter **only via stdin** — the prompt is no longer written to disk (`spawnHeadlessClaude` takes the prompt in memory), and the persisted `experimenter-prompt.txt` audit copy is redacted — so skill code cannot read the nonce back to forge a match. Surfaced by adversarial review; the harness still runs behind the `--i-understand-this-runs-skill-code` ack (this is defense-in-depth, not an OS sandbox).
- **A committed `test.*` config can no longer RAISE `vat skill test`'s built-in cost/runtime caps — only lower them.** `test.maxBudgetUsd` / `test.maxTurns` / `test.timeout` fed the same precedence slot as the CLI flags, so a subject repo you were merely testing could commit `maxBudgetUsd: 100` and silently bill a run far above the built-in $5 / 50-turn / 5-minute ceilings. A config-sourced value is now clamped to the built-in cap (with a one-line stderr note when clamped); a CLI flag, being explicit operator intent, may still exceed it. Surfaced by adversarial review.
- **`vat skill test` no longer cross-wires two injected plugins that share a directory basename.** The staged plugin-root dir was keyed on `basename(pluginDir)` alone, so two different `--with` plugins at e.g. `…/a/my-plugin` and `…/b/my-plugin` collided onto one staged root — the second silently inherited the first's `CLAUDE_PLUGIN_ROOT` and `.claude-plugin/` manifest, producing a misleading result. The staged segment is now keyed on the full resolved plugin path (basename kept as the readable slug, disambiguated by a hash of the full path).
- **A `files:` entry's `integrity: true` byte check is no longer silently skipped when the file was already link-bundled.** In `applyFilesConfig`, a non-glob entry whose source had already been materialized by link traversal short-circuited past the integrity verification — so a requested byte check simply didn't run for that file. The byte check now runs against the link-bundled dest (which lands at `entry.dest`) on the skip path, exactly as it would on the copy path.
- **A broken `vibe-agent-toolkit.config.yaml` is no longer silently ignored by `vat skill review` / `vat skill test` (regression fix).** The shared config walk-up (`loadConfigCached`, via `resolveSkillPackagingConfig`) swallowed a *present-but-broken* config to `undefined` — indistinguishable from "no config." That silently downgraded `vat skill review` (which previously errored on a bad config through the throwing `loadConfig`) and would let `vat skill test` apply defaults / stage the wrong subject against a config the author clearly intended. A broken config now raises a typed `ConfigLoadError` that skill-resolving commands surface (review reports it; `vat skill test` exits 2 with a clean message), while `vat audit` — a bulk linter that must keep scanning — explicitly catches it and falls back to config-free validation. An *absent* config still resolves to `undefined` as before. The error is cached so a broken config re-throws without re-parsing across a multi-skill scan.
- **`vat skill test run` now rejects a bad usage flag with a clean message and preflight exit code (2) instead of a raw stack trace.** Flag validation and config loading (`--auth`/`--require-auth` values, numeric `--max-turns`/`--max-budget-usd`/`--timeout`/`--stall`, the persisted test config) ran *before* the command's first `try`, so a malformed flag surfaced as an unhandled promise rejection (stack dump, exit 1). They now run inside a preflight guard: an unrecognized value prints `Error: --auth must be one of: …` and exits 2 without ever reaching the harness. `--auth`/`--require-auth` are validated on the run path too (previously only `configure` checked them), via a shared `auth-flags` helper so the two commands cannot drift. The `--dry-run` help no longer claims to print "the exact assembled command" (it shows the model flag; budget/turns/permission flags are added at spawn time).
- **`vat skill test`'s scrubbed-env deny-list now blocks the OS-linker and Node module-resolution code-injection vars.** A skill-under-test's committed config can forward named host env vars into the headless `claude` child via `test.passEnv`/`test.env`. The deny-list already refused `NODE_OPTIONS`/`NODE_EXTRA_CA_CERTS` (code injection before any userland code runs) but not their exact siblings — `LD_PRELOAD`, `LD_LIBRARY_PATH`, `DYLD_INSERT_LIBRARIES`, `DYLD_LIBRARY_PATH` (native `.so`/`.dylib` injection), `NODE_PATH` (module-resolution hijack), and `GIT_SSH_COMMAND` (arbitrary command on a `git:` source clone). These are now deny-only: a config naming one is ignored with a warning, the protected value wins. (The feature already runs behind an explicit `--i-understand-this-runs-skill-code` ack and a loud security warning; this closes a defense-in-depth gap surfaced by adversarial review.)
- **`vat resources validate` no longer flags inline `data:`/`blob:` resources as `LINK_UNKNOWN` warnings.** A `data:` URI embeds its own payload and a `blob:` URL references an in-memory object — neither has a target to fetch or an anchor to resolve, so there is nothing to validate. They previously fell into the "unknown link type" catch-all (any href containing `:` that wasn't `http(s)`/`mailto`) and surfaced as warnings, which is noise for the extremely common inline-image pattern (`<img src="data:image/svg+xml,…">`). A new `embedded` link type classifies them and skips validation, mirroring how `external`/`email` links are already skipped. Genuinely unrecognized schemes (`javascript:`, `tel:`, `ftp:`) still classify as `unknown`.
- **`vat resources validate` no longer emits false-positive `LINK_BROKEN_ANCHOR` errors for `#fragment` links in HTML files.** HTML fragment anchors are frequently resolved at runtime by client-side JavaScript — hash routers, SPA `#/route` links, hash-encoded query params (`#id=1&mode=x`) — rather than by a literal element `id`/`name` in the markup, and ids can also be injected dynamically at runtime. A static "id not found" is therefore not proof the link is broken. Anchor resolution is now **skipped for HTML targets by default**; markdown heading-anchor validation is unchanged and still errors on a genuine miss. A new `--check-html-anchors` flag (mirroring `--check-external-urls`) opts in to strict HTML anchor resolution for fully-static pages — and even then, structural non-anchors (`#/route`, `#k=v&…`) are skipped since they can never be element ids. This restores clean `vat verify`/`vat resources validate` runs for HTML/SPA projects, reported by an external adopter whose gating CI turned red on functional runtime deep-links.
- **`vat build` now fails when a shipped Claude plugin skill has a broken packaged link.** `vat claude plugin build` never ran a post-assembly link check on the plugin output tree — only the pool packaging path did. A plugin skill whose shipped links were broken (e.g. relative links that assumed pool-packaging relocation but the skill was verbatim tree-copied) previously shipped silently. `vat build` now runs the existing depth-free `checkBrokenPackagedLinks` check against every shipped skill dir after the `claude` phase and fails the build with a `PACKAGED_BROKEN_LINK` error on any dead link. The check is scoped per skill dir — a skill is a self-contained portable unit, so a link that escapes its own directory (even to a sibling skill that co-ships in the same plugin) is a broken shipped link.
- **`vat claude plugin build` no longer double-produces a skill that is both pool-selected and present in the plugin's own `skills/` source tree.** Tree-copy (verbatim, unaware of packaging) and pool-import (packaged, link-rewritten) never coordinated — a skill claimed by both mechanisms shipped as two coexisting copies at different depths inside the same `skills/<name>/` directory, with the raw tree-copy carrying un-rewritten (and therefore potentially dead) relative links. The plugin's resolved pool selector is now excluded from the verbatim tree-copy before it runs, so the pool-packaged copy is the sole source for a colliding skill; the build prints a warning naming the skill and both sources. Non-colliding tree-copy and pool-import usage is unaffected.
- **`validateSkill` no longer silently reports a boundary-escaping AND missing link as a warning-only boundary notice.** `validateLocalLink`'s boundary-escape check returned before the existence check ever ran, so a link that both escaped the skill directory boundary and pointed at a non-existent file was classified `LINK_OUTSIDE_PROJECT` (warning) and never surfaced as broken — this is why `vat claude marketplace validate` could report a shipped tree with a dead, boundary-escaping link as 0 errors. Existence is now checked before boundary classification: a missing target is always `LINK_INTEGRITY_BROKEN` (error), regardless of whether it also escapes the boundary. A link that escapes the boundary but resolves to an existing file is unaffected (still a warning).
- **Skill-test eval-suite schema hardened after an adversarial review of the Postel liberalization.** Four issues the `id`/passthrough widening introduced or left open, all verified against the real `dxa` adopter suites in `app-platform`:
- **String eval ids are now validated as filesystem-safe path segments** (`[A-Za-z0-9_-]+`). A string `id` names a per-eval working directory, and the experimenter substitutes it verbatim into `<workspaces>/<id>`; an id like `year:extraction` previously passed parse, then failed on Windows (illegal filename) — surfacing as a *misleading* "escapes the eval directory" copy error. Rejected at parse with a clear message instead. dxa's hyphenated ids (`dollar-quote-recovery`) are unaffected.
- **Numeric `1` and string `"1"` no longer slip past the uniqueness check.** Ids are deduped on their stringified form, since both name the same workspace directory and would otherwise silently clobber each other's staged files.
- **A near-miss typo of the optional `files` field is now flagged** (`filez` → "did you mean files?"). Under plain `.passthrough()` such a typo was silently swallowed and the eval ran in an empty workspace. The check is a single-edit match scoped to recognized fields, so legitimate adopter extras (`name`, `category`, `notes`, `_category_note`) still pass through untouched.
- **`stageEvalWorkspaces` no longer mislabels copy failures as containment escapes.** Containment (`joinUnderRoot`) and the filesystem copy are now in separate try/catch blocks, so a permission/illegal-filename/disk error reports accurately instead of as "escapes the eval directory."
- **Skill-test `expected_output` is now optional, and is fed to the grader as context when present.** The pass/fail verdict is always decided per `expectations` entry, so `expected_output` is no longer required (per Postel's Law) — this unblocks real adopter suites (e.g. `dxa-consumption`) that grade with `expectations` alone. Previously the field was accepted but consumed by nothing; the experimenter prompt now passes it to the grader as the author's prose description of a correct result, informing judgment without becoming a checklist item. Still validated as a non-empty string when present.
- **`vat claude plugin build` now copies a tree-copied skill's `files:` artifacts into the distributed plugin (#127).** A skill that ships build-provided artifacts in its own directory via `files: [{ source, dest }]` but lives in a plugin's source tree was distributed by a verbatim tree-copy that skipped its `files:` step, so the shipped plugin was missing those artifacts. Build now applies each tree-copied skill's `files:` config into `skills/<name>/`, exactly as it already does for shared-pool skills — removing the need for an external inject-into-dist script (which VAT couldn't see, producing false `LINK_TO_GITIGNORED_FILE` and `missing-bundled-file` findings).
- **`vat verify` no longer false-flags skills in plugins distributed by verbatim tree-copy (`vat build --only claude`).** A plugin that ships its skills by copying its own `skills/` tree (`source:` set, `skills: []`) builds correctly, but two verify checks still assumed the shared-skill-pool model and failed a byte-correct artifact: `files-config-dests` looked for a skill's `files:` dests only under `dist/skills/<name>/` and missed the plugin tree where build actually wrote them, and `PUBLISHED_SKILL_NOT_IN_PLUGIN` was blind to `source:`, flagging every skill a tree-copy plugin ships. Both checks (and `vat build`) now agree on where a tree-copied skill lands, so the false failures are gone. (Whether private `.claude/skills/**` skills should count as "published" is unchanged and tracked separately.)
- **`ExternalLinkValidator.clearCache()` and `getCacheStats()` now operate on both caches (issue #113).** Slice 2 introduced a second cache instance for authenticated-link results (per-OS-user scoping); the existing `clearCache()` / `getCacheStats()` methods continued to touch only the anonymous cache, so an adopter rotating a token would see stale `401`/`403` entries until the auth cache TTL expired. Both methods now clear/sum across both caches.
- **`ExternalLinkCache` IO errors degrade to a cache miss instead of aborting validation (issue #113).** `loadCache()` previously threw on anything other than `ENOENT` / `SyntaxError` (e.g. `EACCES` on a permissions-restricted cache file, `EROFS` on a read-only filesystem); `saveCache()` had no try/catch (write errors propagated). A failed read / write on the status-cache file would abort the whole `vat resources validate` run. Both paths are now fail-soft: a read failure returns an empty in-memory cache, a write failure no-ops while the in-memory cache stays authoritative for the remainder of the run. Cost of a bad cache entry: one extra fetch. Cost of a bad cache entry under the previous behavior: the whole run.
- **Lazy-loaded embedding providers no longer mislabel model/runtime failures as "not installed" ([#118](https://github.com/jdutton/vibe-agent-toolkit/issues/118)).** `loadPipeline` in `transformers-embedding-provider.ts` wrapped both the dynamic `import('@xenova/transformers')` and the model download/inference in a single `catch` that always rethrew a fixed `@xenova/transformers is not installed` message, swallowing the real error (not even as `cause`) — so a model-download or `onnxruntime-node` native-backend failure on an installed package was reported as a missing dependency. The two failure modes are now separated: an import failure keeps the actionable install hint (now with the original error attached as `cause`), while a model/inference failure throws `Failed to load transformers model '<model>'` preserving `cause`. The sibling `onnx-embedding-provider.ts` was audited: its install-hint `catch` was already correctly scoped to the import alone, but its model download (`ensureModelFiles`) and session creation (`InferenceSession.create`) previously bubbled raw errors with no provider/model context, so they now throw `Failed to download ONNX model '<model>'` / `Failed to load ONNX model '<model>'` with `cause` preserved.
- **Transformers.js integration tests now skip on Windows CI instead of flaking.** `transformers-embedding-provider.integration.test.ts` and the Transformers.js block of `comparison.integration.test.ts` skip on Windows (in addition to skipping when the optional `@xenova/transformers` dependency is absent), matching the existing `onnx-embedding-provider` test. These tests download a model over the network and load the `onnxruntime-node` native backend — both flaky in Windows CI. Such a failure was previously mislabeled `@xenova/transformers is not installed` by an over-broad `catch` in the provider's `loadPipeline` (the package was installed; the model download/inference is what failed), which is also why an availability-only guard did not prevent it.
- **Config-first skill discovery now honors `..` in `skills.include` patterns.** `vat build`, `vat verify`, and `vat skills validate` all funnel through `discoverSkillsFromConfig`, which previously passed every include pattern to a single downward-only crawl rooted at `projectRoot` — so an include like `"../../docs/skills/*/SKILL.md"` (common in monorepos where SKILL.md sources live alongside, not inside, the package) silently matched zero skills. `vat audit` accepted the same config only because it has a separate filesystem-first walker. Each include pattern is now split into a literal base + glob remainder via `picomatch.scan`, patterns are grouped by their resolved absolute base, and the crawler runs once per base — making config-first discovery agree with audit. User-supplied excludes stay anchored to `projectRoot` so patterns like `docs/private/**` keep their original meaning, and a pattern resolving to a nonexistent base now silently produces zero matches.
- **Anchor validation no longer reports a false `LINK_BROKEN_ANCHOR` for un-indexed target files (#112).** Previously a fragment link to any file the resource registry had not parsed (e.g. a target outside the crawl) was reported as a broken anchor. Anchor checks now skip targets absent from the fragment index — affecting markdown and HTML alike — while genuinely missing fragments in indexed files are still reported.
- **`vat resources validate` no longer crashes on same-stem `.md` + `.html` sibling files (#116).** Making HTML first-class added `.html`/`.htm` to the crawl, and same-stem siblings (e.g. `index.md` + `index.html`) previously produced an uncaught `Duplicate resource ID` exception that aborted the whole command. Fixed by the extension-suffixed ids above (siblings now get distinct ids), with `DUPLICATE_RESOURCE_ID` as a graceful backstop for any genuine post-normalization collision.
- **Post-build link checks now cover bundled HTML (#116).** `checkBrokenPackagedLinks` and the unreferenced-file check previously scanned only `.md`, so a broken `<a href>`/`<img src>` inside a packaged `.html`/`.htm` file shipped with a green build. Both checks — and the reachability traversal — now extract HTML links via the same parser, so broken links in packaged HTML surface as `PACKAGED_BROKEN_LINK` (failing the build) and an HTML file referenced only by other HTML is no longer falsely flagged `PACKAGED_UNREFERENCED_FILE`.
- **Deferred-artifact existence parity in the link walker (issue #129 carry-forward).** `walk-link-graph`'s `checkDeferred` guarded only the `files:` *source* branch with `!existsSync`; the *dest* branch deferred unconditionally. An existing real file at a `files:` dest (e.g. a gitignored artifact already on disk) was therefore silently downgraded to the `LINK_DEFERRED_ARTIFACT` info code, masking a genuine `LINK_TO_GITIGNORED_FILE` / directory-target signal. Both branches now share the existence guard: a path is treated as deferred only when it does not yet exist on disk.
- **`computeDeferredPaths` resolves `files:` sources exactly as the packager does (issue #129 carry-forward).** The deferred-source set was computed with `resolve(projectRoot, source)`, which let an absolute-looking source escape the project root, while the packager copies with `resolve(join(projectRoot, source))`. The two now use the identical expression, so an absolute-looking source roots under the project root in both places and the deferred set matches what the build actually copies.
### Internal
- **Skill-test eval fixtures excluded from the remaining link/structure validators (CI hygiene, no adopter-facing change).** The intentionally-broken eval fixtures (`resources/skills/evals/**` — non-portable SKILL.md samples, a fake plugin for `vat audit`) are test input, not real docs/code. They were already excluded from the repo-root resource validation, ESLint, and repo-structure checks; now also from the `vat-development-agents` package config (so `vat verify`'s resources phase stops failing on the fixtures' deliberate `LINK_BROKEN_FILE`s) and the `project-validation` dogfooding system test (hardcoded exclude list). Every exclusion site cross-references the others.
- **Eval fixtures hold clean, realistic code — incidental smells removed.** Two fixtures carried code-quality issues unrelated to what their eval tests: the `release-notifier-plugin` notifier script (a payload that only needs to *exist* so `vat audit` can flag the skill's local-script dependency) now validates its `--changelog` path instead of opening it blind, and the `vat-knowledge-resources` starter config dropped a redundant `TODO` comment (the eval's prompt already states the task). Fixtures that are themselves the *subject under review* (e.g. the vat-agent-authoring analyzer the eval asks an agent to improve) keep their VAT-domain flaws by design.
- **Unified `resolveSkillSource` skill-source resolver (#132, foundation).** A `skill-source/` module in `@vibe-agent-toolkit/agent-skills` that materializes a typed source union (`workspace` / `npm` / `url(+sha256)` / `path` / `vendored`) to a hardened, content-addressed staged directory through a per-user, `0700`, uid-checked fetch cache. The git-URL parser moved from `@vibe-agent-toolkit/cli` to `@vibe-agent-toolkit/utils`. No user-facing CLI surface yet — this is the resolver consumed by `vat skill test`.
- **Authenticated external-URL resolution foundation (issue #113, slice 1).** A pure `link-auth/` engine in `@vibe-agent-toolkit/utils` (host-glob provider selection, ordered token sources with no shell, header rendering with `Authorization` redaction, `github`/`sharepoint` macros) plus a strict `resources.linkAuth` config schema in `@vibe-agent-toolkit/resources`. Not yet wired into validation — consumer integration and the `LINK_AUTH_*` codes land in later slices — so there is no user-facing behavior yet.
- **`corpus/seed.yaml` is now generated from the upstream Anthropic marketplaces (issue #99, slice 1b).** A committed importer (`bun run import-marketplace [--allow-shrink]`) fetches the `claude-plugins-official` and `knowledge-work-plugins` catalogs, deduplicates by `source` URL, and rewrites the seed — replacing the previously hand-maintained list. Re-import is guarded against accidental shrinkage (refuses to overwrite on a 0-plugin fetch or a >20% drop unless `--allow-shrink`); current entry counts and audit provenance live in the generated seed header.
- **Empirical compatibility harness (issue #100).** A research scaffold (`packages/dev-tools/src/compat-empirical/`) for measuring skill compatibility across `claude-code`, `claude-cowork`, and `claude-chat` — it joins VAT's static predictions with deterministic runtime observations and an LLM-judge read into a reality-vs-prediction matrix, as evidence for future detector improvements. Lives entirely in the private `dev-tools` package with n…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wires the slice-1 pure engine into
ExternalLinkValidatorand adds the fiveLINK_AUTH_*outcome codes. End-to-end: when an adopter setsresources.linkAuthinvibe-agent-toolkit.config.yaml,vat resources validatebypassesmarkdown-link-checkfor any URL whose host is claimed by a provider, issues an authenticatedfetch()against the rewritten URL, and classifies the response per design §7 into one of:LINK_AUTH_DEADnotFoundMeaning: dead)LINK_AUTH_DEAD_OR_UNAUTHORIZEDLINK_AUTH_FORBIDDENLINK_AUTH_UNAUTHORIZEDLINK_AUTH_UNVERIFIEDLINK_AUTH_DEAD = erroris the only error-severity code; design issue #113 §7 establishes that an authenticated 404 against an honest-404 host (e.g. SharePoint) is high-confidence link rot, satisfying the rule-design corpus-evidence bar.What's in the PR
packages/resources/src/link-auth-fetch.ts—fetchAuthenticated()with bounded redirect loop, cross-origin Authorization stripping (§8) that stays sticky across the rest of the chain (defeats token-laundering), 429/Retry-After honoring (§5.2) parsing delta-seconds and HTTP-date forms with a 60s DoS cap and a 250ms good-neighbor floor.packages/resources/src/link-auth-classify.ts— pure(status, providerCheck) → outcome+codeper §7.packages/resources/src/link-auth-config-build.ts— adopter config → engine config bridge with post-expansion validation againstInlineProviderSchema(catches typo'd macro overrides that pass throughMacroProviderSchema's.passthrough()); compile-time_KeysAgreetype assertion locks the schema's top-level field set to the engine'sProviderinterface.ExternalLinkValidatorwiring + a secondExternalLinkCacheinstance for auth-branch results, keyed by the rewritten URL (the originalblob/URL 404s — caching that would poison results) and scoped to a per-OS-user subdirectorycacheDir/auth-${sanitizedOsUser}/(§6.3). One-shotconsole.warnwhen the OS-user fallback chain lands at'default'(surfaces the cross-user-leak risk).version: 1; entries written under any other version produce a miss (forward-compat for slice 3).packages/agent-schema/test/docs/validation-codes.test.tsasserts everyCODE_REGISTRYentry has a matching### \CODE`heading indocs/validation-codes.md` — 126 assertions covering all 63 codes.docs/validation-codes.md.196 new tests across
link-auth-classify(13),link-auth-fetch(19),external-link-validator-auth(25),link-auth-config-build(10),validation-codes(+126 doc-iterator, +2 LINK_AUTH_* registry), andexternal-link-cache(+1 version-gate). Security-load-bearing tests pin the cross-origin Authorization strip, path-traversal sanitizer (table-driven over 9 pathologicalosUserinputs), post-expansion validation, unverified-no-cache invariant, cache-hit re-classification, andObject.hasOwnprototype-pollution defense on the{ use }discriminator.Adopter-visible breaking changes
LinkAuthConfig(the Zod-inferred adopter type) renames toLinkAuthProjectConfig— eliminates IDE auto-import ambiguity with the engine'sLinkAuthConfigfrom@vibe-agent-toolkit/utils. Engine type unchanged. TheLinkAuthConfigSchemaname is unchanged.auth-${osUser}/subdirectory undercacheDir; pre-existingexternal-links.jsonentries lacking the newversionfield trigger a one-time re-fetch on first run after upgrade.Fixed
ExternalLinkValidator.clearCache()andgetCacheStats()now operate on both caches. Previously they touched only the anonymous cache, so an adopter rotating a token would see stale 401/403 entries until the auth cache TTL expired.Deferred
.cmd-shim system test for the token dispatcher,VAT_LINKAUTH_ALLOW_COMMAND=0opt-out, contributor docs.Test plan
vat resources validateagainst a project with aresources.linkAuth.providers: [{ use: github }]configLINK_AUTH_DEAD_OR_UNAUTHORIZEDfires (warning) for a private GitHub URL the test identity can't seeLINK_AUTH_UNAUTHORIZEDfires whenGITHUB_TOKENis intentionally stale🤖 Generated with Claude Code