Add 0.2 archive format support with strict raw tar-header validation, versioned build-manifest dispatch, and classified entry verification

.changeset/protocol-0-2-consumer-support.md

@@ -0,0 +1,15 @@
+---
+"@agent-facets/protocol": minor
+---
+
+**Consumer support for archive format `0.2` (pre-1.0 breaking minor).** The protocol now verifies both legacy `0.1` and current `0.2` `.facet` archives with strict, exact `facetVersion` dispatch and no fallback between versions. This is the consumer-first release: verification support ships before any producer emits `0.2`.
+
+**Breaking API — `validateFacetArchive`.** The result shape is now `{ ok: true; data: VerifiedFacetArchive } | { ok: false; failure: ArchiveVerificationFailure }`. The previous `{ ok: false; errors: ValidationError[] }` arm is replaced by a single tagged `failure`. The success payload type `VerifiedArchive` is renamed to `VerifiedFacetArchive` and is now a discriminated union on `archiveVersion`: the legacy `0.1` arm keeps the flat `assets: VerifiedAsset[]` list, while the current `0.2` arm exposes `entries: VerifiedEntry[]` (each tagged `manifest` | `primary-asset` | `skill-companion` | `archive-only`). Consumers that read `.assets` unconditionally should migrate to the version-agnostic helpers `listVerifiedFiles(archive)` and `verifiedFileHashes(archive)`.
+
+**Structured failures.** `ArchiveVerificationFailure` is a tagged union (`container`, `invalid-json`, `duplicate-members`, `unsupported-facet-version`, `schema-violation`, `decompression`, `integrity`, `entry-integrity`, `validation`); classify on `failure.code` rather than parsing messages. No expected failure mode throws.
+
+**New public API.** `VerifiedFacetArchive`, `VerifiedEntry`, `ArchiveVerificationFailure`, `ValidateFacetArchiveResult`, `listVerifiedFiles`, `verifiedFileHashes`; versioned build-manifest and lockfile schemas plus their exact-dispatch parsers `parseBuildManifestDocument` and `parseLockfileDocument`; the shared archive plan (`planArchiveEntries`, `validateSupplementaryPath`, `portableCollisionKey`); strict raw tar-header validation (`validateRawTarEntries`, `RawTarValidationOptions`); and the archive-format constants `FACET_ARCHIVE_VERSION` (`0.2`), `LEGACY_FACET_ARCHIVE_VERSION` (`0.1`), and `SUPPORTED_FACET_VERSIONS`. `parseFacetArchive` now returns a version-tagged parsed build manifest and a structured `failure`.
+
+**Transitional exports retained.** `BuildManifestSchema`/`BuildManifest`, `LockfileSchema`/`Lockfile`, and `LOCKFILE_VERSION` (which equals the legacy value `1`, not the current `0.2`) remain exported and `@deprecated` for the compatibility window; they are removed once the engine lockfile-migration and producer work lands. Prefer the versioned parsers and `CURRENT_LOCKFILE_VERSION` in new code.
+
+This release intentionally carries **no** `@agent-facets/adapter` or `agent-facets` (CLI) version bump: the adapter API `0.0`→`0.1` cutover and the CLI `0.2` producer ship in later, separately gated releases. Other implementations of the spec (e.g. the registry) adopt this published package to gain dual-format verification.

openspec/changes/support-non-asset-files/design.md

@@ -59,13 +59,13 @@ Rationale: today's outer-exclusivity check and `collectArchiveEntries` already c
 
 ### D4: Build manifest — unconditional 0.2 output; strict version dispatch; pre-1.0 minor releases
 
-Every build produced after this change SHALL emit `facetVersion: 0.2`, whether or not the facet declares supplementary files. Asset-only facets therefore use the same current format as facets with supplementary files. Producers SHALL NOT conditionally emit `0.1`; `0.1` remains a legacy input format supported by consumers during a compatibility window and MAY be deprecated by a separate future change. "After this change" means from the producer release (Migration Plan step 5) onward: the earlier consumer-side bridge release intentionally continues emitting `0.1` while `0.2` verification deploys everywhere, and that sequencing is not an exception to this rule but the path to enabling it.
+Every build produced by the final CLI release SHALL emit `facetVersion: 0.2`, whether or not the facet declares supplementary files. Asset-only facets therefore use the same current format as facets with supplementary files. Producers SHALL NOT conditionally emit `0.1`; `0.1` remains a legacy input format supported by consumers during a compatibility window and MAY be deprecated by a separate future change. "By the final CLI release" distinguishes source implementation from publication: the complete producer may be implemented, tested, and merged while the released CLI still emits `0.1`, because no `agent-facets` Changeset is merged during that preparation window. A protocol-only consumer release and the registry deployment precede the held CLI release gate; withholding that release is the activation mechanism, not a runtime dual-format flag.
 
 The `0.2` build manifest SHALL replace the `assets` map with a single `files` map: canonical inner-tar path → `sha256:<hex>`, covering every entry — `facet.json`, primary asset files, and supplementary files. The map carries hashes only. Asset/supplementary classification is NEVER read from the build manifest; it is derived from the embedded `facet.json` via the archive plan (D3). The completeness rule is single: the `files` key set MUST exactly equal the observed inner-tar entry set.
 
 Verifiers SHALL dispatch on `facetVersion` exactly once, at parse time: the exact legacy schema and rules apply to `0.1`; the rules above apply to `0.2`; any other version returns a structured `UNSUPPORTED_FACET_VERSION` failure carrying the observed version and supported versions. A malformed `0.2` manifest MUST NOT be reinterpreted as `0.1` — no fallback between versions. A `files` key in a `0.1` manifest (or an `assets` key in `0.2`) fails schema validation, making the illegal combinations unrepresentable in validated data.
 
-The CLI SHALL render unsupported-version failures as upgrade guidance. For a known format transition, one CLI-side compatibility table SHALL map the facet format to the minimum supporting CLI release, producing guidance such as: “This facet uses archive format 0.2, which this CLI does not support. Update agent-facets to <minimum version> or later.” For an unknown future format, the CLI SHALL advise updating to the latest release without inventing a minimum version. Already-released CLIs cannot be retrofitted and MAY continue showing their existing generic validation error; a consumer-first bridge release SHOULD add this handling before any producer emits `0.2`.
+The CLI SHALL render unsupported-version failures as upgrade guidance. For a known format transition, one CLI-side compatibility table SHALL map the facet format to the minimum supporting CLI release, producing guidance such as: “This facet uses archive format 0.2, which this CLI does not support. Update agent-facets to <minimum version> or later.” For an unknown future format, the CLI SHALL advise updating to the latest release without inventing a minimum version. Already-released CLIs cannot be retrofitted and MAY continue showing their existing generic validation error. The final CLI release includes both the consumer rendering and producer switch after the protocol package and registry are already ready; a separately published bridge CLI is optional rather than a prerequisite.
 
 This archive-format boundary is distinct from package release versioning. While `@agent-facets/protocol` and `@agent-facets/adapter` remain pre-1.0, breaking contract changes SHALL increment each package's minor version rather than its major version. This change therefore ships in the next minor release of each package. The permanent protocol release policy SHALL be updated by this change to encode the pre-1.0 rule; after 1.0, breaking changes SHALL require a major release.
 
@@ -222,24 +222,24 @@ The `installation` spec's "Facet operations require compatible selected adapters
 - **[Unsupported format surprises installers]** → Build output SHALL display the emitted `facetVersion` and complete entry listing; install SHALL return a structured unsupported-version failure, and the CLI SHALL render actionable upgrade guidance from its single compatibility table.
 - **[Third-party adapter breakage]** → SDK-helper adapters inherit companion support via the helpers; only custom-I/O adapters must implement the widened contract. The tagged unions make the migration mechanical and exhaustively checkable.
 - **[A positional `0.0` adapter is silently accepted by a tagged CLI]** → The identifier bump to `0.1` (D8) makes the wire-contract change visible to exact-identifier compatibility; a `0.0` adapter is unsupported and fails closed before any contract method or state write. Fixtures MUST prove a positional `0.0` bundle is rejected by a `{0.1}` CLI.
-- **[Old `0.0` adapters break at the CLI cutover]** → Consumer-first release ordering (Migration Plan step 4 before step 5): SDK and all three first-party adapters publish `0.1` before any CLI requires it; existing `0.0` CLIs keep selecting compatible `0.0` releases. Recovery from a broken install is the reinstall command the compatibility diagnostic already surfaces.
+- **[Old `0.0` adapters break at the CLI cutover]** → Staged release ordering: SDK and all three first-party adapters publish `0.1` before the held CLI-only Changeset is merged; existing `0.0` CLIs keep selecting compatible `0.0` releases. Recovery from a broken install is the reinstall command the compatibility diagnostic already surfaces.
 - **[Two declaration sites could confuse authors]** → The disjointness rule (D1/D7) yields a precise error pointing at the correct site; edit-flow detection (D11) writes declarations to the right place automatically.
 - **[Existing source manifests use slash-namespaced assets or duplicate skill/command names]** → Their published `0.1` archives remain consumable, but rebuilding as `0.2` fails with actionable validation errors before output is changed. Authors MUST rename the assets; invalid `0.2` manifests are never interpreted using legacy rules.
 
 ## Migration Plan
 
-Consumer-first: verification support ships everywhere before any producer can emit the new format.
+Consumer-first publication is controlled by package-specific Changesets. Source implementation may continue in parallel, but no released producer emits the new format until both consumer and adapter gates are proven.
 
-1. **protocol (next minor release)**: adopt the pre-1.0 breaking-release policy; add the D3 plan operation, D7 path grammar, D9 current-format asset-name grammar and shared namespace validation, D4 versioned build-manifest schemas with strict dispatch and structured unsupported-version results, D5 verification, D6 tagged parsed results, and D10 lockfile `0.2` with exact legacy-1/current-0.2 dispatch, per-file integrity schemas, and mismatch result types — plus immutable fixtures for both archive versions.
-2. **engine/CLI bridge (consumer side)**: loaders and cache consume the tagged parsed result; verified legacy lockfiles and receipts migrate during normal installs while frozen legacy state retains legacy behavior; unsupported versions and per-file integrity mismatches render actionable diagnostics. Producers still emit `0.1` in this bridge release.
-3. **cafe registry (out of repo, hard gate)**: deploys `0.2` verification while retaining `0.1`. Producer enablement MUST NOT ship before this lands.
-4. **adapter SDK (next minor release) + first-party adapters**: tagged payload unions, atomic bundle helpers (D8), injected-failure tests; the SDK bumps `ADAPTER_API_VERSION` `0.0`→`0.1` and every first-party adapter migrates and declares package/runtime API `0.1` — claude-code, opencode, and codex. This SDK/adapter release publishes `0.1` to npm *before* step 5. Until it does, existing `0.0` CLIs keep selecting the highest compatible `0.0` adapter release; no CLI whose supported set is `{0.1}` ships until all three first-party adapters have published `0.1`.
-5. **engine/CLI producer release**: every build emits archive `0.2` and writes only lockfile `0.2`; materialize passes skill bundles; receipts write the `0.2` ownership shape; create and edit ship the D11 first-class `README.md`/`README` authoring flow; generic supplementary-file detection/add/remove flows land.
-6. **docs** (Article III): update `docs/specification/archive.mdx` (membership rules, single `files` hash map, version dispatch), `build.mdx` (plan derivation, validation-before-cleanup, displayed version), `manifest.mdx` (both `files` fields, minimum-producer-version warning, linked Agent Skills naming convention, single-segment asset names versus nested companion paths, shared skill/command namespace), `integrity.mdx` (all-entry and per-locked-file coverage), `lockfile.mdx` (lockfile `0.2`, per-materialized-file integrity, legacy-alpha-1 migration, stable-v1 regeneration boundary), `commit.mdx` (receipt ownership and transactional reconciliation), `install.mdx` (materialization boundary, atomic skill bundles, mismatch diagnostics), `docs/guides/create-your-first-facet.mdx` and `docs/guides/install-facets.mdx` (asset-only phrasing, README workflow), root `README.md`.
+1. **Protocol-only release (first handoff)**: merge the proposal, protocol-model, and archive-verification stack; adopt the pre-1.0 breaking-release policy; publish the D3 plan operation, D7 path grammar, D9 current-format asset-name grammar and shared namespace validation, D4 versioned build-manifest schemas with strict dispatch and structured unsupported-version results, D5 verification, D6 tagged parsed results, and D10 lockfile `0.2` schemas — plus immutable fixtures for both archive versions. Its pre-1.0 minor Changeset names only `@agent-facets/protocol`: no adapter package and no `agent-facets` CLI release is attached.
+2. **Parallel registry lane (out of repo)**: after the protocol release is published, the cafe registry pins that npm version directly, migrates to the tagged verifier and complete file-hash view, proves `0.1` retention plus `0.2` acceptance, and deploys. It does not require a local facets checkout or a released `0.2` CLI. The deployed registry remains a hard gate for the final CLI release.
+3. **Parallel facets source lane (unreleased)**: engine and CLI loaders, lockfile/receipt migration, per-file materialization, producer output, create/edit authoring, diagnostics, and documentation may be completed, reviewed, and merged while the registry lane proceeds. The candidate source emits `0.2` for tests and stage interoperability, but no `agent-facets` Changeset is merged, so released CLIs continue emitting `0.1`. There is no long-lived runtime flag or dual current-producer mode.
+4. **Adapter SDK + first-party adapter release**: publish the tagged payload unions, atomic bundle helpers, and injected-failure coverage; bump `ADAPTER_API_VERSION` `0.0`→`0.1`; and publish claude-code, opencode, and codex with package/runtime API `0.1`. This Changeset names only the SDK and three adapters. Existing `0.0` CLIs keep selecting the highest compatible `0.0` releases; no CLI whose supported set is `{0.1}` ships until all three first-party adapters have published `0.1`.
+5. **Held final CLI gate**: prepare a tiny, unmerged `agent-facets`-only Changeset PR after source implementation is complete. Before the user authorizes its merge, verify the published protocol API from a clean install, all three published adapter declarations, the deployed registry's dual-version behavior, the full repository suite, and a candidate CLI `0.2` build/publish/readback against stage. Merging the held Changeset and generated version-package PR is the sole public activation: every new build then emits archive `0.2`, lockfiles/receipts use `0.2`, materialization passes skill bundles, and create/edit ship the first-class README and supplementary-file flows.
+6. **Documentation and release notes (Article III)**: complete documentation before the held CLI gate is authorized. Update `docs/specification/archive.mdx` (membership rules, single `files` hash map, version dispatch), `build.mdx` (plan derivation, validation-before-cleanup, displayed version), `manifest.mdx` (both `files` fields, minimum-producer-version warning, linked Agent Skills naming convention, single-segment asset names versus nested companion paths, shared skill/command namespace), `integrity.mdx` (all-entry and per-locked-file coverage), `lockfile.mdx` (lockfile `0.2`, per-materialized-file integrity, legacy-alpha-1 migration, stable-v1 regeneration boundary), `commit.mdx` (receipt ownership and transactional reconciliation), `install.mdx` (materialization boundary, atomic skill bundles, mismatch diagnostics), `docs/guides/create-your-first-facet.mdx` and `docs/guides/install-facets.mdx` (asset-only phrasing, README workflow), root `README.md`, and the protocol-only, adapter-only, and final CLI release notes without duplicating version sources.
 
 Before the future stable lockfile v1 release, legacy-alpha-1 parsing SHALL be removed and replaced with actionable delete-and-regenerate guidance for old-shape numeric-1 files.
 
-Rollback: before any `0.2` artifact is published, producer rollout MAY revert to the bridge release. After publication or multi-file skill materialization, producer emission MAY be paused, but consumers MUST retain `0.1` and `0.2` verification plus receipt-aware deletion. Removing `0.1` support requires a separately reviewed deprecation change. For the adapter API axis, rollback means restoring/reinstalling compatible adapter and CLI releases: because compatibility is exact-identifier and cannot be inferred from package semver, a `0.1` CLI cannot be made to accept a `0.0` adapter by changing versions — recovery is reinstalling a `0.1` adapter (or downgrading the CLI to a `0.0` release paired with `0.0` adapters), never a version bump alone.
+Rollback: before the held CLI Changeset is merged, public producer rollout is cancelled simply by leaving that release gate unmerged; the published protocol, deployed registry, and adapter releases remain backward-compatible consumer preparation. After a `0.2` artifact is published or a multi-file skill is materialized, producer emission MAY be paused, but consumers MUST retain `0.1` and `0.2` verification plus receipt-aware deletion. Removing `0.1` support requires a separately reviewed deprecation change. For the adapter API axis, rollback means restoring/reinstalling compatible adapter and CLI releases: because compatibility is exact-identifier and cannot be inferred from package semver, a `0.1` CLI cannot be made to accept a `0.0` adapter by changing versions — recovery is reinstalling a `0.1` adapter (or downgrading the CLI to a `0.0` release paired with `0.0` adapters), never a version bump alone.
 
 ## Open Questions