diff --git a/openspec/changes/support-non-asset-files/adversarial/artifacts/design.md b/openspec/changes/support-non-asset-files/adversarial/artifacts/design.md new file mode 100644 index 00000000..6c207186 --- /dev/null +++ b/openspec/changes/support-non-asset-files/adversarial/artifacts/design.md @@ -0,0 +1,169 @@ +## Context + +Facet archives currently have one implicit membership rule: `facet.json` plus the conventional file for every declared skill, agent, and command. The same assumption appears in schema validation, build collection, per-file hashing, archive verification, parsed archive data, installation receipts, and the adapter SDK. Supporting files cannot therefore be added only at archive assembly; their declaration, integrity, classification, and ownership must remain consistent across the whole pipeline. + +This design treats the embedded `facet.json` as the single source of truth for archive membership and classification. The build manifest records hashes, not a second description of which paths are assets or supplementary files. A supplementary path inside a declared skill directory belongs to that skill; every other supplementary path is archive-only metadata and is never materialized. + +The change crosses the published protocol, engine, adapter SDK, first-party adapters, CLI error presentation, and the cafe registry's verifier. Existing `facetVersion: 0.1` archives must remain consumable, while an archive containing supplementary files is intentionally not consumable by a legacy verifier. + +## Goals / Non-Goals + +**Goals:** + +- Authors MUST be able to explicitly declare regular files to include in a facet archive. +- Build and verification MUST derive exactly the same canonical archive-entry set from `facet.json` and MUST reject missing, extra, unsafe, duplicate, or colliding paths. +- Every inner-archive file MUST have a per-file hash, and supplementary bytes MUST affect the archive integrity hash. +- A declared file below a declared skill directory MUST install, update, and delete as part of one skill operation. +- Supplementary files outside declared skill directories MUST remain verified archive metadata and MUST NOT acquire independent asset identity, scope, lockfile tuples, or install destinations. +- New consumers MUST continue to accept valid legacy `0.1` archives. + +**Non-Goals:** + +- This change does not add recursive discovery, glob declarations, or implicit inclusion of source-tree files. +- This change does not display README content, add `facet info`, or define a registry presentation API. +- This change does not materialize companion files for agents or commands. +- This change does not preserve symlinks, hard links, directory entries, executable bits, timestamps, ownership, or other filesystem metadata. +- This change does not make supplementary files selectable or independently installable. + +## Decisions + +### 1. `facet.json` declares an explicit list of canonical file paths + +The facet manifest SHALL gain an optional top-level `files` array of strings. Each member names exactly one source-root-relative regular file using `/` as its separator. Patterns, directories, and recursive discovery SHALL NOT be supported. + +A declared path MUST: + +- be non-empty, relative, and already canonical; +- contain no empty, `.` or `..` segment, backslash, NUL, absolute-path prefix, drive prefix, or URL-like prefix; +- resolve through existing parents to a regular file inside the facet root; +- not be a symlink or hard link; +- not be `facet.json`, `build-manifest.json`, `archive.tar.gz`, or any conventional primary asset path derived from the manifest; and +- not collide with another derived archive path by exact spelling, canonical Unicode form, portable case folding, resolved source identity, or file/directory prefix. + +Paths under `skills//` SHALL be accepted only when `` is a declared skill and the path has content below that directory. `skills//SKILL.md` remains derived from the skill declaration and MUST NOT also appear in `files`. This makes skill ownership a function of path plus the existing skill declaration rather than a second owner field that could disagree. Declared paths under `agents/` or `commands/` are permitted but remain archive-only metadata. + +Supplementary files MAY be empty and MAY contain arbitrary bytes. Asset-specific rules such as non-empty Markdown and no YAML front matter continue to apply only to primary asset files. Build SHALL hash and preserve supplementary bytes exactly. + +**Alternatives considered:** + +- Glob patterns and directory declarations were rejected because membership would depend on ambient source-tree contents and review of `facet.json` would not reveal the archive's exact file set. +- Descriptor-local companion lists were rejected because they would create two declaration mechanisms and would not cover root metadata files. +- An object per file carrying an asset kind or owner was rejected because kind and ownership are already derivable, and duplicated classification could drift. + +### 2. Archive format version 1 carries a path-to-hash table for every file + +An archive with a non-empty `files` declaration SHALL use `facetVersion: 1` in `build-manifest.json`. Version 1 SHALL replace the misleading `assets` hash map with a `files` map from canonical inner-archive path to `sha256:` computed over the exact file bytes. The map SHALL include `facet.json`, every primary asset file, and every declared supplementary file. It SHALL carry hashes only; asset/supplementary classification SHALL be derived from the embedded facet manifest. + +The inner archive entry set for version 1 SHALL equal: + +1. `facet.json`; +2. each conventional primary asset path derived from the embedded manifest; and +3. each canonical path in `facet.json.files`. + +No other entry is valid. Entries SHALL be regular files only and lexicographically ordered by canonical UTF-8 path bytes. Existing deterministic metadata rules remain unchanged. The integrity hash SHALL continue to cover the canonical uncompressed inner tar bytes, so any supplementary-file change alters integrity. + +A new producer MAY continue to emit version `0.1` when `files` is absent or empty. This preserves old-consumer interoperability for asset-only facets. A new verifier SHALL dispatch on the build-manifest version: it SHALL apply the exact legacy schema and membership rules to `0.1`, the version 1 rules above to `1`, and reject unsupported versions with structured data. It SHALL NOT reinterpret a malformed version 1 archive as `0.1`. + +This archive-version boundary is separate from release versions: publishing version 1 support requires a new major release of the protocol package because archive acceptance and the public parsed result change, and a new major release of the adapter SDK because its asset operations change. + +**Alternatives considered:** + +- Reusing `facetVersion: 0.1` was rejected because old consumers reject the expanded entry set and the build-manifest hash shape changes. +- Keeping an `assets` map and adding a second `files` map was rejected because the two maps could overlap, omit entries, or disagree about classification. +- Emitting version 1 for every asset-only build was rejected as an unnecessary compatibility loss; content that fits the legacy contract can remain legacy-encoded. + +### 3. Build and verification share one path-derivation operation + +The protocol package SHALL expose one pure operation that validates a facet manifest's file declarations and derives a tagged archive plan. The plan SHALL distinguish manifest, primary assets, skill companions, and archive-only supplementary files. Build collection and archive verification MUST consume this same operation rather than maintaining separate allowlists. + +Build SHALL resolve the plan against the source root, validate containment and regular-file identity, read every planned path as bytes, and fail with structured errors before writing `dist/` if any path is missing, unsafe, duplicated, colliding, or not a regular file. The existing cleanup of `dist/` MUST NOT occur until all declared source inputs have been validated, preventing a declaration from being destroyed before its missing-file error can be reported. + +Verification SHALL parse tar headers without normalizing them into a lossy map. It SHALL reject duplicate paths, non-regular entries, non-canonical paths, unsafe paths, and prefix collisions before exposing contents. After validating and parsing `facet.json`, it SHALL derive the expected plan and compare the expected and observed path sets for exact equality. It SHALL then require one version 1 `files` hash for every expected path, no hash for any other path, and byte-verify every hash. All expected failures SHALL remain structured result variants rather than thrown errors. + +The successful parsed result SHALL carry the verified manifest, primary assets, skill-companion bytes grouped by owning skill, and archive-only supplementary bytes as distinct tagged data. It SHALL not represent classification through optional fields whose combinations can disagree. + +**Alternatives considered:** separate engine and verifier derivation was rejected because the current outer-exclusivity drift demonstrates that duplicated membership logic is a security boundary. + +### 4. Skills use a tagged bundle contract; other assets remain single-file + +The adapter SDK's install, read, and delete requests/results SHALL become tagged unions keyed by asset type: + +- a skill variant carries its `SKILL.md` text plus a canonical map of companion paths relative to the skill root and their bytes; +- agent and command variants carry their existing single Markdown content; and +- no variant for supplementary files exists. + +This prevents an agent or command request from accidentally carrying companions and prevents a skill request from omitting its bundle shape. Metadata continues to apply to the primary asset, and adapters SHALL reconstruct tool-specific metadata only in the primary file; companion bytes SHALL be preserved without front-matter transformation. + +A skill install SHALL be one adapter operation. It MUST stage the complete replacement, remove previously owned companion paths that are absent from the new bundle, and commit or roll back without leaving a partial bundle. Skill deletion MUST remove the primary file and all recorded owned companions as one operation, while retaining unrelated files not listed as owned. Expected adapter failures MUST be structured result values. + +First-party filesystem helpers SHALL centralize containment checks, staging, commit/rollback, owned-file removal, and empty-directory pruning so adapters do not duplicate this security-sensitive behavior. Adapters MAY choose tool-specific roots and representations, but MUST NOT allow a companion path to escape the resolved skill root. + +**Alternatives considered:** + +- Passing every file as an independently installable asset was rejected because it would create false asset identity, scope, metadata, and lockfile semantics. +- Letting the engine write companions directly was rejected because adapters own all storage paths and formats. +- Deleting the entire skill directory was rejected because it can remove unowned user files; deletion is based on recorded ownership instead. + +### 5. The machine-local receipt, not the lockfile, records skill-file ownership + +The version-controlled lockfile SHALL retain its existing `{scope, type, name}` asset tuples. Supplementary files SHALL NOT appear as lockfile assets and SHALL NOT require a lockfile-version bump. + +The machine-local receipt SHALL replace its uniform asset record with a tagged union. Agent and command records retain scope, type, and name. A skill record additionally requires the complete set of owned installed paths, including its primary file and adapter-derivable companion paths. There is no optional `files` field whose presence implicitly decides the record kind. + +On update, the installer SHALL pass the prior owned set and the new verified bundle to each adapter so stale companions are removed. On facet removal, it SHALL use the receipt alone to delete all owned skill files without cache or network access. Receipt path containment and project-identity checks remain mandatory. A legacy receipt containing only skill tuples SHALL be migrated conservatively: the primary skill file is known and removable, but unknown historical companions do not exist because legacy archives could not install them. + +The install journal SHALL snapshot receipt changes and adapter operations so a failure restores the previous receipt and materialized state. Archive-only supplementary files SHALL remain in the verified archive/cache and parsed artifact but SHALL never enter the receipt. + +**Alternatives considered:** putting companion paths in the lockfile was rejected because the lockfile records facet assets, not machine- and adapter-specific materialization ownership, and because doing so would make supplementary files look independently installable. + +### 6. Rollout is consumer-first and documentation is part of the change + +Rollout SHALL occur in this order: + +1. Release version 1 verification and legacy `0.1` compatibility in every consumer, including the cafe registry, while producers still default to legacy output. +2. Release the adapter SDK major and updated first-party adapters, then update installation receipt migration. +3. Enable producer support for `facet.json.files` and version 1 archives. +4. Publish fixtures proving cross-implementation acceptance and rejection at both version boundaries. + +The authoring and build documentation SHALL warn that older builders tolerate unknown manifest fields and can therefore ignore `files`; projects using supplementary files MUST pin or require a producer version that supports archive version 1. Registries that have not deployed version 1 support will reject those archives by design. + +The following files MUST be updated together with implementation: + +- `docs/specification/archive.mdx` +- `docs/specification/build.mdx` +- `docs/specification/manifest.mdx` +- `docs/specification/integrity.mdx` +- `docs/guides/create-your-first-facet.mdx` +- `docs/guides/install-facets.mdx` +- root `README.md` + +`docs/specification/lockfile.mdx` MUST be reviewed and SHOULD explicitly state that companion ownership is receipt-only while lockfile asset tuples remain unchanged. This design does not require a lockfile schema change. + +Rollback MAY disable production of version 1 archives, but consumers and the registry MUST retain version 1 verification once such archives have been published. Already-published version 1 artifacts cannot be made consumable by legacy clients without republishing an asset-only version. + +## Risks / Trade-offs + +- **[Old builders accept but ignore the new unknown `files` field]** → Documentation MUST declare the minimum supporting producer release, examples SHOULD pin it, and CI compatibility fixtures MUST prove that only supporting producers emit the declared files. This cannot be repaired retroactively in already-released tolerant parsers. +- **[Path aliases or crafted tar headers bypass membership checks]** → One canonical path validator and archive-plan derivation MUST be shared; verifier tests MUST cover traversal, absolute and drive paths, backslashes, duplicate headers, Unicode/case aliases, prefix collisions, symlinks, hard links, and undeclared entries. +- **[A failed skill update leaves a half-written directory]** → The adapter contract MUST require stage/commit/rollback behavior, and integration tests MUST inject failure at each write/delete step. +- **[Receipt corruption causes over-deletion]** → Receipts remain untrusted; project identity, adapter-root containment, and exact owned-path validation MUST precede deletion. Unowned paths MUST never be deleted. +- **[Arbitrary companion bytes increase memory or decompression pressure]** → Existing caller-supplied decompression limits SHALL apply to the complete archive, and registry/CLI policy MAY impose total-size, per-file-size, and entry-count limits without changing archive semantics. +- **[Two supported archive versions create implementation branches]** → Version dispatch MUST occur once at parsing, with immutable fixtures for both schemas and no fallback between them. +- **[Cafe accepts producer output before it can verify it]** → Producer enablement MUST remain gated until registry version 1 verification is deployed. +- **[Conditional legacy output surprises authors]** → Build output MUST display the emitted archive version and complete file listing. + +## Migration Plan + +1. Add canonical path, archive-plan, versioned build-manifest, and parsed-result types in the protocol package with legacy fixtures unchanged. +2. Add version 1 build and verification fixtures, including byte tampering and every path-security failure class. +3. Migrate engine build/load/cache code to consume the tagged parsed result without materializing archive-only files. +4. Release the adapter SDK major, migrate first-party adapters, and add atomic skill-bundle contract tests. +5. Migrate receipt loading/writing and exercise install, update, removal, rollback, frozen install, offline removal, and pulled-lockfile drift scenarios with multi-file skills. +6. Deploy cafe verification for both archive versions before enabling publication of version 1 artifacts. +7. Update all listed documentation and enable producer support. + +Rollback SHALL stop new version 1 production and restore the prior adapter package only before multi-file skills are installed. After version 1 publication or materialization, verifier support and receipt-aware deletion MUST remain available even if authoring support is temporarily disabled. + +## Open Questions + +None. Policy limits for archive size and entry count remain consumer configuration rather than protocol-format decisions. diff --git a/openspec/changes/support-non-asset-files/adversarial/reviews/design-review.md b/openspec/changes/support-non-asset-files/adversarial/reviews/design-review.md new file mode 100644 index 00000000..bf2b0fd4 --- /dev/null +++ b/openspec/changes/support-non-asset-files/adversarial/reviews/design-review.md @@ -0,0 +1,94 @@ +# Design comparison: Main vs. Adversary + +This review compares **Main** at `design.md` with **Adversary** at `adversarial/artifacts/design.md`. + +## Grading bar + +The designs were graded for value delivered against the reconciled proposal, RFC 2119 precision, atomic and testable requirements, correct OpenSpec/design mechanics, explicit compatibility and rollout boundaries, security completeness at archive/path trust boundaries, and coverage sufficient to derive specs and tasks without inventing policy later. + +## Coverage summary + +Both versions correctly preserve explicit archive membership, reject globs, keep supplementary files distinct from independently installable assets, treat supplementary content as opaque bytes, materialize companions only for skills, leave the lockfile asset model unchanged, conditionally preserve legacy `0.1` output for asset-only facets, and require the registry to understand the new archive format before accepting it. + +Main is stronger on author-facing declaration ergonomics, choosing the format revision `0.2` consistently with the current `0.1` convention, concrete scaffold/edit surfaces, and explicitly keeping archive-only files out of adapter inputs. Adversary is substantially stronger at the security and lifecycle boundaries: one normalized archive plan, exhaustive collision and tar-entry validation, strict version dispatch, tagged adapter and receipt contracts, explicit atomic replacement/rollback semantics, consumer-first rollout, old-builder behavior, and protocol-package semantic versioning. + +The central difference is that Main often expresses invariants through optional parallel fields and prose (“only ever populated for `type === 'skill'`”), while Adversary encodes variants as tagged data and requires shared derivation. That difference is material because adapter operations, receipt deletion, and archive membership are security and data-loss boundaries. + +## Decision-by-decision divergences and merge recommendations + +### D1–D2: Declaration shape and exact enumeration + +**Main is stronger on ownership ergonomics.** `SkillDescriptor.files` makes companion ownership visible where the skill is declared, while a disjoint top-level `files` list cleanly expresses archive-only metadata. Adversary's single root-relative list is simpler and makes archive membership literal in one place, but skill ownership must then be inferred from path containment. + +Main overstates that invalid ownership is “structurally impossible”: both lists still contain unrestricted strings, and disjointness, path safety, declared-skill membership, and collision freedom remain runtime schema constraints. Its two declaration sites also require a single downstream normalization step or they will encourage duplicated build/verifier logic. + +**Merge recommendation:** keep Main's two authoring sites, but require one pure protocol operation to validate both and normalize them into the tagged archive plan described by Adversary (`manifest`, `primary-asset`, `skill-companion`, `archive-only`). Build, hashing, verification, parsed results, and installation MUST consume that plan. Preserve exact enumeration/no globs and Main's embedded-manifest rationale. + +### D3: Build-manifest shape and version boundaries + +**Main is stronger on the archive revision identifier:** `0.2` follows the documented numeric progression from `0.1`; Adversary's `1` is an unnecessary naming jump. Both correctly retain byte-identical `0.1` output when no supplementary files are present. + +**Adversary is stronger on hash-map authority.** Main's parallel `assets` and `files` maps duplicate classification already derivable from the embedded manifest and permit overlap/disagreement states that must be detected later. The reconciled proposal requires an unambiguous distinction, but it does not require that distinction to be duplicated in the build manifest. One all-entry path-to-hash table plus the embedded manifest and normalized archive plan is unambiguous and has one completeness rule. + +Main also calls the protocol work “additive” in Migration Plan step 1 and names only the adapter SDK major release. That conflicts with the reconciled proposal's explicit protocol/archive breaking boundary and `openspec/specs/protocol/spec.md`, which requires backward-incompatible protocol requirements to ship in a new major version. + +**Merge recommendation:** use conditional `facetVersion: 0.2`, retain the exact `0.1` schema for legacy output, and use a single `files` hash map containing every inner entry in `0.2`; derive classification only from embedded `facet.json`. Require strict one-time dispatch by version with no malformed-`0.2` fallback to `0.1`. State separately that the published protocol package ships this in a new major release, as does the adapter SDK. + +### D4 and D6: Shared derivation, path safety, and archive verification + +**Adversary is decisively stronger and exposes a blocking gap in Main.** Main covers traversal, absolute paths, backslashes, primary-path collisions, duplicates, missing declarations, and exact observed membership. It does not settle several ways those checks can be bypassed or become platform-dependent: empty/`.` segments, NUL and drive-prefixed paths, canonical Unicode/case aliases, resolved source identity, file/directory prefix collisions, symlinks/hard links, non-regular tar entries, duplicate tar headers, and parsers that first collapse entries into a lossy path map. + +Adversary also catches two operational details Main omits: source inputs must be validated before any `dist/` cleanup can destroy a declared input, and every expected parser/build failure must remain a structured result rather than an exception. + +**Merge recommendation:** add Adversary's shared archive-plan decision and exhaustive path/tar checks to Main. Specify that verification validates raw headers before constructing a map, rejects duplicate and non-regular entries, compares expected and observed canonical path sets exactly, and verifies exactly one hash per expected path. Specify regular-file containment and source-identity checks at build time, including symlink/hard-link policy and portable alias/prefix collisions. Add a test matrix for every named failure class. This is blocking before specs/tasks because it defines the supply-chain boundary. + +### D5: Opaque bytes and parsed representation + +**Main is stronger on a concrete implementation touchpoint** by widening `ArchiveEntry.content` and explicitly preserving binary/empty content. Adversary agrees on exact bytes but goes further by requiring the successful parsed result to keep primary assets, skill companions grouped by owner, and archive-only supplementary files as distinct tagged data. + +`string | Uint8Array` alone does not encode which content is prompt text and which is opaque, so downstream code can still apply the wrong transformation. + +**Merge recommendation:** keep Main's opaque-byte requirements, but make the normalized and parsed public results tagged by entry kind as Adversary requires. Text decoding/front-matter logic applies only after narrowing to a primary asset; supplementary data remains bytes. + +### D7: Adapter contract and atomic skill lifecycle + +**Adversary is decisively stronger and exposes a blocking type/atomicity defect in Main.** Main proposes optional `companions`/`companionPaths` parameters beside `assetType` and relies on the prose invariant that they are populated only for skills. That represents illegal combinations (agent with companions, skill call accidentally omitting its bundle) and gives implementations no exhaustive branch. The widened methods also do not, by themselves, guarantee that custom adapters stage a complete replacement or roll back partial writes/deletes. + +Adversary uses tagged variants keyed by asset type, makes a skill bundle one operation, requires removal of formerly owned but now absent companions, and explicitly requires stage/commit/rollback plus structured expected failures. + +**Merge recommendation:** replace Main's optional-parameter trio with tagged request/result unions. A skill variant MUST carry primary text and a canonical companion-byte map (empty is legal); agent/command variants MUST NOT carry companions. Define one atomic replacement/delete contract for the entire skill bundle, including rollback and stale-owned-file removal, and return structured failure values. Centralize containment, staging, commit/rollback, owned-path deletion, and empty-directory pruning in SDK helpers, while requiring equivalent behavior from custom-I/O adapters. Add injected-failure tests at every write/delete/commit boundary. + +### D8: Receipt ownership and drift removal + +**Adversary is stronger.** Main again uses an optional companion list whose legal presence depends on `type`, and its migration explanation is internally inconsistent: a truly legacy receipt cannot refer to companions because legacy archives could not install them, so there is no justified “one-install-cycle” unknown companion orphan. Conversely, once a supporting version installs companions, forgetting their ownership is unacceptable because offline removal can no longer be exact. + +Adversary requires a tagged receipt record and a complete owned set for skills, keeps archive-only files out of receipts, validates receipt paths as untrusted input, and couples receipt rollback to adapter rollback. + +**Merge recommendation:** parse persisted legacy records at the receipt boundary and refine them into an internal tagged union: agent/command records have no companion field; skill records require a complete canonical owned-path set (with legacy skill tuples migrated to the known primary plus an empty companion set). Persist the refined shape after a successful install. Store canonical paths relative to the adapter-owned skill root, validate containment and project identity before deletion, never delete unowned paths, and journal receipt plus materialization as one rollback unit. Remove the unsupported orphan-cycle claim. + +### D9–D10: Authoring and materialization boundary + +**Main is stronger on authoring coverage** by addressing `facet edit`, and its D10 data-flow boundary is excellent: archive-only files cannot be materialized accidentally because adapters never receive them. Adversary should have named both explicitly. + +However, Main's recommendation that `facet create` scaffold and declare a README by default makes every newly scaffolded facet opt into `0.2`, old-client rejection, and the old-builder-ignore hazard. That conflicts with consumer-first rollout and weakens the practical value of conditional legacy output. + +**Merge recommendation:** retain D10 and edit-flow discovery/add/remove support, but do not make supplementary README declaration an unconditional scaffold default. Make it an explicit author choice or gate the default until the producer minimum version and registry/consumer rollout are in place. If removal of vanished declarations is in scope, decide it now with deterministic behavior; do not leave it as “if cheap” task policy. + +### Rollout, documentation, risks, and open questions + +**Adversary is stronger on rollout.** It explicitly sequences verifier consumers and cafe before producers, requires immutable cross-version fixtures, warns that tolerant old builders may accept `files` but silently omit the bytes, and states that verifier support cannot be rolled back after `0.2` artifacts are published. Main only partially captures registry sequencing and does not address old builders ignoring the new manifest fields. + +**Main is stronger on documentation breadth** by adding install documentation and concrete authoring guidance. Its open question about registry README presentation is already a declared non-goal, and archive policy limits can be explicitly deferred to consumer configuration rather than left unresolved. The `facet edit` removal behavior needs a decision if it is to generate tasks. + +**Merge recommendation:** adopt Adversary's consumer-first rollout and rollback constraints, including minimum-producer documentation and cross-version accept/reject fixtures; retain Main's added install documentation. Close the README-presentation question as out of scope, state that size/count policy remains consumer configuration for this change, and settle edit removal behavior before task generation. + +## Blocking cross-cutting items + +1. **Security boundary:** define the shared normalized archive plan and exhaustive raw-tar/path/collision/regular-file checks before specs or implementation tasks are considered complete. +2. **Atomic lifecycle:** replace optional adapter/receipt companion fields with tagged variants and specify observable all-or-nothing install/update/delete plus rollback and offline removal behavior. +3. **Compatibility boundary:** state both archive revision (`0.2` conditionally) and package major-version requirements; require strict version dispatch, consumer-first cafe rollout, old-builder warnings, and immutable compatibility fixtures. +4. **Single source of truth:** avoid parallel build-manifest classification maps; membership/classification comes from embedded `facet.json` through the shared archive plan, while the build manifest carries hashes. + +## Overall merge recommendation + +Use Main as the structural base because its two declaration sites, `0.2` naming, authoring flow, and materialization boundary are concrete and useful. Replace its parallel hash-map, optional adapter parameters, optional receipt ownership, partial path grammar, and under-specified rollout with Adversary's single normalized plan, one all-entry hash map, tagged unions, atomic rollback contract, exhaustive security checks, and consumer-first compatibility plan. The four blocking items above should be resolved in the design before delta specs or tasks lock in weaker contracts. diff --git a/openspec/changes/support-non-asset-files/adversarial/state.json b/openspec/changes/support-non-asset-files/adversarial/state.json index cdfa89f7..d888ce05 100644 --- a/openspec/changes/support-non-asset-files/adversarial/state.json +++ b/openspec/changes/support-non-asset-files/adversarial/state.json @@ -13,6 +13,18 @@ "comparedAt": "2026-07-13T17:59:26Z", "reconciledAt": "2026-07-13T18:14:00Z", "notes": "Adopted all 7 review findings: added mandatory Non-goals section; explicit manifest declaration as single source of archive membership (syntax deferred to design); full validation boundary (missing/undeclared/unsafe/colliding paths); dropped premature build-manifest assets-map commitment in favor of asset/supplementary distinction; atomic multi-file skill lifecycle with drift-removal receipts; supplementary files prohibited from becoming independently addressable assets; compatibility upgraded from forward-compat note to BREAKING with explicit version boundary + legacy compat tests; Impact extended with security/compat test matrix, both guides, and root README. Partially adopted lockfile.mdx wording (review required, update conditional on design outcome)." + }, + "design": { + "artifactId": "design", + "mainPaths": ["design.md"], + "adversarialPaths": ["adversarial/artifacts/design.md"], + "reviewPath": "adversarial/reviews/design-review.md", + "dependencies": ["proposal"], + "status": "reconciled", + "authoredAt": "2026-07-13T18:10:26Z", + "comparedAt": "2026-07-13T18:19:40Z", + "reconciledAt": "2026-07-13T18:41:00Z", + "notes": "Adopted all 10 review findings (nothing material rejected; Main's structural base retained per the review itself). Added D3 shared tagged archive-plan derivation as single source of membership/classification; 0.2 build manifest replaces parallel assets+files maps with one all-entry hash map + strict version dispatch; protocol package and adapter SDK each marked as new major releases; D7 grammar expanded with exhaustive path/tar checks (NUL, drive prefixes, Unicode/case-fold aliases, prefix collisions, symlink/hardlink identity, duplicate headers, non-regular entries, dist/ cleanup ordering) + per-class test matrix; D8 adapter contract rewritten to tagged payload unions with atomic stage/commit/rollback skill bundles; D9 receipt rewritten to tagged records with complete owned path sets, legacy tuples refined to empty companion sets (removed the inconsistent one-install-cycle orphan claim), receipt+materialization as one rollback unit; D10 scaffold writes README undeclared during rollout window and edit gains scaffold-or-remove for vanished declarations; Migration Plan reordered consumer-first with cafe as hard gate and irreversibility note; old-builder-ignores-files hazard added to Risks; all three Open Questions closed as decisions." } } } diff --git a/openspec/changes/support-non-asset-files/design.md b/openspec/changes/support-non-asset-files/design.md new file mode 100644 index 00000000..bcecd875 --- /dev/null +++ b/openspec/changes/support-non-asset-files/design.md @@ -0,0 +1,233 @@ +## Context + +The facet pipeline enforces one file per asset end to end. Build collects only `facet.json` plus conventional asset paths (`collectArchiveEntries`, `packages/protocol/src/build/content-hash.ts`); archive verification rejects every other inner-tar entry at Step 6b outer-exclusivity (`packages/protocol/src/integrity/validate-archive.ts`); the adapter contract carries exactly one content string per asset (`installAsset(scope, type, name, content, metadata)`, `packages/adapter/src/types.ts`); and the machine-local receipt records only `(scope, type, name)` asset tuples (`packages/engine/src/install/receipt.ts`). + +The reconciled proposal introduces **supplementary files**: manifest-declared non-asset files that ship in the archive, are integrity-protected like everything else, and — only when they live inside a skill's directory — materialize atomically with that skill. Files elsewhere (root-level `README.md`, `LICENSE`, extras near agents/commands) ship but never touch disk at install time. + +Constraints inherited from the proposal: + +- Every archive entry MUST be derivable from an explicit declaration in the embedded `facet.json` (the outer-exclusivity trust root). No auto-discovery. +- Supplementary files MUST NOT become independently addressable assets (no asset type, adapter metadata, install scope, or lockfile asset tuples). +- The change is **BREAKING** at the archive-format level and MUST have an explicit version boundary; legacy asset-only archives MUST remain valid. + +A single principle organizes this design: **the embedded `facet.json` is the sole source of truth for archive membership and entry classification.** The build manifest records hashes, never a second description of which paths are assets or supplementary files. Every stage — build collection, hashing, verification, parsing, installation — derives membership and classification from the manifest through one shared operation (D3), so the classifications cannot drift apart. + +## Goals / Non-Goals + +**Goals:** + +- Define the manifest declaration shape, archive/build-manifest representation, verification rules, materialization boundary, and adapter contract for supplementary files. +- One shared derivation of the archive-entry set, consumed by build and verification alike — no duplicated allowlist logic. +- Preserve deterministic output within each archive format. Every build produced after the format transition MUST emit canonical `facetVersion: 0.2` output; consumers MUST continue accepting valid legacy `0.1` archives during the compatibility window. +- Make lockfile `0.2` the adapter-agnostic source of truth for every materialized logical file and its canonical per-file integrity, while the machine-local receipt records what this machine owns for rollback and offline removal. +- Make illegal states unrepresentable at the security- and data-loss-critical boundaries (adapter operations, receipt records, parsed archive results) via tagged unions, not optional fields plus prose invariants. + +**Non-Goals:** + +- No `facet info` command or README rendering (future capability; this change only makes the bytes available). +- No companion-directory semantics for agents or commands. +- No filesystem metadata preservation (exec bits, symlinks, hard links, timestamps, ownership) for supplementary files. +- No glob/pattern declaration in v1 (see D2). +- No registry-side (cafe) implementation — sequencing is a hard constraint in the Migration Plan, but the code is out of this repo. + +## Decisions + +### D1: Declaration shape — per-skill `files` plus top-level `files` + +The facet manifest gains two declaration sites, each owning a disjoint region of the tree: + +- **`SkillDescriptor.files?: string[]`** — companion files for one skill, as paths **relative to the skill directory** (e.g. `references/art.md` resolves to `skills/cowsay/references/art.md`). These are the only supplementary files that materialize, and they install/remove atomically with their owning skill. +- **Top-level `files?: string[]`** — repo-relative paths for everything else (`README.md`, `LICENSE`, `agents/notes.md`, `ideas.txt`). Shipped and hashed, never materialized. Top-level entries MUST NOT resolve under `skills/` — skill companions have exactly one declaration site. + +Rationale: ownership (which skill do these files belong to?) is the load-bearing semantic — it drives materialization, atomic lifecycle, and receipt bookkeeping. Giving each tree region exactly one declaration site makes ownership unambiguous at the point of declaration. The lists themselves are unrestricted strings, so disjointness, path safety, declared-skill membership, and collision freedom are enforced as schema-narrowing constraints by the shared archive-plan operation (D3) — the schema shape makes ownership *unambiguous*, and the validator makes it *checked*; neither claim substitutes for the other. + +*Alternative considered:* a single top-level `files` list with ownership derived by `skills//` path prefix. Rejected: one declaration site is simpler, but ownership becomes an inferred invariant instead of a declaration-site fact, and a path under `skills//` needs a bespoke validation rule instead of failing the obvious "companion declared on a skill that exists" check. + +### D2: Explicit per-file enumeration; no globs + +Both `files` lists enumerate exact paths. The embedded `facet.json` is the trust root for outer exclusivity; a glob (`skills/cowsay/**`) in the embedded manifest would let an attacker add undeclared files to a materialized skill directory while still "deriving" from the manifest — precisely the supply-chain hole Step 6b exists to close. Authoring ergonomics are addressed by the edit flow (D11), not by the artifact format. + +*Alternative considered:* glob expansion at build time (source manifest has globs, embedded manifest gets exact paths). Rejected for v1: the archive currently embeds the source `facet.json` verbatim (its per-file hash equals the source file's), and rewriting it at build breaks that property. MAY be revisited as a pure authoring convenience later. + +### D3: One shared archive-plan derivation + +The protocol package SHALL expose a single pure operation that validates both declaration sites (per D7's grammar) and derives a **tagged archive plan**: every planned entry is classified as exactly one of `manifest`, `primary-asset`, `skill-companion` (carrying its owning skill), or `archive-only`. Build collection, per-entry hashing, archive verification, the parsed archive result, and installation MUST all consume this one operation. No stage maintains its own membership or classification logic. + +Rationale: today's outer-exclusivity check and `collectArchiveEntries` already construct membership independently — the exact duplicated-allowlist drift this change would otherwise multiply across four more call sites. Membership is a security boundary; it gets one implementation. + +*Alternative considered:* separate build-side and verify-side derivations (the status quo, extended). Rejected: duplicated membership logic at a trust boundary is how producers and verifiers drift into accepting different sets. + +### D4: Build manifest — unconditional 0.2 output; strict version dispatch; pre-1.0 minor releases + +Every build produced after this change SHALL emit `facetVersion: 0.2`, whether or not the facet declares supplementary files. Asset-only facets therefore use the same current format as facets with supplementary files. Producers SHALL NOT conditionally emit `0.1`; `0.1` remains a legacy input format supported by consumers during a compatibility window and MAY be deprecated by a separate future change. + +The `0.2` build manifest SHALL replace the `assets` map with a single `files` map: canonical inner-tar path → `sha256:`, covering every entry — `facet.json`, primary asset files, and supplementary files. The map carries hashes only. Asset/supplementary classification is NEVER read from the build manifest; it is derived from the embedded `facet.json` via the archive plan (D3). The completeness rule is single: the `files` key set MUST exactly equal the observed inner-tar entry set. + +Verifiers SHALL dispatch on `facetVersion` exactly once, at parse time: the exact legacy schema and rules apply to `0.1`; the rules above apply to `0.2`; any other version returns a structured `UNSUPPORTED_FACET_VERSION` failure carrying the observed version and supported versions. A malformed `0.2` manifest MUST NOT be reinterpreted as `0.1` — no fallback between versions. A `files` key in a `0.1` manifest (or an `assets` key in `0.2`) fails schema validation, making the illegal combinations unrepresentable in validated data. + +The CLI SHALL render unsupported-version failures as upgrade guidance. For a known format transition, one CLI-side compatibility table SHALL map the facet format to the minimum supporting CLI release, producing guidance such as: “This facet uses archive format 0.2, which this CLI does not support. Update agent-facets to or later.” For an unknown future format, the CLI SHALL advise updating to the latest release without inventing a minimum version. Already-released CLIs cannot be retrofitted and MAY continue showing their existing generic validation error; a consumer-first bridge release SHOULD add this handling before any producer emits `0.2`. + +This archive-format boundary is distinct from package release versioning. While `@agent-facets/protocol` and `@agent-facets/adapter` remain pre-1.0, breaking contract changes SHALL increment each package's minor version rather than its major version. This change therefore ships in the next minor release of each package. The permanent protocol release policy SHALL be updated by this change to encode the pre-1.0 rule; after 1.0, breaking changes SHALL require a major release. + +*Alternatives considered:* conditional `0.1`/`0.2` producer output (rejected: creates two current producer modes and prolongs ambiguity about which format a newly built facet uses; the user-facing rule is simpler when all new output is `0.2`); parallel `assets` + `files` maps (rejected: duplicates classification already derivable from the embedded manifest and violates the single-source-of-truth principle); dropping `0.1` verification immediately (rejected: existing published facets remain valid and require a compatibility window); package major releases (rejected: project policy uses minor releases for breaking changes while packages remain pre-1.0). + +### D5: Verification — raw-header validation, exact set equality, fail-closed + +Verification of a `0.2` archive SHALL: + +1. Validate raw tar entries **before** constructing any path-keyed map (a lossy map silently collapses duplicate paths — a smuggling vector). Duplicate paths, non-regular entries (symlinks, hard links, directories, devices), and unsafe or non-canonical paths (per D7) are each structured rejections. +2. Validate the embedded `facet.json` and derive the expected entry set via the archive plan (D3) — never from the build manifest. +3. Compare expected and observed canonical path sets for **exact equality** (undeclared extra entries and declared-but-missing entries are both rejections, as today). +4. Require exactly one `files` hash per expected path and no hash for any other path, then byte-verify every entry against its hash. + +Every expected failure mode SHALL remain a structured result variant — no thrown errors escape the verification contract. Older verifiers fail closed on `0.2` archives (unknown entries → outer-exclusivity rejection), which is the correct security posture for a consumer that cannot enforce the new rules. + +### D6: Supplementary files are opaque bytes; parsed results are tagged + +Supplementary content is read, hashed, archived, and written **verbatim**: no front-matter merge, no line-ending normalization, no empty-content rule, binary permitted. `ArchiveEntry.content` widens to `string | Uint8Array` (hashing and `nanotar` already accept bytes). + +`string | Uint8Array` alone does not encode which content is prompt text and which is opaque, so the successful parsed/verified archive result SHALL carry entries as **tagged data**: primary assets, skill companions grouped by owning skill, and archive-only supplementary bytes are distinct variants (mirroring the D3 plan). Text decoding and front-matter reconciliation apply only after narrowing to a primary asset; supplementary data stays bytes end to end. Classification via optional fields whose combinations can disagree is prohibited. + +### D7: Path validation grammar + +At build (inside the D3 operation) and at archive verification, every declared supplementary path MUST satisfy: + +- non-empty, relative, and already canonical: no empty, `.`, or `..` segments; no backslashes; no NUL bytes; no absolute-path, drive, or URL-like prefixes; +- **regular files only** at build time: the declared path MUST resolve through existing parents to a regular file inside the facet root — symlinks and hard links are rejected (resolved source identity is checked, not just spelling); +- the exact root path `facet.json` is excluded because it is the authoritative embedded manifest; the basename `facet.json` MAY appear at any other path (for example, `skills/example/examples/facet.json`); +- supplementary entries MUST NOT collide with any conventional primary asset path derived from the manifest; names owned only by the outer archive, including `build-manifest.json` and `archive.tar.gz`, MAY appear as supplementary inner-archive paths; +- site rules: top-level entries MUST NOT resolve under `skills/`; per-skill entries MUST NOT be `SKILL.md` and MUST resolve below their skill's directory; +- collision-free across the whole planned entry set, where collision includes: exact spelling, canonical Unicode form (NFC/NFD aliases), portable case folding (case-insensitive filesystems), resolved source identity, and file/directory prefix conflicts (`foo` as a file vs. `foo/bar`). + +Missing declared files fail the build with structured errors, and **all source inputs SHALL be validated before any `dist/` cleanup runs** — a declared input must never be destroyed before its missing-file error can be reported. Each failure class above maps to a distinct structured `ValidationError`, and the test suite SHALL carry a matrix with at least one case per class (traversal, absolute/drive paths, backslashes, NUL, empty/`.` segments, Unicode/case aliases, prefix collisions, symlinks, hard links, duplicates, exact-root-`facet.json` collisions, conventional-primary-path collisions, undeclared entries, missing declarations, tampered bytes). + +### D8: Adapter contract — tagged asset payloads, atomic skill bundles (BREAKING) + +The adapter SDK's install, read, and delete requests/results SHALL become **tagged unions keyed by asset type**: + +- the **skill** variant carries the `SKILL.md` text plus a canonical map of companion paths (relative to the skill root) to bytes — an empty map is legal and is how a companion-less skill is expressed; +- **agent** and **command** variants carry their existing single content string and structurally cannot carry companions; +- no variant exists for supplementary files (they never reach adapters — D12). + +This replaces the earlier optional-parameter shape (`companions?` beside `assetType`), which represented illegal combinations (an agent with companions; a skill call silently omitting its bundle) and policed them only by prose. Tagged variants give implementations an exhaustive branch. + +A skill install SHALL be **one adapter operation with all-or-nothing semantics**: stage the complete replacement bundle, remove previously-owned companion paths absent from the new bundle, and commit — or roll back leaving no partial bundle. Skill deletion likewise removes the primary file plus all recorded owned companions as one operation, never touching unowned files. Expected failures are structured result values. + +The SDK's filesystem helpers SHALL centralize the security-sensitive machinery — companion-path containment within the resolved skill root, staging, commit/rollback, owned-path removal, empty-directory pruning — so adapters built on the helpers (including `claude-code`) inherit correct behavior. Custom-I/O adapters MUST satisfy the same observable contract. Integration tests SHALL inject failures at every write/delete/commit boundary. Front-matter reconciliation applies only to the primary file; companion bytes are written verbatim. + +Engine's skip-if-identical logic extends per-companion: unchanged companions are skipped; changed ones are journaled with previous bytes for rollback. + +*Alternatives considered:* optional `companions?` parameters (rejected: optional-fields-as-discriminator, see above); separate per-companion install/delete methods (rejected: multiplies journal entries and failure surfaces, and permits a partially-companioned skill between calls); engine writing companions directly (rejected: adapters own all storage paths and formats); deleting the whole skill directory (rejected: destroys unowned user files — deletion is ownership-based). + +### D9: Asset names follow Agent Skills, remain single-segment, and share defined namespaces + +The canonical `0.2` asset-name grammar SHALL follow the Agent Skills `name` field convention: https://agentskills.io/specification#name-field. Facets normatively interprets the specification's enumerated character ranges as ASCII: an asset name MUST contain 1–64 lowercase ASCII letters (`a-z`), digits (`0-9`), or hyphens; MUST NOT start or end with a hyphen; and MUST NOT contain consecutive hyphens. `/` is invalid in every asset name. + +Facets SHALL apply this same grammar to skills, commands, and agents. Applying the Agent Skills grammar to commands and agents is a Facets extension that gives all asset types one naming convention and one protocol validator. Protocol schemas, validator comments, generated schema documentation, and user-facing naming documentation MUST link to the Agent Skills `name` field as the external convention being implemented while stating Facets' normative ASCII interpretation. + +A skill named `review` is represented by the top-level directory `skills/review/`, whose required primary file is `skills/review/SKILL.md`. The manifest skill name, installed directory name, and materialized `SKILL.md` name metadata MUST agree. Declared companion paths beneath that root MAY contain directories of arbitrary safe depth, such as `scripts/run.ts`, `references/api.md`, or `assets/logo.png`; those path separators are not part of the skill name. A command named `review` is represented by `commands/review.md`, and an agent named `review` by `agents/review.md`. + +Skills and commands SHALL occupy one logical namespace: the skill-name and command-name sets MUST be disjoint. A facet declaring both skill `review` and command `review` fails with a structured collision error identifying `skills.review` and `commands.review`. Agents remain in a separate namespace and MAY share a name with a skill or command. + +The single-segment grammar and shared namespace SHALL be validated before archive planning, adapter selection, or filesystem writes. Local `0.2` builds and `0.2` archive verification MUST consume the same protocol validation. Legacy `0.1` verification SHALL retain the previous multi-segment and cross-type-collision rules so existing archives remain consumable; there is no fallback from an invalid `0.2` manifest to the `0.1` grammar. + +`parseAssetNameSegment` becomes the canonical current-format asset-name parser. Multi-segment parsing remains isolated to the legacy `0.1` verifier and MUST NOT appear in current manifest types or authoring APIs. Internally composed or slash-namespaced assets are not part of the `0.2` model; any future composition design must preserve single-segment asset identities rather than encoding hierarchy into names. + +*Alternatives considered:* retaining slash-separated internal names (rejected: conflates asset identity with filesystem hierarchy and contradicts the one-directory/one-file source model); using a Facets-only naming grammar without citing Agent Skills (rejected: loses the shared ecosystem convention even though the effective constraints align); interpreting “Unicode lowercase alphanumeric” beyond the specification's explicit `a-z` and `0-9` ranges (rejected: Unicode category and normalization behavior would make portable validation weaker and less deterministic); validating skill/command collisions only after adapter selection (rejected: facet validity would vary by adapter); putting agents in the shared namespace (rejected: agents do not occupy the skill/command invocation namespace). + +### D10: Lockfile 0.2 pins every materialized file; receipt mirrors machine ownership + +`facets.lock` SHALL use `lockfileVersion: 0.2` for the current alpha schema. Version dispatch MUST use exact equality, never numeric ordering: legacy numeric `1` identifies the previous alpha schema, while numeric `0.2` identifies this schema. `FACET_ARCHIVE_VERSION` and `LOCKFILE_VERSION` SHALL remain separate constants that both currently equal `0.2`; their equality is release alignment, not a permanent invariant, because archive and resolution formats may evolve independently. + +Every lockfile asset entry SHALL contain its adapter-agnostic identity (`scope`, `type`, `name`) plus a required, deterministically sorted `files` array. Each file record SHALL be `{ path, integrity }`, where `path` is the canonical inner-archive path and `integrity` is the `sha256:` hash of that archive entry's exact canonical bytes. + +- A skill entry's `files` SHALL contain `skills//SKILL.md` plus every declared companion beneath `skills//`. +- An agent entry's `files` SHALL contain exactly `agents/.md`. +- A command entry's `files` SHALL contain exactly `commands/.md`. +- Archive-only supplementary entries, including root `README.md`, SHALL NOT appear in `assets[].files` because they are not materialized; the facet-level integrity continues to pin them. +- Companion files remain subordinate file-integrity records inside their owning skill entry. They SHALL NOT become independent assets, acquire scopes, or receive standalone asset tuples. + +The lock writer SHALL derive `assets[].files` from the verified D3 archive plan's materialized subset. For every included path, it SHALL persist the recomputed hash that has already been reconciled with the 0.2 build manifest's `files` map; it MUST NOT trust or blindly copy a self-declared build-manifest value. + +Before any materialization, install SHALL require exact agreement among: + +1. the lockfile facet-level integrity and the recomputed archive integrity; +2. the lockfile asset identities and the verified materialization plan; +3. every lockfile asset's complete file path set and the files owned by that planned asset; +4. every lockfile per-file integrity, the recomputed archive-entry hash, and the corresponding verified build-manifest hash. + +Any disagreement SHALL return structured failure data containing the facet, asset, canonical path, expected integrity, and actual integrity when available. Frozen mode SHALL fail without rewriting. Normal resolution MAY write a new lock entry only after all checks against the newly resolved artifact succeed. + +Drift checking SHALL operate per locked file. Verbatim companion files are hashed directly from disk. For primary files whose adapter representation differs from archive bytes, the adapter `readAsset` contract SHALL return canonical logical content so the engine can compare the corresponding locked canonical integrity without encoding adapter-specific bytes in `facets.lock`. Reports SHALL identify the exact locked path that drifted. + +The machine-local receipt SHALL mirror the successfully committed lockfile asset/file ownership set so offline removal and rollback remain exact even after a pulled lockfile drops an entry. The receipt remains adapter-agnostic and stores no adapter-encoded hashes. Receipt and lockfile changes SHALL commit in the same install transaction as materialization; rollback restores all three. Receipts remain untrusted input: identity, path containment, and file-integrity record validation MUST precede deletion, and unowned paths MUST never be deleted. The receipt schema version SHALL become `0.2`; legacy receipt version `1` MAY be refined to primary-only file sets because the legacy system could not install companions. + +A current loader SHALL recognize legacy numeric lockfile version `1` only as the previous alpha schema. Normal install MAY migrate a verified legacy lockfile to `0.2`; frozen legacy installs retain legacy behavior and do not rewrite. A `0.2` archive requires a `0.2` lockfile. When the stable lockfile v1 schema is eventually released, support for legacy-alpha numeric `1` SHALL be removed rather than reinterpreted or shape-sniffed: an old alpha lockfile SHALL fail with an actionable instruction to delete and regenerate it. The future stable v1 schema then owns numeric `1` exclusively. + +*Alternatives considered:* facet-level integrity alone (rejected: cryptographically protects the archive but cannot directly attribute drift to one materialized file); companion paths without per-file hashes (rejected: records ownership but not file-level integrity); adapter-encoded hashes in the lockfile (rejected: makes a portable facet resolution vary by adapter and machine); independent companion asset tuples (rejected: companions have no independent identity or scope); permanently coupling lockfile and archive version constants (rejected: they describe different artifacts and will eventually diverge); preserving legacy numeric `1` after stable v1 launches (rejected: one version identifier cannot safely select two schemas). + +### D11: `README.md` is first-class in create and edit; extensionless `README` is supported + +`README.md` SHALL be the preferred conventional facet document. `facet create` SHALL generate `README.md` by default. The exact extensionless path `README` SHALL also be recognized as a first-class README by `facet edit` and the build/manifest workflow. Both remain normal top-level supplementary-file declarations in `facet.json.files`; the manifest SHALL NOT gain a README-specific field or duplicate source of truth. + +The interactive `facet create` wizard SHALL include a dedicated README step or card, separate from asset management. README SHALL be enabled by default but optional. The wizard SHALL seed editable `README.md` content from the facet name and description, allow the author to open and edit that content before confirmation, and allow the author to disable README creation. The confirmation preview SHALL list `README.md` explicitly. On apply, the wizard SHALL atomically write `README.md` and add its exact path to top-level `files`. The generated template is an initial value only; later identity edits MUST NOT silently regenerate or overwrite authored README content. + +The `facet edit` wizard SHALL show the exact root paths `README.md` and `README` in a dedicated facet-level README panel rather than generic supplementary-file reconciliation. For each recognized path, behavior depends on its current state: + +- present and declared: offer Edit or Remove; +- present but undeclared: offer Adopt or Edit-and-Adopt; +- declared but missing: offer Scaffold at that same path or Remove Declaration; +- absent and undeclared: offer Create, defaulting to `README.md`. + +If both `README.md` and `README` exist, the dedicated panel SHALL show both independently; neither file is silently ignored or overwritten. Adopt SHALL preserve existing bytes unless the author explicitly edits them. Remove SHALL queue both file deletion and declaration removal. Scaffold/Create SHALL queue the file write and declaration addition. All README operations remain transactional: no file or manifest change occurs until the existing Apply confirmation, and the confirmation summary SHALL identify the exact README path and operation. + +`facet edit`'s generic scanner SHALL still detect undeclared files inside declared skill directories and offer to add them to that skill's `files`. It SHOULD detect other common root-level supplementary files such as `LICENSE`, but `README.md` and `README` SHALL be routed only through the dedicated README panel so they do not appear twice. + +For any declared supplementary file other than `README.md` or `README` that has vanished from disk, edit SHALL offer scaffold-or-remove, mirroring the existing missing-asset flow. + +*Alternatives considered:* always requiring README (rejected: first-class does not mean mandatory); generating extensionless `README` by default (rejected: `README.md` is the preferred authored format); adding a README-specific manifest field (rejected: duplicates top-level `files` membership); silently regenerating README after identity edits (rejected: destroys authored documentation); treating README only as a generic discovered file (rejected: misses the intended first-class authoring experience). + +### D12: Materialization boundary is engine logic, not adapter logic + +Engine passes companions only inside skill-variant payloads (D8); archive-only supplementary files never reach `materialize`. Adapters never see non-skill supplementary files, so the "ships but does not materialize" rule cannot be violated by an adapter bug — the data simply isn't handed over. + +## Risks / Trade-offs + +- **[Old builders silently ignore `files`]** — manifest validation tolerates unrecognized fields, so an old CLI builds a facet that *declares* supplementary files but *omits* the bytes, with no error. → Documentation MUST state the minimum producer version for `0.2`; examples SHOULD pin it; compatibility fixtures MUST prove which producer versions emit declared files. This hazard cannot be repaired retroactively in already-shipped tolerant parsers — docs and fixtures are the only lever. +- **[Path aliases or crafted tar headers bypass membership checks]** → One shared plan derivation (D3) + the D7 grammar + raw-header validation before lossy maps (D5); the per-failure-class test matrix is mandatory. +- **[A failed skill update leaves a half-written directory]** → Atomic stage/commit/rollback in the adapter contract (D8) with injected-failure tests at every write/delete/commit boundary. +- **[Receipt corruption causes over-deletion]** → Receipts are untrusted (D10): containment + project-identity checks precede deletion; unowned paths are never deleted. +- **[Lockfile growth from per-file integrity]** → Only materialized files are copied into `assets[].files`; archive-only metadata remains covered by facet integrity. File records are canonical and sorted for stable, reviewable diffs. +- **[Legacy alpha numeric `1` conflicts with future stable v1]** → Current releases explicitly classify it as legacy-alpha-1 and migrate to `0.2`; stable v1 removes that parser and emits an actionable delete-and-regenerate error for old-shape numeric-1 files. +- **[Adapter transformations obscure raw drift comparison]** → Lockfile hashes remain canonical and adapter-agnostic; `readAsset` must project installed primary content back into its canonical logical form. Adapter-specific bytes never enter version-controlled resolution state. +- **[Generated README content overwrites author edits]** → The template is applied only on explicit Create or Scaffold. Identity changes and ordinary edit sessions MUST preserve existing README bytes unless the author chooses to edit them. +- **[All newly built facets require a 0.2-capable consumer]** → Verification support SHALL deploy consumer-first, including cafe, before producer enablement. A bridge CLI SHOULD recognize `0.2` and render the D4 upgrade message; older pre-bridge CLIs may still show a generic validation failure. Immutable fixtures MUST prove that new consumers continue accepting valid `0.1` archives. +- **[Two supported archive versions create implementation branches]** → Version dispatch happens exactly once at parse time, with no cross-version fallback and immutable fixtures for both schemas (D4/D5). Removing `0.1` support requires a separate future deprecation change. +- **[Arbitrary companion bytes inflate archives / decompression pressure]** → Size/count limits are consumer and registry policy, not protocol format (see Open Questions closure); existing decompression handling applies to the whole archive. +- **[Unsupported format surprises installers]** → Build output SHALL display the emitted `facetVersion` and complete entry listing; install SHALL return a structured unsupported-version failure, and the CLI SHALL render actionable upgrade guidance from its single compatibility table. +- **[Third-party adapter breakage]** → SDK-helper adapters inherit companion support via the helpers; only custom-I/O adapters must implement the widened contract. The tagged unions make the migration mechanical and exhaustively checkable. +- **[Two declaration sites could confuse authors]** → The disjointness rule (D1/D7) yields a precise error pointing at the correct site; edit-flow detection (D11) writes declarations to the right place automatically. +- **[Existing source manifests use slash-namespaced assets or duplicate skill/command names]** → Their published `0.1` archives remain consumable, but rebuilding as `0.2` fails with actionable validation errors before output is changed. Authors MUST rename the assets; invalid `0.2` manifests are never interpreted using legacy rules. + +## Migration Plan + +Consumer-first: verification support ships everywhere before any producer can emit the new format. + +1. **protocol (next minor release)**: adopt the pre-1.0 breaking-release policy; add the D3 plan operation, D7 path grammar, D9 current-format asset-name grammar and shared namespace validation, D4 versioned build-manifest schemas with strict dispatch and structured unsupported-version results, D5 verification, D6 tagged parsed results, and D10 lockfile `0.2` with exact legacy-1/current-0.2 dispatch, per-file integrity schemas, and mismatch result types — plus immutable fixtures for both archive versions. +2. **engine/CLI bridge (consumer side)**: loaders and cache consume the tagged parsed result; verified legacy lockfiles and receipts migrate during normal installs while frozen legacy state retains legacy behavior; unsupported versions and per-file integrity mismatches render actionable diagnostics. Producers still emit `0.1` in this bridge release. +3. **cafe registry (out of repo, hard gate)**: deploys `0.2` verification while retaining `0.1`. Producer enablement MUST NOT ship before this lands. +4. **adapter SDK (next minor release) + first-party adapters**: tagged payload unions, atomic bundle helpers (D8), injected-failure tests; claude-code migrates. +5. **engine/CLI producer release**: every build emits archive `0.2` and writes only lockfile `0.2`; materialize passes skill bundles; receipts write the `0.2` ownership shape; create and edit ship the D11 first-class `README.md`/`README` authoring flow; generic supplementary-file detection/add/remove flows land. +6. **docs** (Article III): update `docs/specification/archive.mdx` (membership rules, single `files` hash map, version dispatch), `build.mdx` (plan derivation, validation-before-cleanup, displayed version), `manifest.mdx` (both `files` fields, minimum-producer-version warning, linked Agent Skills naming convention, single-segment asset names versus nested companion paths, shared skill/command namespace), `integrity.mdx` (all-entry and per-locked-file coverage), `lockfile.mdx` (lockfile `0.2`, per-materialized-file integrity, legacy-alpha-1 migration, stable-v1 regeneration boundary), `commit.mdx` (receipt ownership and transactional reconciliation), `install.mdx` (materialization boundary, atomic skill bundles, mismatch diagnostics), `docs/guides/create-your-first-facet.mdx` and `docs/guides/install-facets.mdx` (asset-only phrasing, README workflow), root `README.md`. + +Before the future stable lockfile v1 release, legacy-alpha-1 parsing SHALL be removed and replaced with actionable delete-and-regenerate guidance for old-shape numeric-1 files. + +Rollback: before any `0.2` artifact is published, producer rollout MAY revert to the bridge release. After publication or multi-file skill materialization, producer emission MAY be paused, but consumers MUST retain `0.1` and `0.2` verification plus receipt-aware deletion. Removing `0.1` support requires a separately reviewed deprecation change. + +## Open Questions + +None remaining — the three raised during drafting are closed as decisions: + +- Registry README presentation: out of scope (proposal non-goal); the all-entry `files` map makes it cheap for a future change. +- Archive size/entry-count limits: consumer and registry configuration policy, not protocol format, for this change. +- Edit behavior for vanished declared files: decided in D11 (scaffold-or-remove, mirroring the missing-asset flow). diff --git a/openspec/changes/support-non-asset-files/proposal.md b/openspec/changes/support-non-asset-files/proposal.md index 92753e87..b0ba4882 100644 --- a/openspec/changes/support-non-asset-files/proposal.md +++ b/openspec/changes/support-non-asset-files/proposal.md @@ -5,11 +5,14 @@ The facet pipeline enforces a one-file-per-asset invariant end to end: build col ## What Changes - **Facets can track non-asset files.** The facet manifest gains a way to declare files that are not skills, agents, or commands (e.g. `README.md`, `LICENSE`, `DEVELOPMENT.md`, `ideas.txt`). Every supplementary archive entry — including every skill companion file — MUST be derivable from an explicit manifest declaration; archive membership stays explicit and reviewable, with no recursive auto-discovery. Declaration syntax (per-file vs. pattern) is a design decision. Missing declared files, undeclared entries, unsafe paths (traversal, absolute, backslashes), and colliding resolved paths SHALL fail build validation. +- **Asset identity is simplified in 0.2.** Skill, command, and agent names MUST follow the Agent Skills `name` field convention as a single ASCII segment; slash-namespaced assets are no longer valid in current-format manifests. Skills and commands SHALL share one logical namespace and MUST NOT use the same name; agents remain separate. Legacy 0.1 archives retain their legacy validation during the compatibility window. - **Integrity covers every file, not just assets.** Non-asset files MUST be hashed per-entry in the build manifest and included in the tar bytes that produce the content-integrity hash. Verification MUST recompute hashes for all entries, asset or not. +- **Lockfile integrity becomes file-addressable.** Lockfile `0.2` SHALL record the verified canonical path and integrity of every materialized primary or companion file inside its owning asset entry. Install MUST reconcile those records against recomputed archive hashes and fail with the exact mismatching path. Archive-only supplementary files remain protected by facet-level integrity without becoming lockfile assets. - **Outer exclusivity is relaxed, not abandoned.** Every inner-tar entry MUST still be derivable from the embedded `facet.json` — the derivable set expands to include declared non-asset files. Undeclared extra files remain a rejection; the supply-chain rationale for the rule is preserved. - **Skill directories become multi-file.** Declared files under `skills//` (beyond `SKILL.md`) SHALL be shipped and SHALL be installed and removed atomically with their owning skill through the adapter contract, with receipt/ownership data sufficient for drift removal. **BREAKING** for the adapter SDK: the adapter asset contract is one content string per asset today, so installing and deleting multi-file skills changes the adapter interface for third-party adapters. - **Everything else ships but does not materialize.** Non-asset files outside skill directories — root-level files like `README.md`, or extra files under `agents/` and `commands/` — SHALL NOT be written to disk at install time. They travel with the archive as facet metadata, available to future surfaces (e.g. a `facet info` command, registry listings). README is the motivating example of a file worth surfacing; this change only makes it shippable and verifiable, not displayed. Supplementary files SHALL NOT become independently addressable assets: no asset type, no adapter metadata, no independent install scope, no lockfile asset tuples. -- **BREAKING (protocol/archive format).** Expanding the archive entry set is backward-incompatible: archives containing non-asset files fail outer-exclusivity verification on older consumers. Design SHALL choose an explicit protocol/archive-version compatibility boundary rather than relying on a forward-compatibility note, and compatibility tests SHALL prove legacy asset-only archives remain valid under the new rules, per the protocol's semantic-versioning discipline (`openspec/specs/protocol/spec.md`). +- **README receives first-class authoring support.** Create SHALL offer editable `README.md` content enabled by default. Edit SHALL expose dedicated create/edit/adopt/scaffold/remove actions for both exact conventional paths, `README.md` and extensionless `README`. README remains optional and uses the same top-level supplementary-file declaration mechanism as every other archive-only file. +- **BREAKING (protocol/archive format).** All builds produced after this change SHALL use `facetVersion: 0.2`; consumers SHALL continue accepting legacy `0.1` archives during a compatibility window. Unsupported versions MUST produce structured failures that the CLI renders as actionable upgrade guidance. Protocol and adapter packages SHALL use their next minor releases for these breaking changes while pre-1.0; removing `0.1` support is a separate future change. ## Non-goals @@ -27,11 +30,12 @@ None — this change modifies existing domains rather than introducing a new one ### Modified Capabilities -- `protocol__schemas`: the facet manifest schema gains a declaration for non-asset files; the build manifest represents and hashes every tracked entry while preserving an unambiguous distinction between installable assets and supplementary files (exact shape settled in design). +- `protocol`: pre-1.0 breaking protocol changes increment the package minor version; 1.0-and-later breaking changes require a major version. +- `protocol__schemas`: the facet manifest schema gains a declaration for non-asset files; current-format asset names follow the single-segment Agent Skills convention and skill/command names are disjoint; the build manifest represents and hashes every tracked entry; lockfile `0.2` requires canonical path/integrity records for every materialized file inside its owning asset entry. - `protocol__content-hashing`: archive assembly collects declared non-asset files and skill-directory files at their source paths; per-entry hashes are recorded for all entries. - `protocol__integrity`: the outer-exclusivity derivation set expands to declared non-asset files and skill-directory files; verification requirements apply to every inner-tar entry. -- `authoring__facets`: build resolves, validates, and archives non-asset files; missing declared files, undeclared entries, unsafe paths, and colliding resolved paths are build errors. -- `installation`: materialization requirements change — skill-directory files install with their skill; non-asset files elsewhere are shipped but never written to disk; the machine-local receipt supports drift-removal of multi-file skills. +- `authoring__facets`: build resolves, validates, and archives supplementary files; create/edit provide dedicated transactional `README.md` and extensionless `README` authoring plus generic supplementary-file reconciliation; missing declarations, unsafe paths, collisions, slash-containing asset names, and skill/command name collisions are structured failures. +- `installation`: materialization requirements change — skill-directory files install with their skill; non-asset files elsewhere are shipped but never written to disk; install reconciles lockfile `0.2` per-file hashes before writes and reports exact drift paths; the machine-local receipt mirrors committed ownership for offline removal and rollback. - `adapter__assets`: the adapter install/read/delete contract extends from one file per asset to multi-file skills. ## Impact @@ -47,7 +51,7 @@ None — this change modifies existing domains rather than introducing a new one **Documentation (Article III)** -This proposal was informed by `docs/specification/archive.mdx` (content rules: path safety, manifest completeness, outer exclusivity), `docs/specification/build.mdx` (steps 2 and 5), `docs/specification/manifest.mdx` (text-asset conventional paths), and `docs/specification/integrity.mdx` (hash definitions, receipt asset tuples). All four SHALL be updated as scoped work in this change, together with the authoring and installation guides (`docs/guides/create-your-first-facet.mdx`, `docs/guides/install-facets.mdx`) and root `README.md`, which describe facets in asset-only terms today. `docs/specification/lockfile.mdx` MUST be reviewed, and updated if receipt/lockfile semantics change for multi-file skills (a design outcome). +This proposal was informed by `docs/specification/archive.mdx` (content rules: path safety, manifest completeness, outer exclusivity), `docs/specification/build.mdx` (steps 2 and 5), `docs/specification/manifest.mdx` (text-asset conventional paths), and `docs/specification/integrity.mdx` (hash definitions, receipt asset tuples). All four SHALL be updated as scoped work in this change, together with the authoring and installation guides (`docs/guides/create-your-first-facet.mdx`, `docs/guides/install-facets.mdx`) and root `README.md`, which describe facets in asset-only terms today. `docs/specification/lockfile.mdx` SHALL be updated for lockfile `0.2`, per-materialized-file integrity, legacy-alpha-1 migration, and the distinction between version-controlled canonical hashes and machine-local receipt ownership. **Systems**