Skip to content

fix: hold the spec to its own schema, and its own conformance claims#64

Merged
macanderson merged 2 commits into
mainfrom
worktree-spec-schema-conformance
Jul 26, 2026
Merged

fix: hold the spec to its own schema, and its own conformance claims#64
macanderson merged 2 commits into
mainfrom
worktree-spec-schema-conformance

Conversation

@macanderson

Copy link
Copy Markdown
Owner

Three reported defects, all the same shape — a normative claim nothing checked — plus a fourth found by generalizing the third.

1. SPEC.md §9's verify example failed the schema SPEC.md ships

Both envelopes carried "id": "v1". But §3.2 grants an id only to query/frames/error, the reference Envelope::Verify/Verified have no such field, and the schema is additionalProperties: false. The spec's own example was invalid against the spec's own definition.

Removed. verify correlates by full frame identity, not by envelope id — which §9's prose already explains.

Root cause, and the actual fix: schema/validate-examples.py validated examples/ but never SPEC.md. The one example surface with no machine check was the one that drifted. It now validates every fenced jsonc block in SPEC.md// comments stripped (string-aware), documented placeholders (sha256:<64 hex>, "…") normalized, then structure checked against the root schema or $defs/ContextFrame as appropriate. Substituting a placeholder never invents a member, so it cannot mask an additionalProperties violation.

CI's existing schema job now covers this. Confirmed by re-introducing the id:

FAIL  SPEC.md:442 [1/2] (envelope verify)
FAIL  SPEC.md:442 [2/2] (envelope verified)
2 failure(s)

2. An ordinary ContextQuery was un-representable in conformant JSON

kinds and anchors were listed as required while both are skip_serializing_if = "Vec::is_empty". An unfiltered, unanchored query — what a host sends when it wants anything relevant — serializes without them and failed validation. Precisely the bug PR #44 fixed for ContextFrame, one type over.

Fixed the schema to require only what the reference serializer always emits. A new test asserts an ordinary query satisfies the schema's own required array (it reads the schema, so it can't drift into a stale snapshot).

I also re-ran the audit that missed this, mechanically cross-referencing every skip_serializing_if field against every schema required list: ContextQuery was the only remaining mismatch across all 16 shared types. The audit flags it pre-fix and is clean post-fix.

3. §G2 claimed an enforcement that did not exist

target_uri appeared nowhere in the conformance or validation code. Implemented rather than downgraded — the schema has carried minLength: 1 from the start and §G1 immediately beside it is enforced, so making the claim true beats weakening it.

4. §D1 was unenforced too (found by generalizing #3)

Auditing the other ten rules citing frame-validity turned up a second instance: the frame's own content_digest was never format-checked. check_frames accepted sha256:abc — the very placeholder validate::tests::rejects_the_placeholder_digest_the_repo_used_in_examples exists to reject — while its evidence string claimed "well-formed digests." The digest anchoring deterministic composition, usage reports, and context/verify was held to a looser standard than the §F5 provenance digests directly below it.

Fixed. The remaining nine frame-validity rules (F1–F5, P1–P3, G1) were confirmed genuinely enforced.

Both new checks are mutation-tested — disabling either makes its test fail, so neither is vacuous:

a relation with target_uri "" passed frame-validity — §G2 is unenforced again

Also

contextgraph-host::wire documented correlation as "negotiated by observation, not by a capability flag" — contradicting §3.2 and the shipped Capabilities::correlation, and inverting a MUST NOT. An implementer following it would send an id to a provider that never declared correlation. Corrected.

Verification

Gate Result
cargo test --workspace 243 passed, 0 failed
cargo fmt --all --check clean
cargo clippy --workspace --all-targets -- -D warnings clean
python3 schema/validate-examples.py OK — all examples validate
conformance-green.sh / conformance-red.sh / host-conformance.sh pass

The red suite matters here: it asserts every --misbehave mode is still caught, so the two new checks don't mask existing detections.

Review notes

Three defects, all the same shape: a normative claim nothing checked.

1. SPEC.md §9's `verify` example was invalid against the schema SPEC.md
   ships. Both envelopes carried `"id": "v1"`, but §3.2 grants an `id` only
   to `query`/`frames`/`error`, `Envelope::Verify`/`Verified` have no such
   field, and the schema is `additionalProperties: false`. Removed — verify
   correlates by full frame identity, not by envelope id.

   Root cause: validate-examples.py checked `examples/` but never SPEC.md,
   so the one example surface with no machine check was the one that
   drifted. It now validates every fenced jsonc block in SPEC.md (comments
   and documented placeholders normalized; structure checked), so CI's
   existing `schema` job catches this class. Verified by re-introducing the
   `id`: the harness fails both envelopes and exits non-zero.

2. An ordinary ContextQuery was un-representable in conformant JSON.
   `kinds` and `anchors` were `required` while both are
   skip_serializing_if = "Vec::is_empty", so an unfiltered, unanchored
   query failed validation — the bug PR #44 fixed for ContextFrame, one
   type over. A test now asserts an ordinary query satisfies the schema's
   own `required` array (reads the schema, so it cannot drift), and a
   cross-audit of all 16 shared types confirms no other type demands a
   field its serializer elides.

3. §G2 claimed "Verified by frame-validity" while `target_uri` appeared
   nowhere in the conformance or validation code — the self-attestation
   §11.1 exists to reject. Implemented rather than downgraded: check_frames
   now rejects an edge with an empty target_uri.

   Auditing the other ten rules citing frame-validity found §D1 unenforced
   too: the frame's own content_digest was never format-checked, and the
   evidence string claimed "well-formed digests" while accepting
   `sha256:abc` — the very placeholder validate.rs exists to reject. Also
   fixed; the remaining nine were confirmed enforced.

Both new checks are mutation-tested: disabling either makes its test fail.

Also corrects contextgraph-host::wire, which said correlation is
"negotiated by observation, not by a capability flag" — contradicting
§3.2 and the shipped Capabilities::correlation, and inverting a MUST NOT.

cargo test --workspace: 243 passed, 0 failed. fmt, clippy -D warnings,
validate-examples.py, and conformance-{green,red}/host all pass.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @macanderson, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@macanderson
macanderson marked this pull request as ready for review July 26, 2026 18:53
Signed-off-by: Mac Anderson <mac@oxagen.sh>
@macanderson
macanderson merged commit 8533416 into main Jul 26, 2026
10 checks passed
@macanderson
macanderson deleted the worktree-spec-schema-conformance branch July 26, 2026 18:56

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @macanderson, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant