fix: hold the spec to its own schema, and its own conformance claims#64
Merged
Conversation
Three defects, all the same shape: a normative claim nothing checked. 1. SPEC.md §9's `verify` example was invalid against the schema SPEC.md ships. Both envelopes carried `"id": "v1"`, but §3.2 grants an `id` only to `query`/`frames`/`error`, `Envelope::Verify`/`Verified` have no such field, and the schema is `additionalProperties: false`. Removed — verify correlates by full frame identity, not by envelope id. Root cause: validate-examples.py checked `examples/` but never SPEC.md, so the one example surface with no machine check was the one that drifted. It now validates every fenced jsonc block in SPEC.md (comments and documented placeholders normalized; structure checked), so CI's existing `schema` job catches this class. Verified by re-introducing the `id`: the harness fails both envelopes and exits non-zero. 2. An ordinary ContextQuery was un-representable in conformant JSON. `kinds` and `anchors` were `required` while both are skip_serializing_if = "Vec::is_empty", so an unfiltered, unanchored query failed validation — the bug PR #44 fixed for ContextFrame, one type over. A test now asserts an ordinary query satisfies the schema's own `required` array (reads the schema, so it cannot drift), and a cross-audit of all 16 shared types confirms no other type demands a field its serializer elides. 3. §G2 claimed "Verified by frame-validity" while `target_uri` appeared nowhere in the conformance or validation code — the self-attestation §11.1 exists to reject. Implemented rather than downgraded: check_frames now rejects an edge with an empty target_uri. Auditing the other ten rules citing frame-validity found §D1 unenforced too: the frame's own content_digest was never format-checked, and the evidence string claimed "well-formed digests" while accepting `sha256:abc` — the very placeholder validate.rs exists to reject. Also fixed; the remaining nine were confirmed enforced. Both new checks are mutation-tested: disabling either makes its test fail. Also corrects contextgraph-host::wire, which said correlation is "negotiated by observation, not by a capability flag" — contradicting §3.2 and the shipped Capabilities::correlation, and inverting a MUST NOT. cargo test --workspace: 243 passed, 0 failed. fmt, clippy -D warnings, validate-examples.py, and conformance-{green,red}/host all pass.
There was a problem hiding this comment.
Sorry @macanderson, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
macanderson
marked this pull request as ready for review
July 26, 2026 18:53
Signed-off-by: Mac Anderson <mac@oxagen.sh>
There was a problem hiding this comment.
Sorry @macanderson, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three reported defects, all the same shape — a normative claim nothing checked — plus a fourth found by generalizing the third.
1.
SPEC.md§9'sverifyexample failed the schemaSPEC.mdshipsBoth envelopes carried
"id": "v1". But §3.2 grants anidonly toquery/frames/error, the referenceEnvelope::Verify/Verifiedhave no such field, and the schema isadditionalProperties: false. The spec's own example was invalid against the spec's own definition.Removed.
verifycorrelates by full frame identity, not by envelope id — which §9's prose already explains.Root cause, and the actual fix:
schema/validate-examples.pyvalidatedexamples/but neverSPEC.md. The one example surface with no machine check was the one that drifted. It now validates every fencedjsoncblock inSPEC.md—//comments stripped (string-aware), documented placeholders (sha256:<64 hex>,"…") normalized, then structure checked against the root schema or$defs/ContextFrameas appropriate. Substituting a placeholder never invents a member, so it cannot mask anadditionalPropertiesviolation.CI's existing
schemajob now covers this. Confirmed by re-introducing theid:2. An ordinary
ContextQuerywas un-representable in conformant JSONkindsandanchorswere listed asrequiredwhile both areskip_serializing_if = "Vec::is_empty". An unfiltered, unanchored query — what a host sends when it wants anything relevant — serializes without them and failed validation. Precisely the bug PR #44 fixed forContextFrame, one type over.Fixed the schema to require only what the reference serializer always emits. A new test asserts an ordinary query satisfies the schema's own
requiredarray (it reads the schema, so it can't drift into a stale snapshot).I also re-ran the audit that missed this, mechanically cross-referencing every
skip_serializing_iffield against every schemarequiredlist:ContextQuerywas the only remaining mismatch across all 16 shared types. The audit flags it pre-fix and is clean post-fix.3. §G2 claimed an enforcement that did not exist
target_uriappeared nowhere in the conformance or validation code. Implemented rather than downgraded — the schema has carriedminLength: 1from the start and §G1 immediately beside it is enforced, so making the claim true beats weakening it.4. §D1 was unenforced too (found by generalizing #3)
Auditing the other ten rules citing
frame-validityturned up a second instance: the frame's owncontent_digestwas never format-checked.check_framesacceptedsha256:abc— the very placeholdervalidate::tests::rejects_the_placeholder_digest_the_repo_used_in_examplesexists to reject — while its evidence string claimed "well-formed digests." The digest anchoring deterministic composition, usage reports, andcontext/verifywas held to a looser standard than the §F5 provenance digests directly below it.Fixed. The remaining nine
frame-validityrules (F1–F5, P1–P3, G1) were confirmed genuinely enforced.Both new checks are mutation-tested — disabling either makes its test fail, so neither is vacuous:
Also
contextgraph-host::wiredocumented correlation as "negotiated by observation, not by a capability flag" — contradicting §3.2 and the shippedCapabilities::correlation, and inverting a MUST NOT. An implementer following it would send anidto a provider that never declared correlation. Corrected.Verification
cargo test --workspacecargo fmt --all --checkcargo clippy --workspace --all-targets -- -D warningspython3 schema/validate-examples.pyconformance-green.sh/conformance-red.sh/host-conformance.shThe
redsuite matters here: it asserts every--misbehavemode is still caught, so the two new checks don't mask existing detections.Review notes
target_urior a malformedcontent_digestwill now correctly fail conformance. That is the intent, but it is the one behavioral change worth a second opinion before the freeze.