Pre-freeze round 2: close the enforcement gaps, and the ones found while closing them#63
Merged
Merged
Conversation
…e it (#54) `ContextQuery.kinds` and `.anchors` were globally `required` in the schema but `skip_serializing_if = "Vec::is_empty"` in the reference type. An unfiltered, unanchored query — the most common query there is, and precisely the one this suite's own `sample_query()` sends — therefore serialized to JSON the schema rejected. Same class of bug ADR 0006 hit on `ContextFrame`; fixing that one frame left the class alive. It survived because schema validation only ever ran over hand-authored transcripts. Curated examples prove the schema accepts what a human wrote; they can never catch the schema disagreeing with the serializer. The structural fix: - `schema/reference-vectors.ndjson` — every envelope variant serialized from the reference Rust types, in both minimal form (empty vecs, None options, where omission bugs live) and maximal form. - `tests/reference_vectors.rs` generates them, fails on drift naming the drifted vector (regenerate: REGENERATE_VECTORS=1), asserts each round-trips through the wire decoder, and asserts every frame is *conformant* — B3 costs, F5 digests, representation invariants — so the file doubles as an attested vector set downstreams can diff against (#52). - `schema/validate-examples.py` validates every vector, so CI's existing schema job covers serialized reference output, not only curated fixtures. Verified red-then-green: restoring the old `required` list fails vector line 4 (query/minimal-unfiltered) while every hand-authored example still passes — which is exactly why the bug lived this long.
… rename (#58, #21, #56) #58 — the schema's `$id` was `https://context-graph-protocol.org/...`, a hyphenated host that was never registered: it returned a DNS failure, so every third party that tried to resolve the schema's public identity got nothing. Repointed at the live `contextgraphprotocol.org`, and `workspace.homepage` now names the site rather than the repo. An `$id` that 404s is barely better than one that never resolves, so the site actually serves the file from `site/public/schema/`. A served copy can drift from the source of truth — worse than a 404, because a stale schema that still resolves silently validates the wrong thing — so `validate-examples.py` asserts the copy is byte-identical and that `$id` matches the URL it is served from. Verified red-then-green by mutating the copy. #21 — residue the mechanical OCP→Context Graph Protocol rename left behind: "An Context Graph Protocol host" (twice) and "Context Graph Protocol protocol version" (three places, the doubled noun the old acronym hid). MIGRATION.md's `ocp-*` references are deliberately kept: naming the old crates is the entire job of a migration guide. #56 — removed docs/future/context-receipt-impact-trace-plan.md. Verified before deleting: it is a completed agentic implementation plan (embedded "REQUIRED SUB-SKILL" directive, `- [ ]` task tracking), it names a stack this repo does not use (Vinext/Vite, Cloudflare Workers vs. the actual Next.js/Fumadocs site), the feature it plans already shipped (site/src/app/_components/ContextReceiptDemo.tsx + site/tests/context-receipt.test.mjs), and nothing links to it. Its sibling *design* doc is referenced from docs/sketches/host-trace.md and stays.
) Requiring each `--misbehave` mode to be caught by the check it *declares* it trips — instead of by any check at all — immediately found a hole, which was the point of tightening it. **§H4 correlation was unenforced.** `drop-correlation-id` claimed to trip a `correlation` check that did not exist. It "went red" only because dropping the id desynchronizes the host's demultiplexer and breaks every other check downstream. So an external implementation — each of the three SDKs — could declare `correlation: true`, never echo an id, and pass the suite. Added a raw-stdio `correlation` probe that asserts the echo verbatim. **§Q1 `kinds` was a filter nothing honored.** `sample_query()` sends `kinds: []`, so every provider was only ever asked the unfiltered question — and not one of the four reference implementations honored the filter at all. They declared `capabilities.query.kinds` and returned whatever they had. That is ADR 0004's dead-capability surface in its subtler form: not an unreachable field, but a reachable one that silently does nothing. Specified rather than dropped (SPEC §5 Q1 + §5.1), because `unsupported_kind` already exists to answer "you asked for kinds I don't serve", which presupposes the filter binds. A host that narrows to `["snippet"]` and silently receives `doc` frames has had its budget spent on content it explicitly excluded. Reference provider now filters; new `kinds-filter` check and `ignore-kinds` misbehave mode. The fixture's second frame became a `snippet` on purpose — with every frame one kind the check passed vacuously, which is the decorative-check sin this round is removing, not adding. **`granted_at` was never stamped.** Every consent decision entered the audit ledger with no time on it, because the host had no way to spell the current instant. Added `format_protocol_timestamp` to contextgraph-types — dependency-free Gregorian arithmetic, since every implementer in every language ports this crate and "pull in chrono" is a cost paid by the whole ecosystem. `ConsentStore::record` stamps on insertion; a caller-supplied instant is preserved (replaying a persisted decision must not rewrite the trail to "now"), and a non-F4 instant is refused rather than stored. **The docs-site witness never ran.** It passed locally and executed nowhere, making it decoration rather than a gate. Now a step in the `site` job. Verified: suite 11/11 green; every misbehave mode caught by its declared check; all three SDK example providers (TS, Python, Go) still 11/11 conformant.
… quotes it (#15) R3 says a host MUST treat frame `content` as untrusted data, delimited as quoted material and never executed as instructions. `compose_context` performed the concatenation that defeats it: `content`, `id`, and the citation label were interpolated into the `<frame …>` fence with no escaping. So a provider could serve content containing `</frame>` and every byte after it landed *outside* the quoted block, read at the host's own level — retrieved text promoted to instruction, which is the exact failure R3 exists to prevent. A `"` in a citation label was the same breakout through a field nobody thinks of as content. SPEC §11.1 already listed this as a known gap; this closes it. Escaping rather than an unguessable fence, deliberately. A random per-composition delimiter is the other standard answer and it is wrong *here*: this module's entire purpose is a byte-stable prompt prefix, so a nonce that changed per turn would bust the provider prompt cache it exists to protect — paying a real, measured cost for a guarantee escaping already provides. Only the delimiter is touched. Escaping `<`/`>` wholesale would mangle the code and markup frame content most often *is*, degrading every honest frame to harden against a rare one. Content is neutralized, never dropped: `<\/frame>` stays legible to whoever reads the prompt. Tests cover the closing-fence breakout, a forged sibling frame, an attribute breakout via a quoted citation label, that ordinary markup and generics survive untouched, and that escaping stays deterministic.
The protocol is named for the graph, and the graph was its least exercised surface. The reference fixture declared `graph: false` and served every frame with `relations: vec![]`, so G1 (labelled edges) and G2 (non-empty target URIs) passed vacuously — there were no edges to validate — and G3's anchor boost was never witnessed at all. G3 also never said what an anchor is compared *against*. Two conformant providers could reasonably match anchors against the frame `uri`, against `relations[].target_uri`, or against neither, and no test could tell graph traversal apart from ignoring `anchors` entirely. - **SPEC §G4 + §8.2**: a frame is *anchored* when its own `uri` equals an anchor (zero hops) or any `relations[].target_uri` does (one hop). A graph-declaring provider given anchors MUST return an anchored frame when it has one, and SHOULD rank anchored first. A floor on what must be found, not a ceiling on how hard a provider may look — deeper traversal stays provider-private. - **Fixture**: now genuinely graph-capable — real `doc.documents` edges, and it honors anchors by stable-partitioning anchored frames first (stable, so composition stays deterministic). - **`anchor-relevance` check**: asks an unanchored question first to discover a URI the provider actually serves, then re-asks anchored on it. Deriving the anchor from the provider's own output is what keeps it fair — the suite never invents a URI and demands the provider know it. It prefers a *relation target*, so passing requires traversing an edge rather than string-matching a frame's own uri. G3's ranking half is a SHOULD, so it is reported, not enforced. - **`ignore-anchors`** misbehave mode, caught by the matching check. - **R3 host check** now asserts breakout-resistance rather than carrying a note that compose_context lacked it — that note went stale when #15 landed. Suite is 12/12 green; every misbehave mode caught by its declared check. The three SDK example providers declare `graph: false`, so the check skips for them and they remain conformant.
The Context Frame spec's sixth question — "why was each item included, and can its effect be evaluated later?" — had provenance for the first half and nothing for the second. A host could say a frame cost 42 tokens and came from retry-policy.md, and nothing about whether including it helped. Not theoretical: a host in this ecosystem already A/B-suppresses recall to measure whether retrieval earns its budget, entirely outside the protocol, because the protocol gave it no vocabulary to say so. **The id is not new (A1).** `FrameId` is already what composition, dedup, usage reports, and `verify` key on. A second attribution id would be free to disagree with the first, and a disagreement between "the frame that was billed" and "the frame that was cited" is the confusion attribution exists to remove. **Three booleans, not a score (A2).** `selected` / `rendered` / `cited` — ADR 0007's `context_use` vocabulary — because they are separately observable and collapse badly. The case that matters is `rendered` but not `cited`: tokens paid, content read, nothing changed. A used/unused flag cannot express it and a 0–1 usefulness score would invent precision nobody measured. `cited` means the output referred to the frame — observable — never that the frame *influenced* it, which is not. **Reconcilable (A3).** Records must be coherent and name a frame the paired usage report actually billed; attribution that cannot be walked back to the bill is not auditable, and is the shape a mis-keyed identity takes. `AttributionReport` pairs the outcome records with `UsageReport`, so cost and outcome finally sit together — value-per-token, which #31 and #8 each supply half of. **Deliberately not on the wire.** No `context/feedback`, no `Capabilities.feedback`. Shipping a negotiated method with no provider consuming it and no check able to witness it would recreate exactly the dead capability surface ADR 0004 removed and §Q1 had to be written to repair — days before a freeze, when the asymmetry runs the other way: adding the method later is family-safe, removing a dead one is not. The vocabulary ships now because it must be shared for scores to compare; the transport is sketched in docs/sketches/attribution-feedback.md with its open questions (privacy of `cited` for an egress provider, batching, what a conformance check could even assert).
#53, #55) Follow-ups to #44 / ADR 0006. All of this is *provider policy* — the wire, the frame shapes, and the conformance surface are untouched; only what the prompt-ingest provider chooses to inline changes. **#53 — segmentation and distillation** - Stack traces are a first-class segment kind with their own distiller, instead of folding into the generic log heuristic. Keeping the exception line plus the top N frames and eliding the tail beats the log distiller for the single most common paste there is. - Tabular ingestion handles CSV and whitespace-aligned columns, not just pipe and tab, and `infer_column_type` recognizes currency, percent, and nullable columns — so a distilled table says what its columns *are*. - Common log timestamps (`2026-07-20 18:00:01,250`, syslog, bracketed) are parsed and normalized to the F4 profile, so the temporal window actually populates. Guarded: a shape that cannot be normalized yields no timestamp rather than a non-F4 string. - Runs of identical log lines collapse before salient-line selection, so a retry storm cannot crowd out the rest of the log. - Classification-precedence edge cases documented as known limits. **#55 — docs and a runnable example** - `docs/prompt-ingestion.md`: paste → intent + anchors + content-addressed evidence, compact-by-default with `[full]` on demand, and how it composes byte-stably. - `contextgraph-host/examples/ingest_paste.rs`: a real end-to-end run. On a 2288-token paste it segments five attachments, serves four frames for 610 tokens (27% of raw), re-composes byte-identically, and pulls `[full]` on the top-ranked frame with a matching content digest. Verified: fmt, clippy -D warnings, and all 22 test suites green; conformance 12/12; every misbehave mode caught; host-conformance, schema, and the docs-site witness green. The example was run, not just compiled.
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Sorry @macanderson, your pull request is larger than the review limit of 150000 diff characters
…e ingest page Two follow-ups. `cargo clippy --workspace --all-targets -- -D warnings` was failing on `manual_contains` in the §G4 anchor predicate I added in a9a5348. My earlier "clippy clean" reading was a **cached** result — the crate had not been re-linted after the edit, so clippy emitted no diagnostics and I misread silence as success. Re-verified here from a clean build: exit 0. Also links docs/prompt-ingestion.md from the docs index. It is deliberately not added to site/content/docs/meta.json: the site mirrors a hand-maintained subset of docs/ (context-reuse.md is likewise absent), and listing a page with no mdx copy would fail the docs-site build, which is now a hard gate.
Adding the `anchor-relevance` check broke all three SDK CI jobs, and the way it broke them is worth recording. `conformance-external.sh` asserts "every check green, **none skipped**" — a deliberately stricter bar than SPEC §12's "green for your declared capability set", because it is how the GOVERNANCE freeze criterion "≥2 independent implementations pass conformance" is machine-verified. A skip there means the external implementation did not exercise that surface at all. The new check correctly *skips* for a `graph: false` provider, and all three examples declared `graph: false` with `relations: []` — exactly the vacuum the Rust fixture had. So the honest fix is to make them graph-capable rather than to relax the gate: weakening "none skipped" would let an SDK silently stop declaring a capability and still pass. Each example now declares `graph: true`, carries a labelled `doc.documents` edge, and ranks anchored frames first (§G4) — Go via `sort.SliceStable`, so equally-anchored frames keep their relative order and composition stays deterministic. All three verified 12/12 with no skips: `go vet` clean, `tsc` builds, and each runs green under conformance-external.sh.
Owner
Author
Follow-up after opening: two real regressions caught by re-running the gatesBoth were mine, and both were hidden by a false-clean reading. Recording them because the second is a trap anyone touching the conformance surface will hit. 1. 2. The new I made the SDKs graph-capable rather than relaxing the gate ( Final state — every gate by exit code, from a clean buildPlus all three SDK example providers at 12/12 with no skips ( |
macanderson
marked this pull request as ready for review
July 26, 2026 18:53
There was a problem hiding this comment.
Sorry @macanderson, your pull request is larger than the review limit of 150000 diff characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds ten issues. The theme is the same one PR #60 started: a guarantee nobody can falsify is not a guarantee. Tightening the checks kept finding surfaces that only looked enforced.
The finds
Three defects surfaced from making an existing check stricter, not from reading the issues:
§H4 correlation was unenforced. The
drop-correlation-idmisbehave mode claimed to trip acorrelationcheck that did not exist. It "went red" only because dropping the id desynchronizes the host's demultiplexer and breaks everything downstream. Any of the three SDKs could have declaredcorrelation: true, never echoed an id, and passed the suite.§Q1
kindswas a filter nothing honored.sample_query()sendskinds: [], so every provider was only ever asked the unfiltered question — and not one of the four reference implementations honored the filter. They declaredcapabilities.query.kindsand returned whatever they had. ADR 0004's dead-capability surface in its subtler form: not an unreachable field, a reachable one that silently does nothing.R3 was defeatable in one line.
compose_contextinterpolated untrustedcontentinto the<frame>fence with no escaping, so a frame containing</frame>ended its own quoted block and everything after it was read at the host's own level. SPEC §11.1 already listed this; it is now closed.What landed
ContextQuery.kinds/.anchorswere schema-requiredbut omitted byskip_serializing_if— the unfiltered query was un-representable in conformant JSON. Fixed, plusschema/reference-vectors.ndjson: every envelope serialized from the reference types in minimal and maximal form, drift-gated in Rust and schema-validated in CI.correlationcheck, §Q1 +kinds-filter,granted_atstamping (with a dependency-free F4 formatter), and the docs-site witness that had never run in CI.uri, one hop viarelations[].target_uri). The fixture is now genuinely graph-capable — it hadgraph: falseand no relations at all, so G1/G2 passed vacuously.selected/rendered/citedagainst the existingFrameId, paired withUsageReportfor value-per-token.$idpointed at an unregistered host; now canonical, served from the site, and drift-gated. Rename residue. Removed a completed plan doc.Deliberate non-additions
#31ships nocontext/feedbackmethod and noCapabilities.feedback. Adding a negotiated capability with no provider consuming it and no check able to witness it would recreate the exact defect ADR 0004 removed and §Q1 had to be written to repair — and the asymmetry favors waiting: adding it later is family-safe, removing a dead one is not. The vocabulary ships (it must be shared for scores to compare); the transport is sketched with its open questions indocs/sketches/attribution-feedback.md.Also worth knowing
Several issues I first selected turned out already satisfied by the prior sweep — #12 (digest verification:
contextgraph_host::verify+ the F5-bytes host check exist), #11 (E1 fingerprint enforcement), #4/#5/#6/#9 (correlation ids,upsert/subscribedropped, error codes), and #52's fixture half (goldens already cover representations and satisfy B3). They can be closed on evidence. I swapped in issues with real residue.Verification
Every gate run locally, green:
cargo fmt --check,cargo clippy --workspace --all-targets -D warnings, 22 test suitesTwo changes were verified red-then-green rather than assumed: restoring the old
requiredlist fails vector line 4 while every hand-authored example still passes (which is why that bug lived so long), and mutating the served schema copy trips the drift guard.