Skip to content

P1: Host conformance: add the H3 version-rejection and crash-isolation scenarios #14

Description

@macanderson

Problem

The conformance suite tests providers onlyrun_conformance (contextgraph-conformance/src/lib.rs:75) drives a provider through handshake/query/shutdown. But read the conformance tables in docs/protocol-surface.md: the highest-stakes rules bind the host:

  • C1/C2 — the host MUST NOT auto-enable egress providers or transmit payloads pre-consent
  • C4 — the host's HTTP transport MUST force egress
  • B2 — the host MUST drop and loudly report budget liars
  • H3 — version mismatch MUST be a named error, never a hang
  • R3 — frame content MUST be treated as untrusted data, never instructions

Today, "conformant host" means "is contextgraph-host, or trusted to behave like it" — which is precisely the self-attestation model the project rejects for providers (docs/protocol-advantages.md §6: "Documentation is advisory; conformance is verifiable"). The moment a second host exists (a TypeScript agent embedding CGP, say), there is no way to verify it honors the consent gate — and the consent gate is the security story.

Proposal

Grow contextgraph-conformance a host harness: an adversarial reference provider (extending the --misbehave machinery) that the host-under-test is pointed at, plus a scripted scenario runner that asserts observable host behavior:

  1. Consent (C1/C2): harness declares egress: true; asserts no query envelope ever arrives before the scenario grants consent. The observable is on the provider side — the harness records what reached it, which is the only trustworthy witness that the payload never left. — landed in Pre-freeze: normative completeness + enforcement + host conformance (top-10 build) #60
  2. Budget audit (B2): harness returns frames summing over budget; scenario asserts the host's composed output excludes them and its report names the provider. — landed in Pre-freeze: normative completeness + enforcement + host conformance (top-10 build) #60 (B4 flood-frames and C6 receipts shipped alongside)
  3. Version rejection (H3): harness acks with contextgraph/2.0; assert a named error, no hang (bounded by timeout). — still missing
  4. Crash isolation: harness dies mid-query; assert the host's overall fan-out still completes. — still missing
  5. R3 (untrusted content): full prompt-rendering can't be black-box tested, but the harness can serve frames containing instruction-shaped content and the scenario can assert the host's compose API (see the prompt-composition issue) delimits rather than inlines. Document the residual gap honestly. — landed in Pre-freeze: normative completeness + enforcement + host conformance (top-10 build) #60 as the R3-delimiting check; the R3-breakout residual is honestly recorded in SPEC §11.1 and tracked in P2: Reference prompt-composition module: global budget, cross-provider dedup, injection-resistant rendering #15

This directly serves the GOVERNANCE.md freeze criterion ("checks are agreed to fully enforce the documented conformance requirements" — impossible while host-binding rules have no check) and makes multi-host adoption safe.

Acceptance criteria

Metadata

Metadata

Assignees

No one assigned

    Labels

    conformanceThe conformance suite and its checksroadmapPart of the steering roadmapsecurityConsent, egress, integrity, injection surface

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions