From 8cbd86c8f39453c35424f72b79ad892d64b35513 Mon Sep 17 00:00:00 2001 From: Egor Merkushev Date: Sat, 13 Jun 2026 01:48:04 +0300 Subject: [PATCH 1/2] Document Hypercode backlog hardening --- CHANGELOG.md | 31 +++++++++++++++++++ DOCS/Backends.md | 11 ++++--- DOCS/Dogfooding.md | 35 +++++++++++++++++++--- DOCS/IntegrityChain.md | 66 +++++++++++++++++++++++++++++++++++++++++ DOCS/TypeSystemDepth.md | 56 ++++++++++++++++++++++++++++++++++ RFC/Hypercode.md | 17 +++++++---- workplan.md | 6 ++-- 7 files changed, 207 insertions(+), 15 deletions(-) create mode 100644 DOCS/IntegrityChain.md create mode 100644 DOCS/TypeSystemDepth.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 3e98240..c297f63 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,37 @@ All notable changes to this project are documented here. The format is based on the directive position (the importer wins specificity ties), import-once for diamonds, cycle detection, cross-file provenance; contracts compose across imports. `ImportHandling` API (`.unsupported`/`.syntaxOnly`/`.loader`). +- Runtime resolver boundary decision record (HC-114): build/generation-time + resolution remains the supported mode for the reference implementation; + embedded runtime resolution is out of scope with explicit revisit conditions, + and HC-115 (OpenFeature bridge) is parked. +- Kustomize comparison demo (HC-120): a 3 tenants x 3 environments example + modeled both as Kustomize overlays and as Hypercode sheets, with CI metrics + for source files, meaningful lines, duplicated lines, and validation of all + 9 tenant/environment targets. +- Ontology backend dogfooding example (HC-121): the real `examcalc` + DomainOntologyPackage modeled as Hypercode and regenerated from IR v2 by a + consumer-side Python adapter; CI checks semantic equality against the + original YAML and records the first dogfooding friction log (F1-F5). +- HC-121 consumer closure notes: Ontology's `ontologyc import-hypercode` + consumes ontology-shaped IR v2 as real `DomainOntologyPackage` sections while + preserving draft-only governance, and Hyperprompt has a runnable + core-Hypercode compile-configuration exercise. +- RFC §10 backlog contracts: `DOCS/IntegrityChain.md` for HC-122 and + `DOCS/TypeSystemDepth.md` for HC-125/HC-126, turning the remaining integrity + chain and richer-type questions into explicit acceptance criteria. + +### Changed +- `@import` trust model and library identity contract are documented: the CLI + loader is for trusted local configuration, embedders resolving untrusted + sheets must provide a policy-bound `ImportHandling.loader`, and + `CascadeSheetReader.read(file:imports:)` requires the entry file identity to + use the same canonical scheme as loader-returned identities. +- Ontology backend validation is stricter: duplicate top-level sections and + duplicate class/relation/policy/state-machine/field ids now fail with a clear + `malformed package` error instead of silently overwriting earlier entries. +- Demo Python helpers use explicit UTF-8 file/subprocess boundaries where they + read generated IR, YAML fixtures, and source trees. ## [0.5.0] — 2026-06-12 diff --git a/DOCS/Backends.md b/DOCS/Backends.md index e31268a..d54a9a0 100644 --- a/DOCS/Backends.md +++ b/DOCS/Backends.md @@ -21,7 +21,7 @@ Compilation to a concrete language or format is **not** part of Hypercode core. ## Worked example: Ontology -`ontologyc` (Swift, separate repo) would add an `import-hypercode` step: +`ontologyc` (Swift, separate repo) provides an `import-hypercode` step: ```text *.ontology.hc + *.hcs @@ -31,6 +31,9 @@ Compilation to a concrete language or format is **not** part of Hypercode core. ``` The `--schema domain-ontology-package` knowledge stays on the Ontology side — -Hypercode never learns the ontology schema. Since `ontologyc` and the Hypercode -reference tooling are both Swift, the consumer can either link the `Hypercode` -library or shell out to the `hypercode` CLI and read the IR file. +Hypercode never learns the ontology schema. The consumer implementation reads +`hypercode.ir/v2` JSON and maps an ontology-shaped graph to +`DomainOntologyPackage` YAML inside `ontologyc`; generic graphs still become +reviewable class drafts. Imports remain draft-only: a Hypercode context may +resolve `approval_status`, but trusted Ontology approval is a governance +decision, not an import side effect. diff --git a/DOCS/Dogfooding.md b/DOCS/Dogfooding.md index 20bf055..b326d95 100644 --- a/DOCS/Dogfooding.md +++ b/DOCS/Dogfooding.md @@ -61,7 +61,34 @@ on the first complete run. Parity on size for one package (201 vs 212 meaningful lines), clear wins on defaults, lifecycle-as-context, diff, contracts and provenance. The compression story starts at the second package, when the kind defaults and -contracts move to a shared `@import`ed baseline. Next adoption step -(remaining scope of HC-121): an `import-hypercode` step inside `ontologyc` -itself, consuming the IR the way `backend.py` does, and the same exercise -for a Hyperprompt configuration. +contracts move to a shared `@import`ed baseline. + +## Entry 2 — Consumer bridge closure (2026-06-13) + +**Ontology:** `ontologyc import-hypercode` now consumes current +`hypercode.ir/v2` as well as the original v1 spike. Generic graphs still +become reviewable class drafts. Ontology-shaped v2 graphs +(`Package > Metadata/Imports/Classes/Relations/Policies/StateMachines/ +Compatibility`) map resolved properties into real `DomainOntologyPackage` +sections, preserving the Backends.md boundary: Ontology owns target-schema +knowledge, Hypercode only emits IR. The real `examcalc` IR emitted by this +repo semantically matches Ontology's canonical YAML after import. + +The bridge deliberately rejects ontology-shaped IR whose resolved +`approval_status` is not `draft`. Hypercode can model lifecycle context, but +trusted Ontology approval remains a governance decision boundary, not an import +side effect. + +**Hyperprompt:** a runnable core-Hypercode exercise was added in the consumer +repo for Hyperprompt compile configuration. It validates/ emits with the +Hypercode CLI and shows that core Hypercode is useful for resolved +configuration profiles (`profile=ci`, `profile=editor`), while also confirming +the earlier dialect decision: Hyperprompt's document-compilation language should +not be replaced by the core Hypercode parser. + +### HC-121 closure + +HC-121 is closed as adoption evidence, not as a mandate to merge dialects. The +remaining F1/F2 pain points feed the type-system depth backlog +([TypeSystemDepth](TypeSystemDepth.md)); integrity evidence feeds HC-122 +([IntegrityChain](IntegrityChain.md)). diff --git a/DOCS/IntegrityChain.md b/DOCS/IntegrityChain.md new file mode 100644 index 0000000..b807434 --- /dev/null +++ b/DOCS/IntegrityChain.md @@ -0,0 +1,66 @@ +# Integrity Chain Backlog (HC-122) + +**Status:** Planned · **Date:** 2026-06-13 · Backlog contract for RFC §10 +"Integrity chain". + +## The gap + +Hypercode now has stable source inputs, IR v2 document/node hashes, generator +examples, and contract validation. What it still lacks is an end-to-end +evidence envelope that binds those pieces into a reviewable supply-chain +record: + +```text +.hc/.hcs sources + -> resolved IR hash + -> generator identity/version + -> generated artifact hashes + -> validator report + -> optional signature / provenance envelope +``` + +Without that record, `hypercode diff` can explain *what* changed and generator +checks can prove freshness, but governance cannot yet answer "which generator +produced this artifact from which reviewed graph?" + +## Planned contract + +HC-122 should introduce a machine-readable `hypercode.attestation/v1` JSON +artifact. The first version should be unsigned by default but structurally ready +for signing. + +Required fields: + +- `schema`: literal `hypercode.attestation/v1`; +- `subject`: package/repository identity and source ref when available; +- `sources`: ordered `.hc`/`.hcs` inputs with path and SHA-256; +- `resolvedIR`: `version`, `documentHash`, optional file path, and SHA-256 of + the emitted IR bytes; +- `generator`: name, version, command, and policy boundary; +- `artifacts`: generated files with path, SHA-256, and source node hash when + node-scoped; +- `validation`: command, exit status, and validator report hash; +- `signature`: optional envelope metadata, not required for v1 conformance. + +## Non-goals for v1 + +- No key management, PKI, DID, or transparency log in core. +- No network calls. +- No claim that generated code is correct beyond the recorded validator result. +- No replacement for SLSA provenance; use SLSA terms where they fit, but keep + the Hypercode artifact small and local first. + +## Acceptance criteria + +- JSON Schema under `Schema/`. +- One CLI command or checked helper that emits the envelope for + `Examples/codegen-demo/`. +- CI validates the envelope schema and verifies all recorded file hashes. +- RFC §8/§10 links to this artifact and states exactly which trust question it + answers. + +## Release posture + +This is not a 0.6.0 blocker. It becomes release-critical only when a downstream +consumer treats generated artifacts as governance evidence rather than local +build outputs. diff --git a/DOCS/TypeSystemDepth.md b/DOCS/TypeSystemDepth.md new file mode 100644 index 0000000..3fde9bd --- /dev/null +++ b/DOCS/TypeSystemDepth.md @@ -0,0 +1,56 @@ +# Type-System Depth Backlog + +**Status:** Planned · **Date:** 2026-06-13 · Backlog contract for RFC §10 +"Type-system depth". + +## The gap + +IR v2 carries scalar typed values (`string`, `int`, `double`, `bool`) and the +contract layer enforces scalar type/bounds/requiredness. Dogfooding exposed the +next real pressure points: + +- **F1 lists:** Ontology fields such as `implements`, `appliesTo`, `states`, + relation `oneOf`, and compatibility lists currently travel as comma-joined + strings. +- **F2 unions:** `cardinality.max` is semantically `int | "*"`, but contracts + cannot express that. + +Those are real gaps, but adding a full configuration-language type system would +violate the core invariant: Hypercode is an addressable topology plus resolved +properties, not a replacement for CUE/Pkl/Dhall/KCL. + +## Design constraints + +- Keep the core smaller than YAML: no arbitrary maps in `.hcs` v1. +- Preserve deterministic IR hashing and JSON Schema compatibility. +- Keep selector contracts monotonic: richer types must still only narrow. +- Preserve consumer ownership of target schemas. Hypercode may type a value; it + must not learn DomainOntologyPackage, Terraform, or prompt-specific schemas. + +## Candidate sequence + +1. **HC-125 list scalars.** Add a minimal list literal or sanctioned list + scalar convention, with IR v2/v3 migration rules and contract syntax such as + `list`. +2. **HC-126 union or pattern constraints.** Cover `int | "*"` either with + explicit unions (`int | string`) or a narrower scalar pattern form for + sentinel values. This must include monotonicity rules. +3. **HC-127 nested property naming convention.** Decide whether flattened keys + such as `card_min` stay a consumer convention or get a core spelling + convention for target-path provenance. + +## Acceptance criteria for any type-depth change + +- RFC and `EBNF/Hypercode_Syntax.md` syntax updates. +- Reader/parser tests for good and bad literals. +- Contract monotonicity tests and HC2104 value-enforcement tests. +- IR schema update and migration note. +- A dogfooding example that removes at least one F1/F2 workaround without + making `.hcs` visibly more complex. + +## Release posture + +This is not a 0.6.0 blocker. The current scalar model is honest and usable; the +type-depth work should wait until at least two consumers need the same richer +shape, so the extension is common infrastructure rather than Ontology-specific +schema leakage. diff --git a/RFC/Hypercode.md b/RFC/Hypercode.md index 6e2b850..8a44325 100644 --- a/RFC/Hypercode.md +++ b/RFC/Hypercode.md @@ -320,8 +320,8 @@ Spec-driven development is converging on the view that the specification is the ### 9.8. Acknowledged Limits * **Context binds at resolve time.** `--ctx` is supplied when resolving: Hypercode's default mode decides context at build/generation time. Runtime feature-flag systems (OpenFeature, LaunchDarkly) decide flag values per request at runtime — a different layer that composes with Hypercode rather than competing with it. Serving dynamic context from a single deployment (e.g., many tenants per process) would require embedding the resolver as a runtime library; that mode raises its own caching, latency, and provenance questions and is **decided out of scope** for the reference implementation (HC-114, decision record with revisit conditions: [`DOCS/RuntimeBoundary.md`](../DOCS/RuntimeBoundary.md)). The resolver remains an embeddable pure function of `(sheet, context)`, but core makes no runtime API commitments — no caching, no sheet hot-reload, no per-request provenance sink. -* **No integrity chain yet.** As noted in §8, nothing verifies the chain end-to-end: signed `.hc`/`.hcs` → resolved-IR hash → generator identity and version → generated-artifact hashes → validator report. SLSA provides the reference vocabulary for such attestations. For the review-compression story to carry governance weight they are eventually required; they are deliberately deferred as future work. -* **Type-system maturity.** IR v2 carries typed values (string/int/double/bool, inferred at parse time with the source lexeme preserved), and the contract layer (§9.4) ships declarations and monotonicity validation. Resolved values are enforced against the effective contract per context (HC2104, `validate --ctx`). What is still missing versus typed configuration languages: richer value types (lists, maps, durations), cross-property constraints, and user-defined predicates. IR v1 remains strings-only and is kept for backward compatibility. +* **No integrity chain yet.** As noted in §8, nothing verifies the chain end-to-end: signed `.hc`/`.hcs` → resolved-IR hash → generator identity and version → generated-artifact hashes → validator report. SLSA provides the reference vocabulary for such attestations. For the review-compression story to carry governance weight they are eventually required; HC-122 now has a concrete backlog contract in [`DOCS/IntegrityChain.md`](../DOCS/IntegrityChain.md). +* **Type-system maturity.** IR v2 carries typed values (string/int/double/bool, inferred at parse time with the source lexeme preserved), and the contract layer (§9.4) ships declarations and monotonicity validation. Resolved values are enforced against the effective contract per context (HC2104, `validate --ctx`). What is still missing versus typed configuration languages: richer value types (especially lists), union/pattern constraints such as `int | "*"`, cross-property constraints, and user-defined predicates. F1/F2 from dogfooding are tracked as HC-125/HC-126 in [`DOCS/TypeSystemDepth.md`](../DOCS/TypeSystemDepth.md). IR v1 remains strings-only and is kept for backward compatibility. ## 10. Open Questions @@ -341,9 +341,13 @@ The original open questions of draft 0.1 have since been answered: Genuinely open: * **Integrity chain** (§8, §9.8): signed sources → resolved-IR hash → - generator identity → artifact hashes → validator report (SLSA vocabulary). -* **Type-system depth** (§9.8): richer value types, cross-property - constraints, user-defined predicates. + generator identity → artifact hashes → validator report (SLSA vocabulary); + planned as HC-122 with the acceptance contract in + [`DOCS/IntegrityChain.md`](../DOCS/IntegrityChain.md). +* **Type-system depth** (§9.8): richer value types, union/pattern constraints, + cross-property constraints, user-defined predicates; dogfooding F1/F2 are + planned as HC-125/HC-126 in + [`DOCS/TypeSystemDepth.md`](../DOCS/TypeSystemDepth.md). ## 11. References @@ -381,6 +385,9 @@ Prior art surveyed in §9: * §10 Open Questions brought up to date: debugging (HC-110), binding time (HC-114) and complexity management (HC-116) recorded as answered; the integrity chain and type-system depth remain open. +* Integrity chain and type-system depth now link to explicit backlog contracts: + `DOCS/IntegrityChain.md` (HC-122) and `DOCS/TypeSystemDepth.md` + (HC-125/HC-126). * §9.4 and §9.8 no longer claim HC2104 is in progress — value-level enforcement shipped in 0.5.0. diff --git a/workplan.md b/workplan.md index adba1ef..2fbfc95 100644 --- a/workplan.md +++ b/workplan.md @@ -104,10 +104,12 @@ P1 — built on v2: ## M9 — Validation & adoption (DOCS/Positioning.md) - [x] HC-120 Kustomize comparison demo — `Examples/kustomize-comparison/`: 3 tenants × 3 envs maintained twice (idiomatic Kustomize components vs `.hc` + `@import`ed tenant sheets); computed metrics (28 files/278 lines/86% duplication vs 5/57/30%), "why is this value here" via `explain`, affected-target precision via `hypercode diff`, and the own failure mode (specificity beats source order) diagnosed via `explain`; `metrics.py --check` validates all 9 targets in CI -- 🔜 HC-121 Dogfooding as the primary adoption path — **first artifact shipped**: the real Ontology `examcalc` package round-trips through IR v2 (`Examples/ontology-backend/`: kind defaults via selectors, `@stage` lifecycle context, contracts, consumer adapter; semantic match with the original verified in CI), friction log started in `DOCS/Dogfooding.md` (F1–F5); remaining: `import-hypercode` step inside `ontologyc` itself + the Hyperprompt configuration exercise -- 🅿️ HC-122 SLSA-like generation attestation chain — signed `.hc`/`.hcs` → IR hash → generator identity/version → artifact hashes → validator report (RFC §8, §9.8) +- [x] HC-121 Dogfooding as the primary adoption path — Ontology `examcalc` round-trips through IR v2 (`Examples/ontology-backend/`: kind defaults via selectors, `@stage` lifecycle context, contracts, consumer adapter; semantic match with the original verified in CI); consumer-side closure added in Ontology (`ontologyc import-hypercode` accepts IR v2 ontology-shaped package graphs, keeps imports draft-only, and semantically matches canonical `examcalc`) and Hyperprompt (runnable core-Hypercode compile-configuration exercise); friction log F1–F5 in `DOCS/Dogfooding.md` +- ⬜ HC-122 SLSA-like generation attestation chain — planned contract in `DOCS/IntegrityChain.md`: signed `.hc`/`.hcs` → IR hash → generator identity/version → artifact hashes → validator report (RFC §8, §9.8); not a 0.6.0 blocker - 🅿️ HC-123 Agent Passport / 0AL integration — attestation chain plugs into 0AL's signed-agent model - [x] HC-124 End-to-end AI codegen demo — `Examples/codegen-demo/`: service spec → IR v2 → Claude-generated module per node (embedded node hash + provenance comments) → `check.py` validates artifacts against the same contracts (HC2104-gen) and scopes regeneration by node hash; `generate.sh` regenerates stale modules via `claude -p`; checked in CI on every push +- ⬜ HC-125 Type-system depth: list scalars — planned in `DOCS/TypeSystemDepth.md`; driven by dogfooding F1 (`implements`, `appliesTo`, `states`, `oneOf`, compatibility lists); requires IR/schema/contract tests and at least one workaround removal +- ⬜ HC-126 Type-system depth: unions or pattern constraints — planned in `DOCS/TypeSystemDepth.md`; driven by dogfooding F2 (`cardinality.max: int | "*"`); must preserve monotonic contracts ## Cross-cutting - [x] HC-090 Swift CI workflow (build + test) — `.github/workflows/swift.yml` From 483bb85f85d38a6c810748689ebb1acfb129dde1 Mon Sep 17 00:00:00 2001 From: Egor Merkushev Date: Sat, 13 Jun 2026 14:40:32 +0300 Subject: [PATCH 2/2] Address dogfooding wording review --- DOCS/Dogfooding.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/DOCS/Dogfooding.md b/DOCS/Dogfooding.md index b326d95..8c2ca95 100644 --- a/DOCS/Dogfooding.md +++ b/DOCS/Dogfooding.md @@ -80,7 +80,7 @@ trusted Ontology approval remains a governance decision boundary, not an import side effect. **Hyperprompt:** a runnable core-Hypercode exercise was added in the consumer -repo for Hyperprompt compile configuration. It validates/ emits with the +repo for Hyperprompt compile configuration. It validates and emits with the Hypercode CLI and shows that core Hypercode is useful for resolved configuration profiles (`profile=ci`, `profile=editor`), while also confirming the earlier dialect decision: Hyperprompt's document-compilation language should