Skip to content

feat: committed-state adapter (verifiable root + /committed routes)#158

Closed
ottobot-ai wants to merge 7 commits into
scasplte2:mainfrom
ottobot-ai:feat/committed-state-adapter
Closed

feat: committed-state adapter (verifiable root + /committed routes)#158
ottobot-ai wants to merge 7 commits into
scasplte2:mainfrom
ottobot-ai:feat/committed-state-adapter

Conversation

@ottobot-ai

@ottobot-ai ottobot-ai commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Stacks on #157 (GitHub can't base cross-fork PRs on a fork branch, so this targets main and currently shows the bump commits too — the diff shrinks to adapter-only once #157 merges). DRAFT until metakit v1.8.0-rc.2 is tagged and on Central (pin set to rc.2; everything validated against the equivalent publishLocal snapshot — 271/271 tests green).

What this does

  • CommittedView[CalculatedState]: fibers → fiber/<uuid>, scripts → script/<id> (canonical Json, SortedMap, fast-path delta). registry/<name> reserved, not yet emitted. No "oracle" in any new code (legacy /v1/oracles route paths retained as existing API surface).
  • ML0Service.make → CommittedApp.makeL0: hand-rolled service collapses into metakit's correct-by-construction assembly; calculated-state hash = two-tier committed root (MPT state-dict + SMT epoch catalog) with the constant onchain breadcrumb; followers validate breadcrumb transitions in combine.
  • Preserved behavior via the smallest wrapper (withConsensusHooks): webhook/subscriber dispatch + checkpoint refresh on consensus; snapshot-backed /v1 routes unchanged (CommittedOnChain unwrapped transparently).
  • Hydration client for bootstrap (peer /committed/catalog → own verify-gated /committed/hydrate; env-configured COMMITTED_HYDRATION_*, inert when unset).
  • 1.8.0 compat: Ratio-backed FloatValue fixes (toBigInt, Ratio comparisons, estimateValueSize decimal rendering); gas re-baselines unnecessary (suites assert bounds, not exact charges).
  • New suites: CommittedAdapterSuite + CommittedHydrationClientSuite (8 tests).

Metakit follow-ups flagged in code

CommittedApp.makeL0 wants an onSnapshotConsensusResult hook + context-aware extraRoutes (dissolves the wrapper); generic HTTP hydration client beside CommittedReplica.

🤖 Generated with Claude Code

@ottobot-ai ottobot-ai force-pushed the feat/committed-state-adapter branch 2 times, most recently from d6027a1 to 0f1e65c Compare June 11, 2026 01:14
ottobot-ai and others added 4 commits June 10, 2026 22:57
@ottobot-ai @claude
feat(models): add committed-view projection of calculated state
d197065 
Project the calculated state into metakit's committed state dictionary
(lifecycle/committed): state-machine fibers under fiber/<uuid>, scripts
under script/<id> (registry/<name> is reserved but not yet emitted).
Values are the records' canonical circe projections; SortedMap keeps the
enumeration order canonical for deltas and snapshots.

The delta override diffs the record maps by case-class equality first and
only serializes records that actually changed — structurally identical to
the default entries-level diff, minus the redundant encoding work.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ottobot-ai @claude
feat(l0): assemble the ML0 service via CommittedApp.makeL0
3f15e4c 
The L0 data application is now built by metakit's CommittedApp.makeL0:
two-tier state-root commitment over CommittedView[CalculatedState], the
constant on-chain breadcrumb (ordinal, mptRoot, catalogRoot) wrapped
around OnChain, and the /committed/... proof/replication routes — all
correct by construction.

Ottochain behavior is preserved by composition:
- orderedCombiner: canonical OttochainMessage batch ordering + per-batch
  latestLogs reset, as the dev combiner handed to makeL0;
- rejectionNotifyingValidator: per-update validation with fire-and-forget
  rejection webhooks, result accumulation unchanged;
- withConsensusHooks: delegating wrapper that keeps the consensus-result
  webhook dispatch (and the notification-side checkpoint cache refresh on
  setCalculatedState) and appends the snapshot-backed routes;
- ML0CustomRoutes now reads through the CommittedReader handed out by
  makeL0 (one atomic cell read per request, consistent with the served
  /committed roots); routes that need the latest SIGNED snapshot
  (/v1/onchain, fiber events, script invocations) moved to
  ML0SnapshotStateRoutes, which unwraps CommittedOnChain[OnChain] so
  response shapes are unchanged.

FLAGGED metakit follow-up: makeL0 exposes no onSnapshotConsensusResult
hook and extraRoutes does not receive the L0NodeContext, hence the
wrapper; it disappears once makeL0 grows those parameters.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ottobot-ai @claude
feat(l0): committed-catalog hydration client
d0caef6 
A node bootstrapped from snapshot download seeds its committed cell from
the on-chain breadcrumb (SeededCatalog): state-dict proofs work, but
catalog proofs and consensus transitions need the full epoch-rollup
contents. CommittedHydrationClient closes the gap over the public routes:
GET <self>/committed/root (act only on hydrated=false), GET
<peer>/committed/catalog, POST <self>/committed/hydrate. The install is
verify-gated server-side (contents must recompose to the attested
catalog root), so peers are untrusted; they are tried in order, and the
loop polls until the cell reports hydrated.

Wired in Main behind config (all-optional, env-driven):
COMMITTED_HYDRATION_SELF_URL, COMMITTED_HYDRATION_PEERS (comma-separated),
COMMITTED_HYDRATION_INTERVAL (default 30s) — a supervised background task
only when self-url and at least one peer are set.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ottobot-ai @claude
test(l0): committed adapter + hydration client suites
fbbab70 
CommittedAdapterSuite:
- projection: fibers -> fiber/<uuid>, scripts -> script/<id>, valid
  CommitKey grammar, deterministic enumeration regardless of in-memory
  assembly order, empty delta on identical states;
- fiber transition: minimal single-key delta that agrees with the default
  structural diff, MPT root change, and delta-application == full rebuild
  (the invariant CommittedState.setCommitted asserts every transition);
- service-level combine: on-chain breadcrumb advances ordinal 0 -> 1 -> 2,
  mptRoot changes with state, catalogRoot changes every snapshot;
- routes: /committed/root responds (ordinal 0, hydrated, roots + combined
  hash) and the committed-cell-backed /v1 custom routes respond.

CommittedHydrationClientSuite drives the client against an in-process
stub network: no-op when already hydrated, peer fallback chain, the
verify-gate rejection path, and full-failure reporting.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ottobot-ai ottobot-ai force-pushed the feat/committed-state-adapter branch from d9bbd43 to 9dee91b Compare June 11, 2026 03:59
@ottobot-ai @claude
Merge main into feat/committed-state-adapter
b029cb7 
Brings merged scasplte2#154 (versionable contracts + the combine consensus fix) under
scasplte2#158's committed-state adapter. Key resolutions:
- ML0Service: keep scasplte2#158's CommittedApp.makeL0 + orderedCombiner/withConsensusHooks,
  source genesis via scasplte2#154's GenesisLoader (preserve genesis-from-file).
- Consensus fix preserved: scasplte2#158 inherits scasplte2#154's CombineRejected combiners and
  orderedCombiner delegates to the fixed Combiner.insert (no batch abort).
- signedOrdering made TOTAL in models (signature tiebreak) per review, so
  orderedCombiner is a plain batch.sorted(signedOrdering); the combine-level digest
  tiebreak is gone.
- CalculatedState: 4-arg genesis + scasplte2#158's committedView. KNOWN GAP (scaladoc):
  registry + reverseNames not yet projected into the committed root -- follow-up.
- Config: keep both hydration (scasplte2#158) and genesis (scasplte2#154) blocks/readers.
- 341+8 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ottobot-ai added a commit to ottobot-ai/ottochain that referenced this pull request Jun 11, 2026
@ottobot-ai @claude
Merge main into feat/committed-state-adapter
a8ac6fb 
Brings merged scasplte2#154 (versionable contracts + the combine consensus fix) under
scasplte2#158's committed-state adapter. Key resolutions:
- ML0Service: keep scasplte2#158's CommittedApp.makeL0 + orderedCombiner/withConsensusHooks,
  source genesis via scasplte2#154's GenesisLoader (preserve genesis-from-file).
- Consensus fix preserved: scasplte2#158 inherits scasplte2#154's CombineRejected combiners and
  orderedCombiner delegates to the fixed Combiner.insert (no batch abort).
- signedOrdering made TOTAL in models (signature tiebreak) per review, so
  orderedCombiner is a plain batch.sorted(signedOrdering); the combine-level digest
  tiebreak is gone.
- CalculatedState: 4-arg genesis + scasplte2#158's committedView. KNOWN GAP (scaladoc):
  registry + reverseNames not yet projected into the committed root -- follow-up.
- Config: keep both hydration (scasplte2#158) and genesis (scasplte2#154) blocks/readers.
- 341+8 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ottobot-ai @claude
refactor(l0): remove committed-catalog peer hydration
658f5e9 
The peer-hydration path (selfUrl/peers config, uriListReader, the
CommittedHydrationClient + Main wiring) duplicated work the framework
already does: tessellation syncs ML0 snapshots to L1 and exposes them via
L1NodeContext.getLastCurrencySnapshot, and stateful validation belongs at
ML0 (validateData has CalculatedState) with L1 staying stateless. The
custom HTTP self/peer pull fought that split.

Keeps the committed-root state commitment (CommittedApp.makeL0,
CommittedView, /committed routes) — that is the verifiable-root/proof
adapter and is unaffected. A seeded-but-unhydrated cell still attests the
current root; serving HISTORICAL proofs (epoch catalog) will come back via
a CatalogJournal (local restart recovery) when proofs are actually served.

341+4 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ottobot-ai ottobot-ai changed the title feat: CommittedState adapter — CommittedApp.makeL0, fiber/script projection, hydration client feat: committed-state adapter (committed-root commitment + /committed routes) Jun 11, 2026
@ottobot-ai ottobot-ai changed the title feat: committed-state adapter (committed-root commitment + /committed routes) feat: committed-state adapter (verifiable root + /committed routes) Jun 11, 2026
@ottobot-ai ottobot-ai force-pushed the feat/committed-state-adapter branch from e6ca901 to 658f5e9 Compare June 11, 2026 20:32
@ottobot-ai @claude
fix(l0): thread persistent CatalogJournal into committed cell
d82fa8a 
The committed adapter's `combine` -> `advanceWork` -> `resolveCatalog` returns
None on a seeded/restarted cell whose work-cache is empty and journal is None,
raising `BreadcrumbUnresolvable` and stalling the metagraph at ordinal 1.

Acquire a LevelDB-backed `CatalogJournal` Resource in ML0 Main (under
`committed-catalog`) and thread it through `ML0Service.make` into
`CommittedApp.makeL0`, so a seeded committed cell hydrates from its own
write-through catalog and can resolve the parent breadcrumb. Test callers keep
the `journal = None` default.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ottobot-ai

ottobot-ai commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator Author

Parking this for now — nothing currently depends on the committed-state adapter. main (plain CheckpointService ML0Service) is the working baseline and its E2E is green; this branch's committed cell stalls the e2e at the seed-hydration path.

Tried wiring a persistent CatalogJournal.levelDb (d82fa8a) — compiles, 341+4 unit tests green — but the e2e still stalls: the journal can't recompose a freshly-seeded cell (empty journal at first seed).

Root cause is a metakit abstraction issue, not the wiring: CommittedApp.makeL0/CommittedState.make expose journal: Option[CatalogJournal[F]] = None, but advanceWork hard-errors BreadcrumbUnresolvable when the parent catalog isn't resolvable, and the cell goes unhydrated on any seed/jump — so the journal is load-bearing, not optional. Proper fix (a committed cell should always have a journal; only persistence is optional) is a metakit change + rc.2 publish.

The adapter is forward-looking infra (verifiable state-root commitments for light-client / L1-pulls-CalculatedState, both deferred). Leaving as draft to resume when those are needed.

@ottobot-ai ottobot-ai mentioned this pull request Jun 15, 2026
Merged
ottobot-ai added a commit that referenced this pull request Jun 15, 2026
@ottobot-ai @claude
feat(committed): total signedOrdering + bump metakit rc.3
fe5eb43 
Adopt #158's total signing order: OttochainMessage.signedOrdering now
tiebreaks on the proof signatures, completing the partial message order
to a TOTAL one (pure — no Hasher). The combiner drops the per-combiner
content-digest tiebreak and just sorts by signedOrdering, so every node
folds the identical sequence (greenfield — no coordination needed).

Bump the metakit pin to 1.8.0-rc.3 (committed-state changes: required
journal + onConsensusResult/extraRoutes hooks + genesis-combine fix).
Until rc.3 lands on Central the metakit-from-source action builds the
v1.8.0-rc.3 tag; the tessellation rc.10 jar fallback is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ottobot-ai added a commit that referenced this pull request Jun 15, 2026
@ottobot-ai @claude
feat(committed): total signedOrdering + bump metakit rc.3
a791638 
Adopt #158's total signing order: OttochainMessage.signedOrdering now
tiebreaks on the proof signatures, completing the partial message order
to a TOTAL one (pure — no Hasher). The combiner drops the per-combiner
content-digest tiebreak and just sorts by signedOrdering, so every node
folds the identical sequence (greenfield — no coordination needed).

Bump the metakit pin to 1.8.0-rc.3 (committed-state changes: required
journal + onConsensusResult/extraRoutes hooks + genesis-combine fix).
Until rc.3 lands on Central the metakit-from-source action builds the
v1.8.0-rc.3 tag; the tessellation rc.10 jar fallback is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scasplte2 pushed a commit that referenced this pull request Jun 15, 2026
@ottobot-ai @claude
feat(committed): committed-state migration — verifiable calc-state ro…
4f80285 
…ot into the currency snapshot (#164)

* docs: committed-state migration plan (unify #117/#158)

Plan of record for adopting metakit committed-state (Approach B) as
hashCalculatedState, salvaging #117's field-level proof endpoint on top.
Records the key reframe: tessellation already roots calculatedStateProof
into the signed currency snapshot + global proof, so a structured root
needs no framework change. The prior roadblock was the journal=Option
stall (seeded cell -> BreadcrumbUnresolvable), not the rooting.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(committed): add CommittedView[CalculatedState] projection (Phase 1)

First slice of the committed-state migration: project the FULL
calculated state (all 4 fields) into metakit's committed dictionary so
the two-tier committed root commits to it, ready to become the currency
snapshot's calculatedStateProof.

Keys are lowercase slash-namespaced CommitKeys: fiber/<uuid>,
script/<uuid>, registry/<name>, reverse/<uuid>. Key derivation is TOTAL
(entries has no error channel; a non-total key would halt combine):
UUIDs always fit a 64-char segment, and an over-long registry name
(render up to 253) falls back to registry/h/<sha256>.

CommittedViewSuite covers the registry readable/hashed-overflow keys,
namespacing, totality, determinism, and empty genesis.

Refs docs/proposals/committed-state-migration.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(committed): wire makeL0 into ML0Service (Phase 1)

ML0Service now assembles the data application via CommittedApp.makeL0, so
the snapshot's calculatedStateProof becomes the two-tier committed root
(MPT state-dict + SMT epoch catalog) — a verifiable calculated-state root
in the signed currency snapshot, no tessellation change required.

- ML0Service: makeL0 with orderedCombiner (total-order sort + latestLogs
  reset before folding) and rejectionNotifyingValidator (per-update
  rejection webhooks); the new onConsensusResult hook refreshes the
  notification-side checkpoint cache + dispatches the snapshot webhook
  (replaces the hand-rolled onSnapshotConsensusResult); extraRoutes keeps
  the existing ML0Routes handlers.
- Main: acquire a LevelDB CatalogJournal Resource (required — without it a
  seeded/restarted node stalls).
- DL1: makeL0 commits CommittedOnChain[OnChain]; the validator's on-chain
  cache and the /onchain route decode the wrapper and read .inner, so the
  fiber sequence checks and the client-facing shape are unchanged.
- Mock fixture serializes the same CommittedOnChain shape.

Depends on metakit Constellation-Labs/metakit#48 (required journal +
onConsensusResult + extraRoutes context). sharedData 358/358 green; full
build compiles. e2e validation pending.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(committed): total signedOrdering + bump metakit rc.3

Adopt #158's total signing order: OttochainMessage.signedOrdering now
tiebreaks on the proof signatures, completing the partial message order
to a TOTAL one (pure — no Hasher). The combiner drops the per-combiner
content-digest tiebreak and just sorts by signedOrdering, so every node
folds the identical sequence (greenfield — no coordination needed).

Bump the metakit pin to 1.8.0-rc.3 (committed-state changes: required
journal + onConsensusResult/extraRoutes hooks + genesis-combine fix).
Until rc.3 lands on Central the metakit-from-source action builds the
v1.8.0-rc.3 tag; the tessellation rc.10 jar fallback is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(committed): state-proof endpoint on the committed MPT (Phase 2)

Re-homes #117's field-level state proof onto the real committed root.
GET /v1/state-machines/:id/state-proof[?field=] and
GET /v1/scripts/:id/state-proof[?field=] return a Merkle-Patricia
inclusion proof for the fiber/script record (committed at fiber/<id> /
script/<id>) against the MPT root whose combined hash IS the snapshot's
calculatedStateProof. A client verifies the proof against the
consensus-signed root, then reads any field off the proven record —
#117's two-level field->fiberRoot->metagraphRoot collapses to one level,
anchored to a real consensus root instead of an off-chain one. ?field=
also surfaces the named stateData field of the proven record.

StateProofHandler delegates to CommittedReader.committed.proveKey (no
hand-rolled tries); wired via makeL0 extraRoutes (now passed the reader).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(deps): bump metakit to rc.4; drop source-build patch

metakit 1.8.0-rc.4 is published to Maven Central with all the
committed-state changes (required journal, onConsensusResult / extraRoutes
hooks, and the genesis-combine fix) — verified in the published jar. So
the temporary metakit-from-source action is no longer needed: removed it
and its callers (ci.yml x2, e2e.yml), and e2e.yml now resolves the
tessellation-sdk version straight from the Central metakit POM again. The
tessellation rc.10 hypergraph-jar fallback is untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ottobot-ai

ottobot-ai commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #164 (merged). #164 is the realized committed-state migration this draft prototyped: it adopts metakit CommittedApp.makeL0 (two-tier committed root → calculatedStateProof), the /committed routes, and total signedOrdering. Closing as superseded.

@ottobot-ai ottobot-ai closed this Jun 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ottobot-ai