diff --git a/docs/driving-port/0001-decouple-mempool-from-chain-state.md b/docs/driving-port/0001-decouple-mempool-from-chain-state.md new file mode 100644 index 000000000..06eab429e --- /dev/null +++ b/docs/driving-port/0001-decouple-mempool-from-chain-state.md @@ -0,0 +1,28 @@ +# Decouple the mempool from chain state in Zaino's driving port + +Zallet today learns of a new block through a side effect: Zaino's mempool +stream ends when the chain tip changes, and the sync loop treats that closure +as its only new-block signal (zcash/zallet, +`zallet-core/src/components/sync.rs:136`). We decided that the driving port +Zallet will consume from Zaino must not carry this coupling. The port serves +the mempool through an abstraction that stands apart from chain state, because +the protocol itself imposes no structural relation between them: a transaction +commits to no block hash. Consensus binds a transaction only to chain +*state* — shielded anchors, nullifier non-membership, transparent prevouts, +expiry height, and the consensus branch id — so a mempool view is coherent +only relative to the tip it was validated against, and the port expresses +exactly that. + +The port therefore provides an explicit chain tip-change subscription, and its +mempool capability is an independent stream whose view is tagged with the tip +it was validated against. Zallet composes the two: on a tip event it resyncs +and resubscribes to the mempool. This costs Zallet a sync-loop rewrite during +adoption. + +## Considered Options + +We rejected canonizing the closure idiom (keeping "stream ends on tip change" +as the contract), because it hides a chain-state dependency in the mempool +abstraction's fine print and forces every future engine to implement the +entanglement. We rejected tip polling by Zallet, because it adds latency and +busy-work to every implementation's hot path. diff --git a/docs/driving-port/0002-raw-bytes-at-the-driving-port.md b/docs/driving-port/0002-raw-bytes-at-the-driving-port.md new file mode 100644 index 000000000..29dac7126 --- /dev/null +++ b/docs/driving-port/0002-raw-bytes-at-the-driving-port.md @@ -0,0 +1,27 @@ +# The driving port hands consumers raw consensus-serialized bytes + +Zaino's driving port traffics in consensus-serialized bytes: blocks, +transactions, and treestate frontiers cross the boundary as bytes. Only +identifiers and locators — heights, block hashes, txids, consensus branch +ids — are typed, and those types come from `zaino-primitives` +(zingolabs/zaino#1402), a zero-dependency crate of checked domain types. We +chose byte payloads over typed domain payloads because the port's consumers +already parse consensus bytes into the types they actually want: Zallet reads +blocks and transactions into `zcash_primitives` types (zcash/zallet, +`backends/zaino/src/chain.rs`), so a typed payload would only add a +conversion detour. + +This deliberately deviates from the typed philosophy of Zaino's driven ports +(zingolabs/zaino#1402: "source traits return typed `Block`, not `Vec`"). +The two boundaries face opposite directions: a driven port feeds Zaino's own +core, which needs structured data, while the driving port feeds external +consumers that own their parsing. + +## Considered Options + +We rejected typing the payloads with `zaino-primitives`, because consumers +would deserialize Zaino's types only to convert them into their own; the +identifier types are the useful subset, and the port takes exactly those. We +rejected typing the payloads with `zcash_primitives`, because Zaino's engines +would then take on librustzcash version-lockstep with the wallet stack — the +coupling the Z3 stack keeps working to shed. diff --git a/docs/driving-port/0003-snapshot-pinning-is-unconditional.md b/docs/driving-port/0003-snapshot-pinning-is-unconditional.md new file mode 100644 index 000000000..549ded0ae --- /dev/null +++ b/docs/driving-port/0003-snapshot-pinning-is-unconditional.md @@ -0,0 +1,38 @@ +# Snapshot pinning is unconditional + +The driving port's snapshot carries its strongest guarantee: every read +through a snapshot observes the chain as of the pinned tip, and that data +stays readable while any clone of the snapshot lives — across reorgs, +without exception. We decided the guarantee is unconditional. An engine +that cannot retain the pinned view for as long as any clone lives is not +an implementation of the port. + +We chose this because the alternative — letting a snapshot lapse — would +export the engine's storage policy into every driver's sync loop. A lapse +answer turns "retake and resync" into a code path every driver must write +and test for an event that a correctly built engine never produces, and it +demotes the port's strongest promise to a conditional one. The burden +lands instead on the adapter, which must retain the pinned view — +reference-count it, copy it, or hold the branch it was answered from — +while any clone lives. Hash-keyed reads over append-only finalised state +pin for free; the tip-relative surfaces (unspent outpoints at an address, +outpoint spend status) are what the adapter must keep answerable as of the +pinned tip after the chain moves on. The mock exemplifies the shape: +snapshots hold an `Arc` of the chain as of their creation, and a scripted +mutation swaps the `Arc` while live snapshots keep the old one. + +One reorg race remains, and it lives outside the snapshot: taking a +snapshot can race the engine's view swap, and that failure crosses the +port as a transient backend failure. Reads through a snapshot never race +a reorg. + +## Considered Options + +We rejected pinning-with-lapse (a `Lapsed` answer any pinned read may +return once the engine has discarded the pinned view), because it forces +every driver to carry a lapse branch, weakens the guarantee that gives +snapshots their meaning, and shapes the port around one engine's retention +limits. We rejected leaving the guarantee unstated, because an +implementation would then satisfy the letter of the trait while serving +mixed-chain reads during reorgs — the exact torn-read hazard the snapshot +exists to exclude. diff --git a/docs/driving-port/CONTEXT.md b/docs/driving-port/CONTEXT.md new file mode 100644 index 000000000..a9126cb0e --- /dev/null +++ b/docs/driving-port/CONTEXT.md @@ -0,0 +1,106 @@ +# The Zallet driving port + +This context covers the contract through which Zallet (and later other +consumers) drives Zaino. The language below was settled in a design review +held in the Zallet repository, where the consumer side of this contract lives. + +## Language + +**Mempool**: +The set of transactions awaiting mining. It is chain-validity-dependent but +chain-structure-independent: consensus binds its transactions to chain state +(anchors, nullifiers, prevouts, expiry, branch id), yet its representation and +service abstraction stand apart from chain state. A mempool view is coherent +only relative to the chain tip it was validated against. + +**Driving port**: +The contract through which consumers drive Zaino. It spans chain reads, +mempool streaming, and transaction broadcast. Zallet is its first driver, and +zainod-for-lightclients is its expected second; zingolib is not a consumer. +_Avoid_: primary port, wallet API, indexer interface + +**Snapshot**: +A pinned view of the best chain, taken through the port. Every read through +a snapshot observes the chain as of the tip it was pinned to, and that data +stays readable while any clone of the snapshot lives — across reorgs. The +guarantee is unconditional: an engine must retain the pinned view for as +long as any clone lives, and an engine that cannot is not an implementation +of the port. The pinned tip is a property of the snapshot, not a query +against the engine. +_Avoid_: view (Zallet's consumer-side name for the same idea) + +**Conformance kit**: +The executable half of the port contract: a suite of invariant cases that +every implementation of the port must pass, shipped with the port itself. +An engine adapter passing the kit is what qualifies it as an implementation. + +**Block locator**: +A descending-by-height sample of blocks a driver believes are on the chain, +offered to the port to locate the fork point between the driver's view and +the best chain. A block's identity is its hash; its height is position, so +presence is always judged by hash. + +**Reported upgrades**: +The network-upgrade schedule as the validator reports it, ascending by +activation height, each upgrade active or pending relative to the current +tip. The port passes the validator's schedule through — activation heights +come from the validator, never from constants of the port's own — and +drivers feed it into their node-compatibility checks. + +**Broadcast**: +Submission of a transaction to the network through the port — there is no +side channel. Acceptance returns the txid and means the engine admitted the +transaction to its mempool, where the mempool stream makes it observable. +Rejection is a domain answer (malformed bytes, or a validation rejection +with the engine's reason), distinct from a backend failure. + +**Tip event**: +The port's explicit signal that the best chain moved, carrying the new tip. +A fresh subscription delivers the current tip first; events may coalesce +under load, but the latest tip always arrives; a reorg is a tip change like +any other, and the new tip may sit at or below the old height. This is how +drivers learn of new blocks — never through a side effect of another stream. + +**Fork point**: +The block a driver's view and the pinned best chain last share: the locator +entry whose block sits highest on the pinned chain, judged by hash and never +by the heights the locator claims. Everything the driver holds above the +fork point is not on the pinned chain. Views sharing no block have no fork +point. + +**Treestate**: +The note commitment tree state of every shielded pool — Sapling, Orchard, +and Ironwood — as of one block of the pinned chain. Every in-view height has +a treestate; what varies per pool is whether its frontier is present, and an +absent frontier means an empty tree, never an error (zcash/zallet#455). + +**Transaction status**: +Where a snapshot places a transaction: mined in the pinned best chain, +orphaned onto a non-best branch, or unknown. The status speaks only of chain +state — mempool presence is deliberately not a status, because the mempool +stands apart from chain state and is observed through its own stream. + +**Spend status**: +Whether the pinned view considers a transparent output spent. Spentness is +authoritative — it comes from the engine's UTXO set — while naming the +spending transaction may need a per-outpoint spend index the engine does not +maintain; an engine that knows the output is spent but not by whom says so +explicitly, and the driver retries rather than concluding the output is +unspent (ZcashFoundation/zebra#10806). An outpoint no in-view transaction +created has no spend status at all. + +**Mempool view**: +The driver-side accumulation of one mempool subscription: the engine's +mempool as of the tagged tip, plus possibly transactions the engine has +since evicted. The stream signals arrivals only, so the view is a +superset of the engine's mempool, trued up by resubscribing; mempool +presence is a hint, never authoritative. + +**Transient failure**: +A backend failure likely to resolve on retry, such as taking a snapshot +while the engine's view is mid-swap. Reads through a snapshot never race a +reorg — pinning is unconditional — so the reorg window touches only the +unpinned surface. Every backend failure crossing the port is classified as +transient or fatal, and drivers decide retry from that classification +alone. A domain rejection is an answer, not a failure, and is never +transient.