Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
146 commits
Select commit Hold shift + click to select a range
52889a0
Add zaino-sync crate: type-level axis markers and trait hierarchy
nachog00 Jul 1, 2026
cd04f0f
Replace erased layer with partially-erased IndexPipeline
nachog00 Jul 1, 2026
3943f3a
Add primitives module with IndexId and BlockHeight newtypes
nachog00 Jul 1, 2026
2a453c2
Add PhaseIndex primitive, document BlockHeight duplication
nachog00 Jul 1, 2026
0a2e2f9
Add bridge impls and DAG builder with topo sort
nachog00 Jul 1, 2026
c828d16
Add engine sync loop, in-memory backend, and end-to-end tests
nachog00 Jul 1, 2026
6ac766f
Add IndexSet API with BridgeDispatch blanket impl
nachog00 Jul 1, 2026
8e22a67
Decouple index BlockContext from set-wide Ctx via ProvideContext
nachog00 Jul 1, 2026
a5d3d11
Narrow per-index contexts with ProvideContext owned projection
nachog00 Jul 2, 2026
f4e583a
Remove SourceRequirements and ContextRequirements
nachog00 Jul 2, 2026
c3a7f0b
Add Scheduler with per-edge firing rules and batch tracking
nachog00 Jul 2, 2026
51d5ba7
Add phantom type state machine to scheduler batch lifecycle
nachog00 Jul 2, 2026
0ee6471
Split IndexPipeline into extract_one / merge / persist
nachog00 Jul 2, 2026
888f5da
DRY bridges into single LocalBridge<I, S> with MergeStrategy trait
nachog00 Jul 2, 2026
0250e2e
Reshape engine to scheduler-driven three-phase dispatch
nachog00 Jul 2, 2026
5f4916d
Derive descriptor from type-level markers via provided method
nachog00 Jul 2, 2026
54efa84
Use domain types for ValueIndex delta, not raw bytes
nachog00 Jul 2, 2026
5f7c62a
Separate persistence from merge via Schema<M> + Encode traits
nachog00 Jul 3, 2026
eda1b79
Use domain newtypes for toy index Schema key/value types
nachog00 Jul 3, 2026
bacd985
Add block availability gating and unified Task enum to scheduler
nachog00 Jul 3, 2026
d35449a
Add BlockBuffer and BlockOffset, type scheduler/engine offsets
nachog00 Jul 3, 2026
339fa68
Reshape engine to buffer-driven supply/demand loop
nachog00 Jul 3, 2026
165a316
Add eviction test and test-only engine accessors
nachog00 Jul 3, 2026
1e6e4bc
Extract sync_streaming as core loop, sync_range delegates to it
nachog00 Jul 3, 2026
dc45add
Extract dispatch_tasks helper from sync_streaming loop
nachog00 Jul 3, 2026
5116cfc
Add async sync_channel with channel-driven supply/demand loop
nachog00 Jul 3, 2026
91dd260
Change pipeline storage from Box to Arc
nachog00 Jul 3, 2026
1274dcc
Add parallel extraction dispatch via spawn_blocking
nachog00 Jul 3, 2026
27d3892
Decompose dispatch into flush/extract/report helpers
nachog00 Jul 3, 2026
7fe6733
Update Cargo.lock for tokio dependency
nachog00 Jul 3, 2026
77a665f
Replace spawn_blocking with rayon for parallel extraction
nachog00 Jul 3, 2026
c7c16bb
Add CumulativeBridge for SelfCumulative index scope
nachog00 Jul 3, 2026
1edd0ad
Add backend reader, atomic batch commit, and watermark
nachog00 Jul 3, 2026
940d425
Add tracing instrumentation, bench harness, and SlowBackend
nachog00 Jul 3, 2026
4eb6ed1
Add debug_assert invariant guards across engine, scheduler, and buffer
nachog00 Jul 3, 2026
aac17b6
Add zaino-primitives crate: Height, BlockHash, TransactionHash
nachog00 Jul 11, 2026
894c17b
Add zaino-source crate: driven port traits for validator queries
nachog00 Jul 11, 2026
29b1cc6
Extract generic QueryError<E>, use thiserror across zaino-source
nachog00 Jul 11, 2026
9fb3323
Add thiserror to zaino-primitives, derive HeightOverflow
nachog00 Jul 11, 2026
3d106c0
Add domain newtypes and aliases to zaino-primitives
nachog00 Jul 11, 2026
880c01c
Add full query trait surface to zaino-source (14 traits)
nachog00 Jul 11, 2026
4be1eb3
Add MockChain adapter for testing (feature-gated)
nachog00 Jul 11, 2026
9ab1790
Add source integration test: MockChain → engine end-to-end
nachog00 Jul 11, 2026
0296db1
Add multi-batch backpressure test through source channel
nachog00 Jul 11, 2026
0caef3d
Add zaino-rpc crate: shared JSON-RPC 2.0 client
nachog00 Jul 11, 2026
346d317
Add zaino-source-zebra-rpc: Zebra JSON-RPC adapter
nachog00 Jul 11, 2026
d3fc575
Modularize zaino-rpc: envelope, retry, client with 11 unit tests
nachog00 Jul 11, 2026
527f507
Add resilience wrapper and structured TransportFailure classification
nachog00 Jul 11, 2026
8c4e935
Rename error types for clarity
nachog00 Jul 11, 2026
2871d6d
Rename TransportError → FetchError, field kind → mode
nachog00 Jul 11, 2026
d7e399f
Add resilience integration tests with mock failure injection
nachog00 Jul 11, 2026
9a2893c
Add Block/Transaction domain types, rename GetBlockBytes → GetBlock
nachog00 Jul 11, 2026
94a67dd
Zebra adapter returns typed Block, sync tests use domain types
nachog00 Jul 11, 2026
8adc43d
Add TODO for upstream zebra PR: CompactDifficulty::to_bits()
nachog00 Jul 11, 2026
1315dec
Add smoke example: live Zebra RPC → typed Block
nachog00 Jul 11, 2026
38b839d
Add HeadersIndex (L,A): first real Zcash index with domain types
nachog00 Jul 11, 2026
1db8838
Decouple sync engine from persistence encoding
nachog00 Jul 11, 2026
e51d8d0
Add offline block parity test for Zebra RPC adapter
nachog00 Jul 11, 2026
e3793f9
Update all Schema impls with encode/decode methods
nachog00 Jul 11, 2026
8fff4ed
Add sync-headers demo: live Zebra → HeadersIndex end-to-end
nachog00 Jul 11, 2026
82824a7
Concurrent provisioner with FuturesOrdered sliding window
nachog00 Jul 11, 2026
6047b5d
Parameterize batch_size, in-cluster benchmark results
nachog00 Jul 11, 2026
757c5f6
Add zaino-persistence crate: driven port for index storage
nachog00 Jul 11, 2026
6b47c90
Add InMemoryBackend + SlowBackend to zaino-persistence
nachog00 Jul 11, 2026
193be06
Wire zaino-sync to use zaino-persistence for backend traits
nachog00 Jul 11, 2026
7adf8e3
Refine persistence: Namespace, RawKey/RawValue, per-method errors
nachog00 Jul 11, 2026
72f6115
Namespace newtype, per-method errors, IndexId→Namespace conversion
nachog00 Jul 11, 2026
0458295
Add zaino-backend-lmdb: LMDB persistence adapter
nachog00 Jul 11, 2026
fc6649b
Demo supports LMDB backend via ZAINO_DB_PATH env var
nachog00 Jul 11, 2026
9ecbc47
Add bench.sh: one-command in-cluster benchmarking
nachog00 Jul 11, 2026
fb8f96f
Structured benchmark summary + Job manifest
nachog00 Jul 11, 2026
2abea9c
Split: zaino-indexes crate + sync-bench in live-tests
nachog00 Jul 11, 2026
d0f6715
bench.sh: stream output + save results locally
nachog00 Jul 11, 2026
1ef454a
Add Containerfile + deploy.sh for k8s Job benchmarks
nachog00 Jul 11, 2026
05bd703
Containerfile + deploy.sh fixes, deploy.sh save format fix
nachog00 Jul 11, 2026
8353f94
Fix k8s Job deployment: correct image name, root SSH
nachog00 Jul 11, 2026
17a0e8a
WIP: extract zaino-convert-zebra, scaffold readstate adapter
nachog00 Jul 11, 2026
4df8257
Add zaino-source-zebra-readstate + shared zaino-convert-zebra
nachog00 Jul 11, 2026
3bfead0
Benchmark supports ReadState provisioner via ZEBRA_STATE_DIR
nachog00 Jul 11, 2026
c185c8e
Add tracing spans to adapter methods (feature-gated)
nachog00 Jul 11, 2026
1ef2310
POC: ReadState header-only path — 335k blocks/s
nachog00 Jul 11, 2026
b5e189b
Add TransparentSpendsIndex + headers_and_spends set
nachog00 Jul 11, 2026
ffc0529
Full chain headers+spends sync kicked: 120 GB LMDB on NVMe
nachog00 Jul 11, 2026
96d1904
Fix Containerfile (add clang), full-chain Job with ReadState
nachog00 Jul 11, 2026
ba21f59
Add tracing subscriber to sync-bench (opt-in via RUST_LOG)
nachog00 Jul 11, 2026
c6b2bd0
Update Cargo.lock for tracing deps
nachog00 Jul 12, 2026
f88483e
Remove accidental empty file
nachog00 Jul 13, 2026
569616c
Delete .claude/pr-ordering.md
nachog00 Jul 13, 2026
d940fa7
Restore old crates to origin/dev (undo accidental global sed changes)
nachog00 Jul 13, 2026
3b169b3
Revert "Restore old crates to origin/dev (undo accidental global sed …
nachog00 Jul 13, 2026
d436af6
Undo accidental sed changes to old crates (6 files)
nachog00 Jul 13, 2026
73b7edc
Add 6 indexes + current_zaino set (8 indexes total)
nachog00 Jul 14, 2026
aa6bbb9
Wire current_zaino index set (8 indexes) into benchmark
nachog00 Jul 14, 2026
f01c797
Update deploy.sh + bench.sh for ReadState + current_zaino
nachog00 Jul 14, 2026
0f5e293
Enable tracing feature on zaino-sync in benchmark
nachog00 Jul 14, 2026
dc864db
Add tracing events to scheduler and engine
nachog00 Jul 14, 2026
fd5d9dc
deploy.sh: pass RUST_LOG to Job when set
nachog00 Jul 14, 2026
2fc8589
Scheduler: emit all available blocks for BlockLocal indexes
nachog00 Jul 14, 2026
23b365a
Demote noisy tracing events to trace, keep milestones at info
nachog00 Jul 14, 2026
a122a76
JSON log output via ZAINO_LOG_JSON=1 (for Loki/Grafana)
nachog00 Jul 14, 2026
df6f184
Parallelize merge+persist across indexes via rayon
nachog00 Jul 14, 2026
97866ad
Add Python/Jupyter and bench log paths to gitignore
nachog00 Jul 14, 2026
e476f59
Add Display derives for descriptor enums and IndexSet::describe()
nachog00 Jul 14, 2026
bc29cfc
Propagate tracing span context into rayon parallel closures
nachog00 Jul 14, 2026
c960986
Convert bench output to tracing and add OTLP export to Tempo
nachog00 Jul 14, 2026
8529a15
Rewrite deploy script with commit-hash job naming and OTLP support
nachog00 Jul 14, 2026
0c09d60
Add bench reports directory with full-chain sequential baseline
nachog00 Jul 14, 2026
095591c
Extract merge_persist_parallel helper, fix sequential merge bug
nachog00 Jul 14, 2026
d5ca0c5
Add batch-emission full chain sync report
nachog00 Jul 14, 2026
652ef5b
Rename binary to sync-bench, fix tracing target for bench events
nachog00 Jul 14, 2026
27df9a5
Add SYNC_FROM env var for targeting specific chain ranges
nachog00 Jul 14, 2026
b34fb9f
Fix deploy script volume resolution fallback
nachog00 Jul 14, 2026
140c000
Import sync engine design docs from zaino-design
nachog00 Jul 14, 2026
56c0a58
Add crate architecture diagram with port-adapter relationships
nachog00 Jul 14, 2026
3559273
Add provision-bench binary for isolating provisioner throughput
nachog00 Jul 15, 2026
cafbc63
Include provision-bench binary in container image
nachog00 Jul 15, 2026
c7e8e67
Wire ReadRequest::CompactBlock from zebra fork into provision-bench
nachog00 Jul 15, 2026
8230e9a
Use optimal ReadRequest per provision-bench mode
nachog00 Jul 15, 2026
01eefab
Add CompactBlock domain types, GetCompactBlock trait, and compact syn…
nachog00 Jul 15, 2026
6721b6c
Add From<&Block> for CompactBlock, impl GetCompactBlock for RPC adapter
nachog00 Jul 15, 2026
63e8237
Split PreIndexCompactBlock (source) from CompactBlock (serving)
nachog00 Jul 15, 2026
b1652a5
Wire SelfCumulative x Append bridge, add SC x Fold tests
nachog00 Jul 15, 2026
4cc9273
Document parallel monoidal reduce as a future optimization
nachog00 Jul 15, 2026
47a4d19
Document SC load_state scan-everything limitation
nachog00 Jul 15, 2026
01644d5
Add Ironwood variant to ShieldedPool
nachog00 Jul 21, 2026
6bb080e
Scaffold the inner driving surface (core, service, wallet, fit)
nachog00 Jul 21, 2026
51cab87
Add scriptable MockIndexerService behind the testing feature
nachog00 Jul 21, 2026
6e195ee
Doc: runtime composition — block-store spine + typed-index layer
nachog00 Jul 22, 2026
35aa5e1
Doc: resolve Q1 (snapshot handle) and Q2 (retain side branches)
nachog00 Jul 22, 2026
a333e1e
Doc: resolve Q4 — whole CompactBlocks + auxiliary indexes only
nachog00 Jul 22, 2026
a58afe9
Doc: resolve Q5 (adopt algorithm, re-skin 3 seams) + full blocks (pas…
nachog00 Jul 22, 2026
99a9200
Doc: invert to FS index engine (compact-block-as-index) + NFS reorg w…
nachog00 Jul 22, 2026
d3844e2
Scaffold zaino-fs + zaino-nfs component seams; add CompactBlockRead
nachog00 Jul 22, 2026
ea4f9c5
Scaffold zaino-runtime: compose seam + loop-wiring sketch
nachog00 Jul 22, 2026
3f929eb
Per-operation errors in zaino-fs; NFS reads are infallible
nachog00 Jul 22, 2026
45a4fc1
Encode NFS readiness in the type (NfsView::Ready/Syncing)
nachog00 Jul 22, 2026
c242206
Component mocks + runtime routing tests (compose seam runs)
nachog00 Jul 22, 2026
aca9731
Doc: driving-port user stories (first-class, traced to capabilities)
nachog00 Jul 22, 2026
ef0e97b
US-1.3: address unspent-set merge (FS index ∪ NFS re-derivation)
nachog00 Jul 22, 2026
debf162
Refactor zaino-runtime: extract resolve policy, clean modules
nachog00 Jul 22, 2026
4156bd3
feat(runtime): full-block/raw-tx passthrough as a generic seam (US-1.7)
nachog00 Jul 23, 2026
a8e4879
refactor(runtime): fold passthrough into the validator Source
nachog00 Jul 23, 2026
ab74949
refactor(runtime): shape the passthrough port in domain terms
nachog00 Jul 23, 2026
e2d6305
test(runtime): anti-misrouting scenario matrix
nachog00 Jul 23, 2026
c47e9e3
refactor(fs): decompose FinalisedState into spine + addon index traits
nachog00 Jul 24, 2026
48745b0
refactor(nfs): decompose NfsSnapshot into spine + recent-facet traits
nachog00 Jul 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,12 @@ build/
.failed-tests
**/proptest-regressions/**
/result
bench-results/

# Python / Jupyter
__pycache__/
*.pyc
.ipynb_checkpoints/

# Notebook logs (captured from k8s)
live-tests/sync-bench/k8s/logs/
Loading