test: migrate miden integration fixtures to protocol/client 0.15.x#5
Draft
revitteth wants to merge 15 commits into
Draft
test: migrate miden integration fixtures to protocol/client 0.15.x#5revitteth wants to merge 15 commits into
revitteth wants to merge 15 commits into
Conversation
…#1882) Bump miden-client from exp-agglayer-v0.2 tag to agglayer-integration-tests branch, along with all base crates to 0.14.0-alpha (miden-protocol, miden-standards, miden-agglayer, miden-tx) and rusqlite to 0.37. API migrations: - AuthFalcon512Rpo → AuthSingleSig with AuthScheme::Falcon512Rpo - ClaimNoteParams → ClaimNoteStorage with typed ProofData/LeafData - create_agglayer_faucet_builder (private) → create_agglayer_faucet (public) - create_bridge_account gains bridge_admin_id + ger_manager_id params - create_claim_note takes (storage, faucet_id, sender_id, rng) - build_p2id_recipient → P2idNoteStorage::into_recipient - create_p2id_note → P2idNote::create - Note::inputs() → Note::storage(), NoteInputs → NoteStorage - Keystore::add_key now async and requires account_id - ClientBuilder::filesystem_keystore now returns Result Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch from miden-node (exp-agglayer-v0.2, 0.13) to testing-node-builder from miden-client (agglayer-integration-tests branch, 0.14.0-alpha). This is the only binary that supports agglayer genesis accounts (bridge_admin, ger_manager, bridge, faucet). Includes sed patch to bind RPC to 0.0.0.0 for Docker port forwarding (upstream testing-node-builder hardcodes 127.0.0.1). Tested: agglayer_bridge_in_out e2e test passes against Dockerized node. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The "geth" engine type is no longer supported by kurtosis-cdk; switch to "ethereum-package" which deploys geth+lighthouse. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The 0.14 miden-client faucet auto-consumes CLAIM notes via the NTX builder, so the proxy no longer needs to execute Phase 2 (faucet consumption + P2ID minting). This removes ~220 lines of Phase 2 logic including P2ID script registration, block-wait polling, foreign account setup, and output note tracking. Also fixes list-unclaimed-notes.sh Docker path (/data → /app/data) to match the testing-node-builder's actual data directory. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add submit_ger_to_miden(): when aggoracle sends insertGlobalExitRoot, create and submit an UpdateGerNote so the bridge account stores the GER on-chain. Without this, the faucet's CLAIM note script fails verify_leaf_bridge (GER lookup returns nothing). - Fix double-scaling bug in CLAIM amount: raw wei bytes now flow through ClaimSubmissionData.amount_wei instead of being pre-scaled to u64. Scaling (÷ 10^10) happens once in submit_claim_to_miden() via EthAmount::scale_to_token_amount(), matching the integration test pattern. Previously the amount was scaled in the RPC handler then scaled again, producing 0. - Fix list-notes.sh: correct DB path (/app/data/ not /data/) and replace non-existent 'inputs' column with 'target_account_id'. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix block number parsing, address encoding, and block hash generation
to enable reliable bridge synchronization and end-to-end deposit flow.
- Parse decimal block numbers from bridge-service ("12" not just "0x12")
- Fix Miden-to-Eth address encoding: 4-byte padding to match bridge
contract's prefix(8)/suffix(8) split at offset 4
- Simplify block hashes to keccak("miden_block_evm_<N>") — pure function
of block number, eliminates false reorg detection from creation order
- Re-tagged pending events use deterministic hashes directly
Verified full flow: L1 deposit → bridge sync → CLAIM → faucet → P2ID → claim
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…service The zkevm-bridge-service's reorg detection computes block hashes via Go's ethclient HeaderByNumber + header.Hash() = keccak256(rlp(header)), NOT from the JSON response's hash field. Our synthetic hash was always different, causing false reorgs on every sync cycle. Fix block_state.rs to build a real alloy_consensus::Header and compute keccak256(rlp(header)) — matching Go's computation exactly. Fix ClaimEvent to use the v2 topic hash (uint256 globalIndex) instead of the old v1 hash (uint32 leafType), and encode the actual globalIndex in the event data so each claim gets a unique primary key in the bridge DB. Add stress test scripts and report documenting 10-deposit full pipeline: L1 deposit → bridge sync → ready_for_claim → CLAIM note → P2ID → claim. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously, eth_getTransactionReceipt returned status=0x1 (success) immediately after the CLAIM note transaction was submitted to Miden. However, a CLAIM note submission always "succeeds" as a valid Miden transaction — regardless of whether the agglayer faucet actually consumes the note. The faucet only consumes a CLAIM note when: 1. The Global Exit Root (GER) is registered in bridge storage 2. The SMT proof verifies against the registered GER 3. The faucet is online and processing notes If any condition fails, the note sits unconsumed forever. The bridge would see a success receipt, mark the deposit as "claimed", but no P2ID note would be minted — deposited funds lost with no error signal. The fix introduces a new TxStatus::AwaitingConsumption state: - After CLAIM note submission: status = AwaitingConsumption (not Confirmed) - Bridge polls getTransactionReceipt: proxy syncs with Miden node, checks if the output note state is "Consumed" - If consumed: transition to Confirmed, synthesize ClaimEvent, return success receipt (status=0x1) - If not consumed yet: return null (tx pending from bridge's POV) - If not consumed within 50 blocks: return reverted receipt (status=0x0) This only affects claimAsset transactions. GER injections and other tx types still confirm immediately. Tested E2E: deposit → bridge sync → claimAsset → receipt pending → CLAIM note consumed at next block → receipt success → bridge confirms. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the aggkit-proxy (vibecoded) implementation with miden-agglayer in the Kurtosis test infrastructure. This includes: - Deploy PostgreSQL for miden-agglayer's persistent store - Run database migration (001_initial.sql) inline via plan.exec - Pass L1 RPC URL and contract addresses (bridge, GER, rollup manager) to the proxy via environment variables - Use miden-agglayer CLI args (--chain-id, --miden-node, --port) - Enable aggsender component alongside aggoracle in aggkit config - Add AggSender configuration for L2→L1 certificate submission - Point AggLayer's full-node-rpc to Miden proxy - Improve miden-node logging (ntx_builder debug) - Patch miden-node ntx-builder for CLAIM failure logging Tested: Kurtosis deploy succeeds, L1 deposit → L2 claim works end-to-end, GER injection active, ClaimEvent logs emitted. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Kurtosis package (kurtosis/miden-cdk/, see commit e7b4660) already deploys gateway-fm/miden-agglayer as the production miden-proxy service. The lib + main + bin code in src/ was the prior vibecoded implementation and has been dead code since the supplant. Worse, it didn't compile against current miden-client (API drift across `Felt::as_int`, `OutputNote::Full`, `create_agglayer_faucet` signature, etc.) so `cargo check` was failing on this branch. What's removed: - src/main.rs and the entire `miden-rpc-proxy` binary. - src/{address_mapper,agglayer_faucet,asset_mapping,block_state, claim_tracker,client,config,decode,error,log_synthesis,receipt, storage,types}.rs — the vibecoded modules. Equivalents (in some cases improved) live in gateway-fm/miden-agglayer. - src/bin/{verify_notes,claim_note}.rs — operator helpers that also drifted off miden-client v0.14.4 (use deprecated `Felt::as_int`). If we want them back, port them against the current API. - examples/gen_signed_tx.rs — same drift class. What's kept: - tests/phase{1,2,3}.rs + tests/common — miden-client integration tests. Independent of the proxy. Currently have their own API drift vs v0.14.4 (separate fix); not blocking the supplant. - kurtosis/miden-cdk/ — Starlark package that deploys miden-agglayer + aggkit + bridge-service + agglayer + miden-node. - scripts/ — operator/test harnesses that drive the kurtosis stack. Cargo.toml dropped to a thin shell: - Pin miden-client to `tag = "v0.14.4"` (was `branch = "agglayer-integration-tests"`) to match gateway-fm/miden-agglayer's pin. - Loose carets on miden-protocol/standards/tx so cargo can resolve the exact subversion v0.14.4 transitively requires. - Drop jsonrpsee/tower/alloy/etc — none of the surviving code uses them. Follow-up (separate work): - Port tests/phase{1,2,3}.rs to miden-client v0.14.4 API (`build_consume_notes` now takes Vec<Note>, etc.). - Run scripts/e2e-kurtosis.sh against the patched miden-agglayer:rd-862 image to validate the full e2e flow. - Once gateway-fm/miden-agglayer's RD-862 fix and agglayer/aggkit's upstream PR land, swap the kurtosis package's image refs to public tagged releases. Refs: - gateway-fm/miden-agglayer#37 - agglayer/aggkit#1597 - Linear RD-862 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…faucet_registry migration End-to-end validation of the RD-862 fix on the kurtosis stack required three small alignments: 1. `params.yaml`: set `aggkit_image: "miden-agglayer/aggkit:rd-862"` so the kurtosis-cdk parser threads our locally-built patched aggkit through to `bridge_service.star::_deploy_aggkit`. Without this the default `ghcr.io/agglayer/aggkit:0.9.0-rc2` was picked up and the `UseUpdateExitRoot` config was silently ignored. 2. `bridge_service.star`: add `UseUpdateExitRoot = true` to the `[AggOracle.EVMSender]` section of the inlined aggkit config template. This is the flag the patched aggkit reads to switch `EVMChainGERSender` from `direct_injection` to `direct_update_exit_root` mode. 3. `miden_services.star`: append `migrations/002_faucet_registry.sql` (CREATE TABLE faucet_registry + idx_faucet_origin) to the inlined migration SQL. Previously only `001_initial.sql` was inlined, so miden-proxy startup failed with `db error ERROR: relation "faucet_registry" does not exist`. Added a comment marking the inline as a snapshot and pointing at the long-term fix (upload the migrations dir as a files-artifact). Validation: kurtosis enclave deployed cleanly with all services RUNNING. aggkit logs show `EVMChainGERSender initialized in direct_update_exit_root mode` followed by `update-exit-root GER transaction submitted` — i.e. the two-root calldata path. miden-proxy logs show 0 `L1 exit roots don't match injected GER` warnings (orphan count = 0). The L1→Miden test deposit completes; both injected GERs committed without decomposition. Refs: - gateway-fm/miden-agglayer#37 - agglayer/aggkit#1597 - Linear RD-862 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
miden-agglayer reads the L1 GER contract address from env var GER_L1_ADDRESS (src/main.rs:80, clap attr `env = "GER_L1_ADDRESS"`). Kurtosis package was passing it as L1_GER_ADDRESS, so the var was silently dropped — the proxy logged: WARN L1 RPC URL or GER contract address missing; L1InfoTreeIndexer disabled. Without it, GER orphan resolution falls back to the racing view-call path in service_send_raw_txn.rs and may produce orphan GERs under deposit load. With the rename, the indexer activates on bring-up and we see e.g. INFO L1InfoTreeIndexer: indexed exit-root pair, event: "UpdateL1InfoTree", mainnet: 2554ea8a..., rollup: 0000..., combined: 716b1dba..., block: 147 INFO L1InfoTreeIndexer batch processed, from: 147, to: 147, head: 147, log_count: 1, indexed: 1 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bumps miden-protocol/standards/tx to 0.15 and miden-client to the upstream protocol-0.15 branch (PR 0xMiden/rust-sdk#2224), in lockstep with gateway-fm/miden-agglayer. Ports phase1-3 fixtures. Auth switched from RPO Falcon512 to Poseidon2 (AuthSingleSig with AuthScheme::Falcon512Poseidon2). Keystore add_key is now async and takes the account id, moved after build. AccountStorageMode folded into AccountType Private/Public (account kind now derives from components). BasicFungibleFaucet becomes FungibleFaucet::builder (name/symbol/decimals/max_supply AssetAmount). build_consume_notes takes Vec Note via InputNoteRecord try_into. cargo check --all-targets green (tests are live and compiled only). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ation The phase3 pytest suite runs against the miden-agglayer proxy (PROXY_URL/MIDEN_NODE_URL/BRIDGE_SERVICE_URL env-driven). Six tests carried stale expectations from the old gutted miden-rpc-proxy and now pass against the real proxy: 80 passed, 7 skipped. Gas (test_tc_3_2): the miden-agglayer proxy deliberately returns eth_gasPrice=0x3b9aca00 (1 gwei, for EVM-tooling compatibility) and eth_estimateGas=0x0 (Miden has no gas), per service.rs. Updated assertions to match (was expecting 0 and 21000). Errors (test_tc_3_1, test_tc_3_12): catch by JSON-RPC .code instead of the RPCError class. pytest loads conftest as the top-level conftest module, so the fixture raises conftest.MidenRPCError while 'from tests.conftest import RPCError' is a different class object that pytest.raises would not match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a standalone zkevm-autoclaimer to the miden-cdk kurtosis package so L2->L1 exits are claimed on L1 automatically (sponsoring claimAsset), gated behind a `miden.deploy_autoclaimer` flag (default off). - bridge_service.star: new _deploy_autoclaimer — funds the claim sponsor on L1 (it pays gas for claimAsset), renders an autoclaim config (SourceNetworkID = l2_network_id so we only sponsor our own rollup's exits; L2RPC = L1 RPC since the claim lands on L1; PolygonBridgeAddress = L1 bridge), reuses the funded claimsponsor keystore, and runs /app/zkevm-autoclaimer from the patched bridge-service image. deploy() gains a deploy_autoclaimer arg. - main.star: MIDEN_DEFAULTS.deploy_autoclaimer = False; threaded into bridge_service.deploy(). Files reformatted by `kurtosis lint --format` (black). Validated independently against an upstream kurtosis-cdk enclave: the autoclaimer found settled L2->L1 exits and sponsored claimAsset on L1 for a backlog of deposits (network 1).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Migrate aggkit-proxy miden test fixtures to protocol/client 0.15.x
Draft — in lockstep with gateway-fm/miden-agglayer#80. Pins track the unreleased
miden-client0.15 branch (PR 0xMiden/rust-sdk#2224); re-pin to thev0.15.xtags when they ship.What this does
miden-client0.15 branch.tests/phase1-3.rs) ported to 0.15: auth RPO→Poseidon2 (AuthSingleSig+AuthScheme::Falcon512Poseidon2); asynckeystore.add_key(key, account_id)(moved post-build);AccountStorageModefolded intoAccountType(kind now derives from components);BasicFungibleFaucet→FungibleFaucet::builder(AssetAmountmax-supply +TokenName);build_consume_notestakesVec<Note>viaInputNoteRecord::try_into; fallibleFelt::new. Compiles clean (cargo check --all-targets); these are live tests (need a node).phase3suite aligned with the miden-agglayer proxy and now 80 passed / 0 failed / 7 skipped against it (env-driven:PROXY_URL/MIDEN_NODE_URL/BRIDGE_SERVICE_URL):eth_gasPrice=0x3b9aca00(1 gwei, EVM-tooling compat) andeth_estimateGas=0x0(Miden has no gas) — assertions updated (were expecting 0 / 21000, from the old gutted proxy)..codeinstead of theRPCErrorclass (pytest loadsconftestas the top-level module, sofrom tests.conftest import RPCErroris a different class object thatpytest.raiseswouldn't match).Questions for the Miden team (0.15 integration)
The open upstream questions surfaced by this migration are protocol/node-level, not proxy-specific, so they are documented in full on the companion PR gateway-fm/miden-agglayer#80 ("Questions for the Miden team"). Summary:
AssetCallbackFlagin vault-key lookups → silent bridge-out tx-loss (3 sites; patch attached on #80).MIDEN_NETWORK_IDtopology pinning (77 in MASM vs L1RollupManagernet 1; no single id satisfies both stock ends).Known open
phase2python tests are pre-existing dead scaffolding — they import aconftestAPI (ClaimAssetParams,GlobalIndex,ProxyRPCClient,CLAIM_NOTE_FELT_SIZE, …) thatconftest.pynever defined, so they fail at collection and have never run. Out of scope here (not touched by this migration); the CLAIM encoding they were meant to unit-test is covered end-to-end by the miden-agglayer bridge-in e2e.