release/7.14.2: XRP THORChain memo, EVM depositWithExpiry, message-signing, Zcash tests#195
Open
BitHighlander wants to merge 21 commits into
Open
release/7.14.2: XRP THORChain memo, EVM depositWithExpiry, message-signing, Zcash tests#195BitHighlander wants to merge 21 commits into
BitHighlander wants to merge 21 commits into
Conversation
Same firmware as the standalone UDP kkemu binary, loaded in-process via
ctypes. Lets python-keepkey exercise the firmware contract that the
keepkey-vault FFI path imposes — most importantly, the caller-driven
polling model (no daemon thread to call kkemu_poll for you).
- keepkeylib/transport_dylib.py: DylibState (process-wide singleton over
ctypes-loaded libkkemu) + DylibTransport (one per iface 0/1).
Pumps kkemu_poll on every read/write so the firmware actually makes
forward progress on caller turns.
- tests/config.py: KK_TRANSPORT=dylib KK_DYLIB=/path/to/libkkemu.dylib
routes the same fixture to the FFI transport instead of UDP.
- tests/test_dylib_confirm_flow.py: regression for the confirm-flow
contract (Initialize, WipeDevice, LoadDevice, GetAddress). Skipped
unless KK_TRANSPORT=dylib so it won't break the default UDP run.
Reproduces the keepkey-vault hang deterministically: Initialize round-
trips fine, wipe_device hangs because confirm_helper busy-loops on a
ButtonAck the dylib silently consumed but never delivered. Caught in
~10s, no electrobun / bun stack required.
Run:
cd tests && KK_TRANSPORT=dylib KK_DYLIB=.../libkkemu.dylib \
PYTHONPATH=..:../keepkeylib python3 -m pytest \
test_dylib_confirm_flow.py -v
…ntics The existing test_dylib_confirm_flow covers the caller-driven polling contract — Initialize / Wipe / LoadDevice / GetAddress — but never asks the firmware for a layout. Two changes that just landed in the firmware emulator runtime PR (BitHighlander/keepkey-firmware#217) need functional coverage that confirm-flow doesn't provide: 1. RINGBUF_CAPACITY in lib/emulator/ringbuf.h was bumped from 32 to 128. DebugLinkState's 2048-byte `layout` plus the rest of the message serializes to ~44 HID reports through the output ring; the previous capacity left effective room for 31 reports, so screenshot capture truncated mid-layout (msg_debug_write ignores emulatorSocketWrite's 0-on-full return). 2. fsm_msgDebugLinkGetState in lib/firmware/fsm_msg_debug.h now does a single display_refresh() instead of force_animation_start() + animate(). The old form overwrote static layouts with stale animation frames or no-ops depending on queue state, so screenshots captured something different from what the user was seeing. Both fixes are functionally invisible to the existing test suite. Without these tests, regressing either change ships green. This commit adds: - tests/test_dylib_screenshot.py — four tests: * test_layout_round_trip_fits_through_ring (RINGBUF_CAPACITY) * test_layout_repeated_reads_no_truncation (RINGBUF_CAPACITY) * test_layout_stable_across_idle_reads (canvas semantics) * test_layout_features_dont_corrupt_capture (iface separation) Constructs a fresh KeepKeyDebuglinkClient against the dylib singleton WITHOUT going through common.KeepKeyTest.setUp — that fixture wipes the device on every test and exercises the confirm-flow path that test_dylib_confirm_flow is itself a pending regression for. Reading a layout doesn't require any of that; we just init and ask DebugLink for the home-screen capture. - tests/config.py — explicit-transport precedence fix: Previously HID/WebUSB were always autodetected first. With a real KeepKey plugged in, KK_TRANSPORT=dylib was silently overridden — the dylib regression suite would either route to hardware or crash on hid.pyx. Now the explicit env var (KK_TRANSPORT=dylib) skips hardware enumeration entirely, the dylib path runs as requested, and the default (no env var set) falls back to the existing UDP behavior. Verified locally: cmake -DKK_EMULATOR=1 -DKK_BUILD_DYLIB=1 -DKK_DEBUG_LINK=ON \ -DCMAKE_POLICY_VERSION_MINIMUM=3.5 -B build-emu . cmake --build build-emu --target kkemulator_dylib KK_TRANSPORT=dylib KK_DYLIB=build-emu/lib/libkkemu.dylib \ PYTHONPATH=keepkeylib:. python -m pytest tests/test_dylib_screenshot.py ======================== 4 passed in 0.36s ======================== Out of scope: SignTx + other multi-step flows that go through confirm_helper. They share the same hang as test_dylib_confirm_flow's test_load_device_with_auto_confirm — copying the pattern would just produce a second red regression for the same underlying firmware bug, not new coverage. Once the confirm-flow regression goes green, signtx expansion is a follow-up.
…ANSPORT, split confirm-flow setUp Three findings from review of PR #14: #1 (High) test_dylib_confirm_flow used common.KeepKeyTest.setUp which calls wipe_device() — the same path the file's pending regression is for. Hangs in setUp can't be classified by xfail or interrupted by pytest-timeout, so test_features_round_trip ("just Initialize") was actually wipe + Initialize. Refactored to construct KeepKeyDebuglinkClient directly in setUp (matching test_dylib_screenshot's pattern), moved wipe + load_device into the one pending test. Tried the reviewer-suggested @pytest.mark.xfail(strict=True) + @pytest.mark.timeout combo. pytest-timeout (both signal and thread methods) cannot interrupt the C-level kkemu_poll busy-loop — the hang locks up the entire test runner instead of failing the test. Switched to @unittest.skip with explicit rationale documenting exactly that, plus the promotion path: when firmware lands the confirm fix, drop the skip; if a future change makes kkemu_poll GIL-friendly, switch back to xfail+timeout. #2 (Medium) tests/config.py treated any non-empty KK_TRANSPORT as "explicit" and skipped HID/WebUSB autodetect, but only "dylib" was actually handled. A typo like KK_TRANSPORT=dyllib silently fell through to UDP with hardware disabled. Now scoped to a _KNOWN_TRANSPORTS set; unsupported values raise at config import, surfacing typos at test collection time. Verified end-to-end: `KK_TRANSPORT=dyllib pytest test_msg_signtx.py` now errors on collection with the typo'd value in the message. #3 (Medium/Low) DylibTransport.ready_to_read appended raw frame bytes to read_buffer but DylibTransport._pump_one stripped the leading '?' HID marker first. Inconsistent stripping corrupts multi-frame message reassembly: _read_headers can scan a stray '?' from one chunk into the middle of contiguous payload bytes from another, decoding the wrong message-type / length. Centralised the read+strip into a private _poll_and_stash helper shared by both ready_to_read (no sleep) and _pump_one (sleeps on miss). Now the buffer always contains continuation+payload bytes only; the leading '?' is stripped at the single point of stashing. Trailing HID padding zeros from short messages are still tolerated by _read_headers' magic-character search. Verified locally: KK_TRANSPORT=dylib KK_DYLIB=build-emu/lib/libkkemu.dylib \ pytest tests/test_dylib_screenshot.py tests/test_dylib_confirm_flow.py ================== 5 passed, 1 skipped in 0.15s ==================
Wires the ZIP-32 §6.1 seed fingerprint binding into the python-keepkey
client to mirror the firmware-side validation.
device-protocol submodule
- URL: keepkey/device-protocol -> BitHighlander/device-protocol
(zcash work pins to fork master while seed_fingerprint sits in
long-term review for upstream; revert when upstream merges.)
- pin: d0b8d80 -> 4337c452 (BitHighlander/master with PR keepkey#27 merged).
- messages_zcash_pb2.py regenerated via docker_build_pb.sh
(kktech/firmware:v8 → libprotoc 3.5.1, the canonical toolchain).
Selective regen — other pb2 files are intentionally NOT
regenerated because they currently include content from
BitHighlander/device-protocol open PRs (#18 SolanaTokenInfo,
#19 TRON clear-signing, #20 TON clear-signing, keepkey#21
EthereumTxMetadata). Until those merge, regenerating them
against current master would back out work that the existing
python-keepkey client relies on.
keepkeylib/zcash.py (new)
calculate_seed_fingerprint(seed) -> 32 bytes
Pure-Python helper. BLAKE2b-256("Zcash_HD_Seed_FP",
I2LEBSP_8(len) || seed). Matches the firmware C
implementation byte-for-byte and the keystone3-firmware
reference vector
seed = 000102...1f
fp = deff604c246710f7176dead02aa746f2fd8d5389f7072556dcb555fdbe5e3ae3
keepkeylib/client.py
zcash_display_address — add expected_seed_fingerprint kwarg
zcash_sign_pczt — add expected_seed_fingerprint kwarg
Both pass through unchanged when the kwarg is None
(backward compatible).
tests/test_msg_zcash_seed_fingerprint.py (new)
Pure-Python helper:
- reference vector (Keystone3 cross-check)
- rejects all-zero, all-0xFF, short, long
Device-backed:
- GetOrchardFVK returns non-empty seed_fingerprint
- fingerprint stable across accounts (bound to seed, not account)
- DisplayAddress: matching expected_seed_fingerprint succeeds,
response carries seed_fingerprint
- DisplayAddress: wrong expected_seed_fingerprint rejected
- DisplayAddress: omitting expected_seed_fingerprint still works
- SignPCZT: wrong expected_seed_fingerprint rejected before
any signing crypto runs
Addresses review of PR #15. Test structure Helper tests (no device) move to a dedicated module: tests/test_zcash_seed_fingerprint_helper.py This module deliberately does NOT import common, transport, or any protobuf bindings, so it runs on a stock dev box: pytest tests/test_zcash_seed_fingerprint_helper.py The previous file inherited common.KeepKeyTest, whose setUp wipes the device — pytest -k 'helper' was never actually offline. Client wrapper coverage Device-backed tests now go through the public client helpers (self.client.zcash_display_address(... expected_seed_fingerprint=...) and self.client.zcash_sign_pczt(... expected_seed_fingerprint=...)) rather than building raw protobuf messages with self.client.call(). Confirms the kwarg pass-through end-to-end. New test test_device_fingerprint_matches_python_helper: cross-checks the device-computed fingerprint against the python-keepkey helper for the same seed (all-allallall mnemonic, empty passphrase). Ties the firmware C, python-keepkey helper, and ZIP-32 §6.1 reference vector to the same byte-for-byte output.
feat(zcash): seed_fingerprint client + tests
… ≤ 7.14.0)
Pairs the device, signs a 1550-byte EIP-1559 transaction with the
all-all-all test mnemonic, and asserts that ECDSA recovery against the
canonical type-2 pre-image yields the device's own address.
Catches a firmware/ethereum.c ordering bug present in 7.x.0 .. 7.14.0
where the empty access-list byte (0xC0) — which closes the EIP-1559 RLP
body and must be the last byte fed to keccak before signing — was being
hashed inside ethereum_signing_init() right after the initial 1024-byte
data chunk, BEFORE the host had a chance to send the remaining
EthereumTxAck frames. For any tx whose data exceeded the single-chunk
threshold, the resulting pre-image was:
keccak( ...header...
|| data_len_prefix
|| data[0..1024]
|| 0xC0 (bug: should be after ALL data)
|| data[1024..end] )
The signature was mathematically valid for that mangled hash so RPCs
accepted the broadcast, but the recovered signer was a wrong-but-
deterministic address. The mempool dropped the tx because the recovered
"from" had no balance / wrong nonce. Production symptom: every Uniswap
Universal Router swap, Permit2 batch, and large multicall hung at
"Confirm in wallet."
Single-chunk transactions (<= 1024 bytes) escaped the bug only by
accident — the misplaced 0xC0 happened to land at the end anyway.
Recovery-based assertion (eth-keys, eth-utils.keccak) — works on any
seed, no golden vectors to capture, the test asserts the actual
invariant: "signature recovers to the signer." Fails on broken
firmware, passes on 7.14.1+.
CI: eth-keys added to the existing pip install line; ships a pure-Python
keccak via eth-utils so no native deps are required.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
requires_message("EthereumTxAck") sends an empty EthereumTxAck as a
discovery probe. The firmware (correctly) rejects that with
Failure_UnexpectedMessage because we're not mid-sign, which skips the
test before the actual assertion runs.
requires_firmware("7.2.1") is sufficient — EthereumTxAck has been part
of the protocol since EIP-1559 support landed in 7.2.1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
eth-utils ships keccak via the eth-hash adapter, which auto-selects between pycryptodome and pysha3 at import time. Without either backend installed, importing keccak raises: ImportError: None of these hashing backends are installed: ['pycryptodome', 'pysha3']. The new EIP-1559 chunked-data regression test imports keccak from eth_utils to build the canonical type-2 pre-image, so it failed at import rather than at the recovery assertion. Adding pycryptodome to the existing pip-install line fixes it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
KeepKeyTest overrides unittest's assertEqual with a 2-arg version (common.py:104) that doesn't accept the optional msg parameter — passing one raises: TypeError: KeepKeyTest.assertEqual() takes 3 positional arguments but 4 were given Print the regression diagnostic before asserting instead. Pytest captures stdout on failure, so the divergence (expected vs recovered, canonical hash, sig values) still surfaces in the failure report. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Upstreaming this test as a permanent regression guard rather than a one-shot bug catcher. Bumping requires_firmware from 7.2.1 (the version where EIP-1559 support originally landed) to 7.14.1 (the first version where the access-list ordering bug is fixed) so CI on broken builds skips this test instead of flagging a known-broken state as a new regression. The header comment already documents the affected range (7.x.0 .. 7.14.0) and the fix landing in 7.14.1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…gnition - Bump device-protocol submodule to 8f80bcd (adds memo field to RippleSignTx) - Update messages_ripple_pb2.py with memo field (field 7, optional string) compatible with protobuf==3.20.3 (old-format serialized_pb descriptor) - Add test_sign_with_thorchain_memo in test_msg_ripple_sign_tx.py: verifies serialized XRPL tx ends with canonical Memos array binary (F9 EA 7D <len> <memo> E1 F1), requires firmware 7.14.2 - Add test_msg_ethereum_thorchain_deposit.py: covers legacy deposit() 0x1fece7b4 selector, new depositWithExpiry() 0x44bc937b selector (requires 7.14.2, no AdvancedMode), and verifies non-THORChain addresses are still blocked without AdvancedMode
feat(7.14.2): XRP THORChain memo + EVM depositWithExpiry tests
* feat(hive): add Hive blockchain support - messages_hive_pb2.py — generated from messages-hive.proto (IDs 1600-1603) - hive.py — get_public_key / sign_tx client helpers - mapping.py — register HiveGetPublicKey, HivePublicKey, HiveSignTx, HiveSignedTx wire IDs - client.py — hive_get_public_key / hive_sign_tx methods on ProtocolMixin * feat(hive): add HiveGetPublicKeys, HiveSignAccountCreate, HiveSignAccountUpdate - messages_hive_pb2.py: regenerated from updated proto; now includes all 10 message types (HiveGetPublicKey/Keys, HivePublicKey/Keys, HiveSignTx/ed, HiveSignAccountCreate/ed, HiveSignAccountUpdate/ed). Added role field to HiveGetPublicKey. - mapping.py: register wire IDs 1604-1609 for the six new message types. - hive.py: add get_public_keys(), sign_account_create(), sign_account_update() helpers. get_public_key() gains optional role parameter. - client.py: add hive_get_public_keys(), hive_sign_account_create(), hive_sign_account_update() mixin methods with @expect decorators.
…format Regenerated using protoc from kktech/firmware:v15 (protobuf 3.17.3). The previous version used the builder API (protobuf 3.20+) which is incompatible with the 3.20.3 Python runtime pinned in CI.
Test bugs fixed (mirrors BitHighlander/keepkey-firmware alpha CI fixes): - ETH THORChain deposit: assertIn(sig_v, [27,28]) -> [37,38] (EIP-155 chain_id=1) - XRP no-memo check: b'\xf9' -> b'\xf9\xea' (0xF9 appears in DER sigs naturally) - Zcash FVK validation: skipTest until feature lands in firmware
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings BitHighlander/python-keepkey fork master into keepkey/python-keepkey master, covering 7.14.1 + 7.14.2 releases and all test/protocol additions needed for the 7.15.0 firmware release.
7.14.2 changes
RippleSignTx.memofield 7)depositWithExpiryrecognition in transaction display7.14.1 changes
pycryptodomeinstall in CI foreth_utils.keccakbackendZcash / emulator
seed_fingerprintclient binding + full test coverageDylibTransport— in-processlibkkemufor screenshot regression testsKK_TRANSPORT, strip-? consistencyTest plan
python -m pytest tests/passes against a 7.14.x device