smite: mine consensus-valid transactions that violate mempool policy#137
smite: mine consensus-valid transactions that violate mempool policy#137NishantBansal2003 wants to merge 1 commit into
Conversation
ekzyis
left a comment
There was a problem hiding this comment.
Only looked at the first commit (5644b83).
I wonder if we also need to unlock the utxos in some cases, like when we fail to broadcast for some reason. Or is that not needed because of snapshot fuzzing, so for every run, bitcoind is also in the same initial state with the utxos unlocked?
But then I wonder why we even need to lock utxos: when would we create two funding transactions with overlapping inputs? Aren't we only creating one funding transaction per run?
My guess is that a crash can still happens when broadcasting the transaction on chain, because it might try to broadcast the same transaction that was mined before which would make the We could ignore such error and add a special case in the minimizer to remove a edit: Nevermind, I just saw the second commit that avoid broadcasting the same transaction twice. Which makes a lot of sense, and since is very lightweight operation we maybe don't need to minimize it?
I think smite will be able to open multiple channels against the target. Right now (in current master) Smite cannot open a channel, since it misses the funding and broadcast operations of the open channel flow in the generators. |
Yep, I think
We have |
Should we care whether the transaction follows the bitcoin core relayable Policy? I think that, if the transaction is consensus-valid, Smite should be able to include it in a block, which would help us cover more edge cases. |
I think any operation that makes redundant or unnecessary RPC calls, should be minimized, since they are time-consuming and actually hurt fuzzing effectiveness
Some reasons I thought against it are:
But I don’t have a strong opinion, so I’m fine if we want to relax the |
e5dc4e7 to
62fbf6c
Compare
It could get mined with out-of-band payments, or because it's the miner's own tx etc. They are non-standard according to Bitcoin Core, but they can get mined. I agree with @erickcestari, I think never broadcasting txs below the dust limit would unnecessarily limit fuzzing effectiveness, or at least I’m not convinced yet why we shouldn’t broadcast them. |
I think I got what @NishantBansal2003 said. If a lightning node is building transactions (funding transactions, commitment transactions, HTLC transactions) that aren't accepted by mempool (dust output, nonstandard script, below static minrelay) this is already a bug that should be reported and the fuzzer should crash with help of oracles. |
morehouse
left a comment
There was a problem hiding this comment.
The first three commits LGTM. I'd be happy to merge them today as a separate PR.
I'm less sure about the last commit. I agree that we don't want to change the default mempool policy used by the target node, but I think it could be quite profitable to preserve the ability to mine non-standard transactions that may trigger edge cases on the target node.
One way to reconcile the two is to use separate bitcoinds for smite and the target. Of course then the bitcoind nodes communicating with each other would add a new source of latency.
Alternatively, we could keep the shared bitcoind and then if sendrawtransaction fails due to some mempool policy rejection, we could fallback to mining the transaction directly with generateblock instead.
The fuzzer may build transactions that are consensus-valid but violate mempool policy, such as those below the minimum relay feerate. sendrawtransaction refuses these, so check testmempoolaccept first and mine the transaction directly with generateblock when it does not meet mempool policy. Signed-off-by: Nishant Bansal <nishant.bansal.282003@gmail.com>
62fbf6c to
c193003
Compare
|
Updated the last commit to mine the tx directly if it violates mempool policy while still being consensus-valid in: c193003 |
morehouse
left a comment
There was a problem hiding this comment.
My main concern is that now the BroadcastTransaction operation may silently mine the transaction as well instead of waiting for MineBlocks to do that.
WDYT about an approach like this?
- If the broadcast fails due to non-standardness, the executor saves the raw tx in it's own "private mempool"
- On the next
MineBlocksoperation, if the private mempool contains anything, the executor appends its private mempool tobitcoind's mempool fromgetrawmempooland mines the combined mempool in the first block withgenerateblock, then mines the remaining blocks normally.
|
|
||
| // If the transaction follows mempool policy, broadcast it normally. | ||
| // Otherwise, mine it directly into a block. | ||
| if !self.accepted_by_mempool(&signed_tx.hex) { |
There was a problem hiding this comment.
Rather than doing another RPC call, can we just check the output from sendrawtransaction to save some overhead?
| // If the transaction follows mempool policy, broadcast it normally. | ||
| // Otherwise, mine it directly into a block. | ||
| if !self.accepted_by_mempool(&signed_tx.hex) { | ||
| self.generate_block_with_tx(&signed_tx.hex); |
There was a problem hiding this comment.
So now our BroadcastTransaction operation may also mine blocks. Maybe this is okay as a practicality, though it would be nice to only do the mining in the actual MineBlocks operation...
One unexpected example: previously if we broadcast the same transaction twice it was a no-op. Now if we do that the transaction gets automatically mined on the second broadcast.
| .run() | ||
| .arg("testmempoolaccept") | ||
| .arg(&txs) | ||
| .arg("0") |
There was a problem hiding this comment.
| .arg("0") | |
| // Disable the high-feerate cap and accept any fee rate for broadcast. | |
| .arg("0") |
| assert!( | ||
| self.get_transaction_confirmations(txid) > 0, | ||
| "generateblock did not mine transaction {txid}" | ||
| ); |
There was a problem hiding this comment.
Is this check necessary -- is there any situation where generateblock succeeds but fails to mine our transaction?
If not, let's drop this check to save an RPC call.
There were a couple of crashes while I was running the funding-flow generator, where we were getting panics in smite because of the restrictive assert statements we have set. These commits fix those issues in smite.
So now we can’t create two transactions with overlapping inputs. We return early when signing or broadcasting a transaction if it has already been broadcast and confirmed, since the wallet can’t sign a transaction whose inputs have already been spent. There were also a couple of crashes around broadcasting, where either
funding_satoshisitself was below the dust limit, or the feerate used to calculate the transaction fee was too low compared tobitcoinddefault policy. So now we reject those inputs as well.These cases do not really contribute to fuzzing, because in practice, if
bitcoinddefault policy rejects them, Lightning nodes should also reject thoseopen_channelmessages before sendingaccept_channel. But let’s say they are not rejected, then I think those cases can be caught once we add theopen_channelandaccept_channeloracle. So in either case, we don’t need to relaxbitcoinddefault policy to exercise the full funding flow