wallet: derivehdkey RPC to get xpub at arbitrary path

doc/multisig-tutorial.md

@@ -9,18 +9,16 @@ This tutorial uses [jq](https://github.com/stedolan/jq) JSON processor to proces
 Before starting this tutorial, start the bitcoin node on the signet network.
 
 ```bash
-./build/bin/bitcoind -signet -daemon
+./build/bin/bitcoin node -signet -daemon
 ```
 
-This tutorial also uses the default WPKH derivation path to get the xpubs and does not conform to [BIP 45](https://github.com/bitcoin/bips/blob/master/bip-0045.mediawiki) or [BIP 87](https://github.com/bitcoin/bips/blob/master/bip-0087.mediawiki).
-
-At the time of writing, there is no way to extract a specific path from wallets in Bitcoin Core. For this, an external signer/xpub can be used.
+This tutorial also uses the default PKH derivation path to get the xpubs and does not conform to [BIP 45](https://github.com/bitcoin/bips/blob/master/bip-0045.mediawiki) or [BIP 87](https://github.com/bitcoin/bips/blob/master/bip-0087.mediawiki).
 
 ## 1.1 Basic Multisig Workflow
 
 ### 1.1 Create the Descriptor Wallets
 
-For a 2-of-3 multisig, create 3 descriptor wallets. It is important that they are of the descriptor type in order to retrieve the wallet descriptors. These wallets contain HD seed and private keys, which will be used to sign the PSBTs and derive the xpub.
+For a 2-of-3 multisig, create 3 wallets. These wallets contain HD seed and private keys, which will be used to sign the PSBTs and derive the xpub.
 
 These three wallets should not be used directly for privacy reasons (public key reuse). They should only be used to sign transactions for the (watch-only) multisig wallet.
 
@@ -31,16 +29,7 @@ do
 done
 ```
 
-Extract the xpub of each wallet. To do this, the `listdescriptors` RPC is used. By default, Bitcoin Core single-sig wallets are created using path `m/44'/1'/0'` for PKH, `m/84'/1'/0'` for WPKH, `m/49'/1'/0'` for P2WPKH-nested-in-P2SH and `m/86'/1'/0'` for P2TR based accounts. Each of them uses the chain 0 for external addresses and chain 1 for internal ones, as shown in the example below.
-
-```
-wpkh([1004658e/84'/1'/0']tpubDCBEcmVKbfC9KfdydyLbJ2gfNL88grZu1XcWSW9ytTM6fitvaRmVyr8Ddf7SjZ2ZfMx9RicjYAXhuh3fmLiVLPodPEqnQQURUfrBKiiVZc8/0/*)#g8l47ngv
-
-wpkh([1004658e/84'/1'/0']tpubDCBEcmVKbfC9KfdydyLbJ2gfNL88grZu1XcWSW9ytTM6fitvaRmVyr8Ddf7SjZ2ZfMx9RicjYAXhuh3fmLiVLPodPEqnQQURUfrBKiiVZc8/1/*)#en65rxc5
-```
-
-The suffix (after #) is the checksum. Descriptors can optionally be suffixed with a checksum to protect against typos or copy-paste errors.
-All RPCs in Bitcoin Core will include the checksum in their output.
+Extract the xpub of each wallet. To do this, the `derivehdkey` RPC is used.
 
 Note that previously at least two descriptors were usually used, one for external derivation paths and one for internal ones. Since https://github.com/bitcoin/bitcoin/pull/22838 this redundancy has been eliminated by a multipath descriptor with <code><0;1></code> at the [BIP-44](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki#change) change level expanding to external and internal descriptors when imported.
 
@@ -49,66 +38,63 @@ declare -A xpubs
 
 for ((n=1;n<=3;n++))
 do
- xpubs["xpub_${n}"]=$(./build/bin/bitcoin rpc -signet -rpcwallet="participant_${n}" listdescriptors | jq '.descriptors | [.[] | select(.desc | startswith("wpkh") and contains("/0/*") )][0] | .desc' | grep -Po '(?<=\().*(?=\))' | sed 's /0/\* /<0;1>/* ')
+ xpubs["xpub_${n}"]=$(./build/bin/bitcoin rpc -signet -rpcwallet="participant_${n}" derivehdkey "m/44h/1h/0h" | jq -r '.origin + .xpub')
 done
 ```
 
-`jq` is used to extract the xpub from the `wpkh` descriptor.
-
-The following command can be used to verify if the xpub was generated correctly.
+The following command can be used to verify if the xpubs were obtained successfully:
 
 ```bash
 for x in "${!xpubs[@]}"; do printf "[%s]=%s\n" "$x" "${xpubs[$x]}" ; done
 ```
 
-As previously mentioned, this step extracts the `m/84'/1'/0'` account instead of the path defined in [BIP 45](https://github.com/bitcoin/bips/blob/master/bip-0045.mediawiki) or [BIP 87](https://github.com/bitcoin/bips/blob/master/bip-0087.mediawiki), since there is no way to extract a specific path in Bitcoin Core at the time of writing.
+As previously mentioned, this step extracts the `m/44'/1'/0'` account instead of the path defined in [BIP 45](https://github.com/bitcoin/bips/blob/master/bip-0045.mediawiki) or [BIP 87](https://github.com/bitcoin/bips/blob/master/bip-0087.mediawiki), because the wallet currently can't sign for a derivation path that's not used in one of its descriptors.
 
 ### 1.2 Define the Multisig Descriptor
 
-Define the multisig descriptor, add the checksum and then, wrap it in a JSON array.
+Define the multisig descriptors.
+
+All RPCs in Bitcoin Core will include the checksum in their output.
 
 ```bash
-desc="wsh(sortedmulti(2,${xpubs["xpub_1"]},${xpubs["xpub_2"]},${xpubs["xpub_3"]}))"
+desc="wsh(sortedmulti(2,${xpubs["xpub_1"]}/<0;1>/*,${xpubs["xpub_2"]}/<0;1>/*,${xpubs["xpub_3"]}/<0;1>/*))"
 
-checksum=$(./build/bin/bitcoin rpc -signet getdescriptorinfo $desc | jq -r '.checksum')
+desc_sum=$(./build/bin/bitcoin rpc -signet getdescriptorinfo $desc | jq -r '.checksum')
 
-multisig_desc="[{\"desc\": \"${desc}#${checksum}\", \"active\": true, \"timestamp\": \"now\"}]"
+multisig_desc="[{\"desc\": \"$desc#$desc_sum\", \"active\": true, \"timestamp\": \"now\"}]"
 ```
 
 `desc` specifies the output type (`wsh`, in this case) and the xpubs involved. It also uses BIP 67 (`sortedmulti`), so the wallet can be recreated without worrying about the order of xpubs. Conceptually, descriptors describe a list of scriptPubKey (along with information for spending from it) [[source](https://github.com/bitcoin/bitcoin/issues/21199#issuecomment-780772418)].
 
-After creating the descriptor, it is necessary to add the checksum, which is required by the `importdescriptors` RPC.
+The checksum for a descriptor without one can be computed using the `getdescriptorinfo` RPC. The response has a `checksum` field, which is appended to the descriptor after `#` to protect against typos or copy-paste errors.
 
-The checksum for a descriptor without one can be computed using the `getdescriptorinfo` RPC. The response has the `checksum` field, which is the checksum for the input descriptor, append "#" and this checksum to the input descriptor.
-
-There are other fields that can be added to the descriptor:
+There are other fields that can be added to the descriptors:
 
 * `active`: Sets the descriptor to be the active one for the corresponding output type (`wsh`, in this case).
-* `internal`: Indicates whether matching outputs should be treated as something other than incoming payments (e.g. change).
 * `timestamp`: Sets the time from which to start rescanning the blockchain for the descriptor, in UNIX epoch time.
 
-Note: when a multipath descriptor is imported, it is expanded into two descriptors which are imported separately, with the second implicitly used for internal (change) addresses.
-
 Documentation for these and other parameters can be found by typing `./build/bin/bitcoin rpc -signet help importdescriptors`.
 
-`multisig_desc` wraps the descriptor in a JSON array and will be used to create the multisig wallet.
+`multisig_desc` concatenates the descriptor in a JSON array and then it will be used to create the multisig wallet.
 
 ### 1.3 Create the Multisig Wallet
 
 To create the multisig wallet, first create an empty one (no keys, HD seed and private keys disabled).
 
 Then import the descriptor created in the previous step using the `importdescriptors` RPC.
 
-After that, `getwalletinfo` can be used to check if the wallet was created successfully.
+After that, `listdescriptors` can be used to check if the wallet was created successfully.
 
 ```bash
-./build/bin/bitcoin rpc -signet createwallet "multisig_wallet_01" disable_private_keys=true blank=true
+./build/bin/bitcoin rpc -signet -named createwallet wallet_name="multisig_wallet_01" disable_private_keys=true blank=true
 
 ./build/bin/bitcoin rpc -signet -rpcwallet="multisig_wallet_01" importdescriptors "$multisig_desc"
 
-./build/bin/bitcoin rpc -signet -rpcwallet="multisig_wallet_01" getwalletinfo
+./build/bin/bitcoin rpc -signet -rpcwallet="multisig_wallet_01" listdescriptors
 ```
 
+The `<0;1>` notation in `desc` caused the creation of two descriptors. One uses the chain 0 for external addresses, and the other uses chain 1 for internal ones (change).
+
 Once the wallets have already been created and this tutorial needs to be repeated or resumed, it is not necessary to recreate them, just load them with the command below:
 
 ```bash
@@ -199,7 +185,7 @@ psbt_2=$(./build/bin/bitcoin rpc -signet -rpcwallet="participant_2" walletproces
 The PSBT, if signed separately by the co-signers, must be combined into one transaction before being finalized. This is done by `combinepsbt` RPC.
 
 ```bash
-combined_psbt=$(./build/bin/bitcoin rpc -signet combinepsbt "[$psbt_1, $psbt_2]")
+combined_psbt=$(./build/bin/bitcoin rpc -signet combinepsbt txs="[$psbt_1, $psbt_2]")
 ```
 
 There is an RPC called `joinpsbts`, but it has a different purpose than `combinepsbt`. `joinpsbts` joins the inputs from multiple distinct PSBTs into one PSBT.

doc/release-notes-32784.md

@@ -0,0 +1,9 @@
+Wallet
+------
+
+- A new `derivehdkey` RPC is available to obtain an xpub or xprv for a
+  derivation path with at least one hardened step from an HD key known to the
+  wallet. This can be used to coordinate a multisig setup, where each signer
+  shares an xpub using a
+  different derivation path than the default single-signature descriptors. The
+  example in `doc/multisig-tutorial.md` is updated to use this RPC. (#32784)

src/key.cpp

@@ -16,6 +16,8 @@
 #include <secp256k1_recovery.h>
 #include <secp256k1_schnorrsig.h>
 
+#include <algorithm>
+
 static secp256k1_context* secp256k1_context_sign = nullptr;
 
 /** These functions are taken from the libsecp256k1 distribution and are very ugly. */
@@ -365,6 +367,19 @@ bool CExtKey::Derive(CExtKey &out, unsigned int _nChild) const {
     return key.Derive(out.key, out.chaincode, _nChild, chaincode);
 }
 
+std::optional<std::pair<CExtKey, KeyOriginInfo>> DeriveExtKey(const CExtKey& ext_key, const std::vector<uint32_t>& path)
+{
+    CExtKey descendant = ext_key;
+    KeyOriginInfo origin;
+    const CKeyID id = ext_key.key.GetPubKey().GetID();
+    std::copy(id.begin(), id.begin() + sizeof(origin.fingerprint), origin.fingerprint);
+    origin.path = path;
+    for (uint32_t i : path) {
+        if (!descendant.Derive(descendant, i)) return std::nullopt;
+    }
+    return std::make_pair(descendant, origin);
+}
+
 void CExtKey::SetSeed(std::span<const std::byte> seed)
 {
     static const unsigned char hashkey[] = {'B','i','t','c','o','i','n',' ','s','e','e','d'};

src/key.h

@@ -8,11 +8,14 @@
 #define BITCOIN_KEY_H
 
 #include <pubkey.h>
+#include <script/keyorigin.h>
 #include <serialize.h>
 #include <support/allocators/secure.h>
 #include <uint256.h>
 
+#include <optional>
 #include <stdexcept>
+#include <utility>
 #include <vector>
 
 struct secp256k1_context_struct;
@@ -255,6 +258,12 @@ struct CExtKey {
     void SetSeed(std::span<const std::byte> seed);
 };
 
+//! Get extended key and origin info for a given path
+//! @param[in] ext_key The extended private key to derive from
+//! @param[in] path The BIP 32 path
+//! @return the resulting extended private key and origin info
+std::optional<std::pair<CExtKey, KeyOriginInfo>> DeriveExtKey(const CExtKey& ext_key, const std::vector<uint32_t>& path);
+
 /** KeyPair
  *
  *  Wraps a `secp256k1_keypair` type, an opaque data structure for holding a secret and public key.

src/rpc/client.cpp

@@ -338,6 +338,8 @@ static const CRPCConvertParam vRPCConvertParams[] =
     { "gethdkeys", 0, "active_only" },
     { "gethdkeys", 0, "options" },
     { "gethdkeys", 0, "private" },
+    { "derivehdkey", 1, "options" },
+    { "derivehdkey", 1, "private" },
     { "createwalletdescriptor", 1, "options" },
     { "createwalletdescriptor", 1, "internal" },
     // Echo with conversion (For testing only)

src/rpc/util.cpp

@@ -20,6 +20,7 @@
 #include <tinyformat.h>
 #include <uint256.h>
 #include <univalue.h>
+#include <util/bip32.h>
 #include <util/check.h>
 #include <util/result.h>
 #include <util/strencodings.h>
@@ -1374,6 +1375,15 @@ std::vector<CScript> EvalDescriptorStringOrObject(const UniValue& scanobject, Fl
     return ret;
 }
 
+std::vector<uint32_t> ParsePathBIP32(const std::string& path)
+{
+    std::vector<uint32_t> out;
+    if (!ParseHDKeypath(path, out)) {
+        throw JSONRPCError(RPC_INVALID_PARAMETER, "Invalid BIP32 keypath");
+    }
+    return out;
+}
+
 /** Convert a vector of bilingual strings to a UniValue::VARR containing their original untranslated values. */
 [[nodiscard]] static UniValue BilingualStringsToUniValue(const std::vector<bilingual_str>& bilingual_strings)
 {

src/rpc/util.h

@@ -156,6 +156,9 @@ std::pair<int64_t, int64_t> ParseDescriptorRange(const UniValue& value);
 /** Evaluate a descriptor given as a string, or as a {"desc":...,"range":...} object, with default range of 1000. */
 std::vector<CScript> EvalDescriptorStringOrObject(const UniValue& scanobject, FlatSigningProvider& provider, bool expand_priv = false);
 
+//! Parse BIP32 path
+std::vector<uint32_t> ParsePathBIP32(const std::string& path);
+
 /**
  * Serializing JSON objects depends on the outer type. Only arrays and
  * dictionaries can be nested in json. The top-level outer type is "NONE".

src/script/descriptor.cpp

@@ -432,10 +432,7 @@ class BIP32PubkeyProvider final : public PubkeyProvider
     bool IsHardened() const
     {
         if (m_derive == DeriveType::HARDENED_RANGED) return true;
-        for (auto entry : m_path) {
-            if (entry >> 31) return true;
-        }
-        return false;
+        return HasHardenedDerivation(m_path);
     }
 
 public:

src/test/bip32_tests.cpp

@@ -9,8 +9,10 @@
 #include <key_io.h>
 #include <streams.h>
 #include <test/util/setup_common.h>
+#include <util/bip32.h>
 #include <util/strencodings.h>
 
+#include <algorithm>
 #include <string>
 #include <vector>
 
@@ -184,6 +186,47 @@ BOOST_AUTO_TEST_CASE(bip32_test5) {
     }
 }
 
+BOOST_AUTO_TEST_CASE(bip32_derive_ext_key)
+{
+    const CExtKey master{DecodeExtKey(test1.vDerive[0].prv)};
+    const std::vector<uint32_t> path{test1.vDerive[0].nChild, test1.vDerive[1].nChild};
+    const auto derived{DeriveExtKey(master, path)};
+    BOOST_REQUIRE(derived);
+    BOOST_CHECK(EncodeExtKey(derived->first) == test1.vDerive[2].prv);
+
+    KeyOriginInfo expected_origin;
+    const CKeyID id{master.key.GetPubKey().GetID()};
+    std::copy(id.begin(), id.begin() + sizeof(expected_origin.fingerprint), expected_origin.fingerprint);
+    expected_origin.path = path;
+    BOOST_CHECK(derived->second == expected_origin);
+
+    const auto root{DeriveExtKey(master, {})};
+    BOOST_REQUIRE(root);
+    BOOST_CHECK(root->first == master);
+    expected_origin.path.clear();
+    BOOST_CHECK(root->second == expected_origin);
+
+    CExtKey max_depth{master};
+    for (auto i{0}; i++ < 255;) {
+        CExtKey next_key;
+        BOOST_REQUIRE(max_depth.Derive(next_key, 0));
+        max_depth = next_key;
+    }
+    BOOST_CHECK(!DeriveExtKey(max_depth, {0}));
+}
+
+BOOST_AUTO_TEST_CASE(bip32_has_hardened_derivation)
+{
+    const std::vector<uint32_t> empty;
+    const std::vector<uint32_t> unhardened{0, 1, 2};
+    const std::vector<uint32_t> hardened{0x80000000U};
+    const std::vector<uint32_t> mixed{0, 1 | 0x80000000U, 2};
+    BOOST_CHECK(!HasHardenedDerivation(empty));
+    BOOST_CHECK(!HasHardenedDerivation(unhardened));
+    BOOST_CHECK(HasHardenedDerivation(hardened));
+    BOOST_CHECK(HasHardenedDerivation(mixed));
+}
+
 BOOST_AUTO_TEST_CASE(bip32_max_depth) {
     CExtKey key_parent{DecodeExtKey(test1.vDerive[0].prv)}, key_child;
     CExtPubKey pubkey_parent{DecodeExtPubKey(test1.vDerive[0].pub)}, pubkey_child;
@@ -202,4 +245,85 @@ BOOST_AUTO_TEST_CASE(bip32_max_depth) {
     BOOST_CHECK(!pubkey_parent.Derive(pubkey_child, 0));
 }
 
+BOOST_AUTO_TEST_CASE(parse_hd_keypath)
+{
+    std::vector<uint32_t> keypath;
+
+    BOOST_CHECK(ParseHDKeypath("1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1", keypath));
+    BOOST_CHECK(!ParseHDKeypath("///////////////////////////", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1'/1", keypath));
+    BOOST_CHECK(!ParseHDKeypath("//////////////////////////'/", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/", keypath));
+    BOOST_CHECK(!ParseHDKeypath("1///////////////////////////", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1/1'/", keypath));
+    BOOST_CHECK(!ParseHDKeypath("1/'//////////////////////////", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("", keypath));
+    BOOST_CHECK(!ParseHDKeypath(" ", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("0", keypath));
+    BOOST_CHECK(!ParseHDKeypath("O", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("0000'/0000'/0000'", keypath));
+    BOOST_CHECK(!ParseHDKeypath("0000,/0000,/0000,", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("01234", keypath));
+    BOOST_CHECK(!ParseHDKeypath("0x1234", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("1", keypath));
+    BOOST_CHECK(!ParseHDKeypath(" 1", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("42", keypath));
+    BOOST_CHECK(!ParseHDKeypath("m42", keypath));
+
+    // A path element's numeric part is capped at 2^31-1; the top bit is
+    // reserved for the hardened marker (h or ').
+    BOOST_CHECK(ParseHDKeypath("2147483647", keypath));  // 0x7fffffff, largest normal index
+    BOOST_CHECK(!ParseHDKeypath("2147483648", keypath)); // 0x80000000, would set the hardened bit
+    BOOST_CHECK(!ParseHDKeypath("4294967295", keypath)); // 0xffffffff
+    BOOST_CHECK(!ParseHDKeypath("4294967296", keypath)); // uint32_t max + 1
+
+    BOOST_CHECK(ParseHDKeypath("m", keypath));
+    BOOST_CHECK(!ParseHDKeypath("n", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("m/", keypath));
+    BOOST_CHECK(!ParseHDKeypath("n/", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("m/0", keypath));
+    BOOST_CHECK(!ParseHDKeypath("n/0", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("m/0'", keypath));
+    BOOST_CHECK(!ParseHDKeypath("m/0''", keypath));
+
+    keypath.clear();
+    BOOST_REQUIRE(ParseHDKeypath("m/0h/1h/2h", keypath));
+    BOOST_REQUIRE_EQUAL(keypath.size(), 3);
+    BOOST_CHECK_EQUAL(keypath[0], 0x80000000U);
+    BOOST_CHECK_EQUAL(keypath[1], 0x80000001U);
+    BOOST_CHECK_EQUAL(keypath[2], 0x80000002U);
+    BOOST_CHECK(!ParseHDKeypath("m/0hh", keypath));
+    BOOST_CHECK(!ParseHDKeypath("m/h0", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("m/0'/0'", keypath));
+    BOOST_CHECK(!ParseHDKeypath("m/'0/0'", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("m/0/0", keypath));
+    BOOST_CHECK(!ParseHDKeypath("n/0/0", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("m/0/0/00", keypath));
+    BOOST_CHECK(!ParseHDKeypath("m/0/0/f00", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("m/0/0/000000000000000000000000000000000000000000000000000000000000000000000000000000000000", keypath));
+    BOOST_CHECK(!ParseHDKeypath("m/1/1/111111111111111111111111111111111111111111111111111111111111111111111111111111111111", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("m/0/00/0", keypath));
+    BOOST_CHECK(!ParseHDKeypath("m/0'/00/'0", keypath));
+
+    BOOST_CHECK(ParseHDKeypath("m/1/", keypath));
+    BOOST_CHECK(!ParseHDKeypath("m/1//", keypath));
+}
+
 BOOST_AUTO_TEST_SUITE_END()