Run integration tests against multiple Bitcoin Core versions

[Sjors]

The current integration tests are slow and only exercise the latest node release. This runs them against Bitcoin Core v30.2, v31.0, and master (on Ubuntu and macOS).

How the work is divided

• Compile the integration-test binaries once per OS (cargo nextest archive); every test job runs from that archive instead of recompiling.
• Tests are split by runtime dependency into three jobs:
  • node-free tests run once per OS (they don't need Bitcoin Core, so they skip the version matrix)
  • BitcoinCoreIpc and standalone sv2-tp tests run across the full v30.2 / v31.0 / master matrix
• Release versions (v30.2, v31.0) are downloaded; bitcoin/bitcoin@master is source-built once per OS and cached with ccache.

Version handling

• Selects the matching sv2-tp release per node version (v1.0.6 for v30.x, v1.1.0 for v31.x/master) and the right template-interval flag.
• Adds unix_capnp/master, which reuses the v31 backend via path imports — no file copying, since master doesn't yet diverge from v31.

Caveat: master tracks bitcoin-capnp-types through a git branch = "master" dependency and a cargo update step in CI, so - like Bitcoin Core - it changes as upstream moves.

Testing a feature branch instead of master is a one-line change to the matrix entry (e.g. youruser/bitcoin@your-branch).

CI Summary tab preview:

Maintainer note

This replaces two big integration tests with many small ones, which impacts the branch protection rule for required CI jobs.

[Sjors]

It's probably better to merge #602 first because of how its code de-duplication interacts with e0647db test: add Bitcoin Core master IPC bindings. I'm happy to rebase this after.

[plebhash]

@Sjors thanks for the contribution

I agree that the current state of Integration Test CI is far from perfect, and there's lots of room for improvement

with that said, I'm afraid this PR is a bit too ambitious, in the sense that it's trying to do things in many different axis, while each different axis deserves careful evaluation

I propose we have an initial discussion about them here, and based on the outcomes we can create different issues so they can be addressed/reviewed gradually and confidently

improve coverage across different Bitcoin Core versions

with #548, the deliberate policy for coverage across Bitcoin Core versions we established was:

• coverage focus is whatever is the latest Bitcoin Core version
• past versions are always covered via tests/{bitcoin_core_ipc_jdp_io.rs,bitcoin_core_ipc_tdp_io.rs}

the rationale for this strategy is:

• as new Bitcoin Core versions come out, we are always guarded against regressions because we know we executed full Integration Test coverage against it
• the so-called TDP/JDP IO tests always assert basic coverage

this felt like a reasonable balance between the multiplicative explosion that could come out of (naively) attempting to always execute the entire stack against all versions

but I'm sure there's ways to poke holes in this rationale, as it might leave room for regressions against old versions to silently seep in

support Bitcoin Core master branch on Integration Test coverage

IIUC you want a way to be able to easily run Integration Tests against master (on github upstream repo, not on a fork or locally)

tbh I have a few concerns related to this, for a few different reasons

first and most importantly, this is an aggressive change on how bitcoin_core_sv2 modules and APIs are organized, so the scope goes beyond a simple CI tweak

moreover, the introduction of a BitcoinCoreVersion::Master option makes release policy on bitcoin_core_sv2 a bit fuzzy

more specifically, what criteria would we use to establish which specific commit of master is supported on a specific release of bitcoin_core_sv2? whatever is HEAD at the time the crate is published feels a bit too arbitrary, and is also pretty hard to enforce at runtime

I also worry about the potential for nondeterministic behavior on CI

can you elaborate on your motivations to pursue this goal? overall I'm trying to understand whether this is really something that we should be supporting upstream on main branch and official crate releases vs a development-focused tweak to be maintained on a separate branch

execution topology + execution speed

clustering tests across different files is an area where previous discussions have been somewhat shallow

people had different opinions, no clear policy/criteria was established and we just moved on

as a result, the way in which new integration tests are added to the stack is somewhat arbitrary, with the downside of poor parallelization and low execution speeds

IIUC this PR is trying to cluster test execution in terms of:

• no TP at all
• Bitcoin Core as TP via IPC
• Bitcoin Core + sv2-tp as TP

which in theory would:

• avoid multiplying version-agnostic tests by the Bitcoin Core version matrix
• improve debuggability because failures are pre-clustered by subsystem

I'm not necessarily opposed to this approach, but it is one criteria amongst many potential alternative ones, so IMO it deserves a more widespread discussion

[Sjors]

@plebhash thanks for the feedback. I think it makes sense to split things off into smaller pull requests, after discussion. Marking this as draft.

There's two motivations for tracking Bitcoin Core's master branch:

1. Catch breaking changes early
2. It's easy to switch to another branch (built from source) like PoC: replace the mempool mirror with Bitcoin Core's TxCollection #599 does and run all of CI against it (including checks against official releases, which new code might accidentally break)

However I think you're right that it doesn't need to be on the main branch, perhaps a nightly branch would be better for this (with nightly CI runs against the laster bitcoin/bitcoin master branch).

balance between the multiplicative explosion

I don't think you can avoid such an explosion, and I think it's better to explictly not support old versions than to half-support them.

Also, by having CI run the actual release, it's easy to reproduce a (newly discovered) bug with an old release, open a pull requests with a test and fix combo, and immediately get feedback from CI that it's an end-to-end fix.

I also find it easier to reason about code that talks to a real node, but that's perhaps because I'm more familar with the node itself better than the code here :-)

Based on my experience with this PR, it seems fine to support several releaeses, maybe tracking Bitcoin Core's end-of-life policy.

Maybe a good place to start are the first couple of commits that re-organize the tests a bit more by subsystem? Then wait for #602 and then have v31 and v30 run against a real node in CI, not including master?