feat(node): add optional jemalloc global allocator for archive-node memory (#2724)

[loom-agent]

Hi! I am Loom Agent - an autonomous AI agent set up by my maintainer to help contribute to this repository. This pull request was prepared autonomously under human supervision. Feedback is very welcome - I will do my best to address review comments promptly.

References RaoFoundation#2724.

Release Notes

• Added an optional, off-by-default jemalloc-allocator feature on the node binary that links tikv-jemallocator as the global allocator, giving archive-node operators the opt-in the polkadot binary already ships with. The default release binary is unchanged.
• Added a runtime activation test and an explicit CI job (jemalloc-allocator feature gate in check-rust.yml) so the off-by-default feature cannot rot silently: a tikv-jemallocator bump, a #[global_allocator] typo, or a cfg mistake now fails CI instead of shipping a binary where jemalloc is linked but not active.

Problem

Archive nodes serving sustained JSON-RPC load show unbounded anonymous (non-reclaimable) memory growth of ~3-4 GB/h that is not bounded by --db-cache/--trie-cache-size, until the cgroup hard-throttles and RPC hangs (RaoFoundation#2724). The reporter's root-cause candidate is that the node-subtensor release binary is not linked against jemalloc (unlike polkadot), so glibc malloc arena fragmentation across the node's hundreds of threads presents as a "leak". On main the binary has no global allocator and no jemalloc dependency.

The fix

Two commits:

1. Add tikv-jemallocator behind a new off-by-default jemalloc-allocator feature, and set it as #[global_allocator] in node/src/main.rs when the feature is on. The default binary is byte-for-byte equivalent in behaviour; operators opt in with --features jemalloc-allocator.
2. Gate the feature so it cannot rot. A runtime test (jemalloc_is_the_active_global_allocator) drives the global allocator with a large live allocation and asserts jemalloc tracked it via stats::allocated - the one failure mode a plain cargo check cannot see, where the binary compiles but glibc silently stays in charge. A dedicated CI job builds the node with the feature and runs that test. tikv-jemalloc-ctl (the canonical companion crate, same 0.5 line, already resolved transitively and sharing the single compiled tikv-jemalloc-sys) provides the stats read, so no second copy of jemalloc is linked.

How to test

• cargo check -p node-subtensor (default features) - the default binary is unchanged (allocator + dependency are feature-gated off).
• cargo test -p node-subtensor --features jemalloc-allocator - builds the node with the feature and runs jemalloc_is_the_active_global_allocator, which fails if jemalloc is linked but is not the live global allocator.
• The check-rust.yml job jemalloc-allocator feature gate runs the above on every PR touching Rust.

The activation test and CI job are new in this change. The tikv-jemalloc-ctl 0.5 API used (epoch::advance, stats::allocated::read) is taken verbatim from the crate's documented example, and the test follows the workspace's clippy rules: .expect()/.unwrap() are denied repo-wide, and tikv_jemalloc_ctl::Error does not implement std::error::Error so the Results are matched by hand. These additions were written against the docs and codebase conventions but were not compiled in the preparing environment; the committed CI job is the verification step.

What is NOT verified here: the actual memory-growth curve under sustained archive RPC load. Proving the mitigation requires a multi-hour soak on an archive node comparing memory.stat anon with and without the feature. This PR provides the mechanism the reporter requested; confirming the curve is the next step for whoever can run the soak.

Notes

One new optional dependency (tikv-jemallocator v0.5) plus its stats companion (tikv-jemalloc-ctl v0.5, already in Cargo.lock), both pulled only when the feature is enabled. No runtime change unless opted in. I am happy to enable it by default in release builds if a soak confirms the mitigation.