Skip to content

Flaky e2e test: gossip test times out waiting for 81 transactions #4321

Description

@reinkrul

Summary

The nuts-network/gossip e2e test intermittently fails on CI with:

Performing assertions...
Waiting for service 'NodeA' to contain 81 transactions..........FAILED: Service 'NodeA' did not get 81 transaction within 10 seconds

Observed on an unrelated PR (#4302, a vcr/verifier nil-pointer fix); the test passed on re-run, confirming it is flaky rather than a regression.

Example failed run: https://github.com/nuts-foundation/nuts-node/actions/runs/26814933661/job/79054197457

Cause

e2e-tests/nuts-network/gossip/run-test.sh creates 81 transactions across 4 nodes (1 root + 20 DIDs × 4 nodes), then asserts every node has gossiped all 81 within a fixed window:

waitForTXCount "NodeA" "http://localhost:18081/status/diagnostics" 81 10
waitForTXCount "NodeD" "http://localhost:48081/status/diagnostics" 81 10

waitForTXCount (e2e-tests/util.sh) polls transaction_count once per second for at most TIMEOUT (10) iterations. Under CI load, gossip propagation of all 81 transactions can take longer than 10 seconds, so the assertion times out even though the network would eventually converge.

Impact

Spurious red CI on unrelated PRs; requires a manual re-run to go green.

Possible fixes

  • Increase the waitForTXCount timeout for the gossip test (e.g. 10 → 30).
  • Make waitForTXCount poll more frequently with a wall-clock deadline instead of a fixed iteration count.

Assisted by AI

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions