Two integration jobs failed during setup when Docker Hub timed out while pulling images:
Both failed before tests ran. Their Spark 4 counterparts passed.
Integration setup currently runs docker compose up -d --wait once, so a temporary registry timeout fails the whole job.
A few possible fixes:
- Retry docker compose pull in CI with backoff, then start with --pull never. This seems safest and does not change local behavior.
- Retry the full integration setup command. This is simpler, but may recreate a partially started environment.
- Limit concurrent image pulls and add backoff to reduce registry pressure.
- Use a registry mirror or preloaded image cache.
Would you prefer the CI-only pull retry, or should the retry behavior live in the Makefile so it can also be used locally?
Happy to send a PR with the preferred approach.
Two integration jobs failed during setup when Docker Hub timed out while pulling images:
Both failed before tests ran. Their Spark 4 counterparts passed.
Integration setup currently runs docker compose up -d --wait once, so a temporary registry timeout fails the whole job.
A few possible fixes:
Would you prefer the CI-only pull retry, or should the retry behavior live in the Makefile so it can also be used locally?
Happy to send a PR with the preferred approach.