ci: test minimum and latest Iceberg dependencies

[konstantinoscs]

What this does

Adds a small compatibility matrix for the Tables SDK dependencies:

• tests the exact minimum versions currently promised: PyArrow 23.0.1 and PyIceberg 0.11.1
• tests the latest stable versions selected by the package resolver
• runs on every PR and once daily, so new upstream releases are checked even when the CLI repository is quiet
• prints the versions installed in each job

This PR does not add upper bounds. The existing >= requirements remain unchanged, so users can install newer stable releases.

Why

pyproject.toml states which versions are installable; this matrix verifies that the oldest promised versions and the newest available versions actually work. If an upstream release introduces a real incompatibility, CI will show it and we can add a temporary cap based on evidence.

The focused suite also keeps the existing managed catalog, S3 Tables, and BYO catalog behavior under test.

Stack

PR 2 of 5. #345 is merged. Next: #347.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0c5caf19-13fe-4d5b-b392-168d1de6f272

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}

[konstantinoscs]

@bradhe take another look. I rethought about this. My initial idea was to bound pyiceberg and arrow to make explicit what version we support and what not as unbounded updates can theoretically break the CLI. However I think we can achieve a quite good level of support without bounding the dependencies by triggering a daily job that basically tests that we still support the two pairs of lowest/lowest and latest/latest pyiceberg and arrow. That gives us a quite early signal if there's ever a breaking change upstream

[bradhe]

Should this just be a separate workflow? Do we need to run the tests on this interval?

[konstantinoscs]

This just runs the tests once a day as well as on every PR.
To me it seems it belongs here. We should run the tests + dependencies compatibility at once? Seems like a reasonable "all things python" coupling.
But If you feel strongly about it, I can move it to another workflow I guess