Skip to content

fix: build delete filters with PyIceberg expressions - #348

Open
konstantinoscs wants to merge 1 commit into
feature/tow-2494-commit-retry-backofffrom
feature/tow-2494-iceberg-filter-expressions
Open

fix: build delete filters with PyIceberg expressions#348
konstantinoscs wants to merge 1 commit into
feature/tow-2494-commit-retry-backofffrom
feature/tow-2494-iceberg-filter-expressions

Conversation

@konstantinoscs

@konstantinoscs konstantinoscs commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

What this does

This PR replaces Tower's fragile PyArrow-filter string parser with native PyIceberg expressions.

Users can now write filters such as:

table.delete(
    (table.column("age") >= 18)
    & (table.column("status") != "inactive")
)

SQL-like strings and native PyIceberg expressions remain supported. Old PyArrow expressions now raise a clear migration error before any write credentials are requested.

Why

The previous implementation tried to understand a filter by parsing its rendered text. That could misread column names or values containing words such as and, or, or not. Building the expression structurally avoids those errors.

S3 Tables and BYO catalog selection and credentials are unchanged.

Testing

Tests cover every comparison operator, logical composition and inversion, nested fields, unusual identifiers and values, and the PyArrow migration error.

@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: de4b6982-abcd-477f-bcd3-29fec62365e2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@konstantinoscs
konstantinoscs force-pushed the feature/tow-2494-iceberg-filter-expressions branch from 000543d to b88edcd Compare August 13, 2026 13:52
@konstantinoscs
konstantinoscs force-pushed the feature/tow-2494-iceberg-filter-expressions branch 2 times, most recently from 0486414 to 04fcdbf Compare August 13, 2026 17:39
@konstantinoscs
konstantinoscs force-pushed the feature/tow-2494-iceberg-filter-expressions branch from 04fcdbf to d808ad0 Compare August 13, 2026 19:18
@konstantinoscs
konstantinoscs requested a review from bradhe August 14, 2026 09:09
@konstantinoscs
konstantinoscs marked this pull request as ready for review August 14, 2026 09:09
@konstantinoscs
konstantinoscs requested review from jo-sm and socksy August 14, 2026 09:12
@konstantinoscs
konstantinoscs force-pushed the feature/tow-2494-iceberg-filter-expressions branch from d808ad0 to 9465063 Compare August 14, 2026 10:42
@konstantinoscs
konstantinoscs force-pushed the feature/tow-2494-iceberg-filter-expressions branch from 9465063 to 3fbbddc Compare August 14, 2026 14:26
@konstantinoscs
konstantinoscs force-pushed the feature/tow-2494-iceberg-filter-expressions branch from 3fbbddc to f949272 Compare August 14, 2026 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant