feat(table): variant predicate pushdown and reject variant partition source - #1607
Open
nssalian wants to merge 2 commits into
Open
feat(table): variant predicate pushdown and reject variant partition source#1607nssalian wants to merge 2 commits into
nssalian wants to merge 2 commits into
Conversation
nssalian
marked this pull request as ready for review
July 29, 2026 23:05
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds predicate pushdown for shredded variant fields and rejects Variant as a partition
source. Closes #1016.
Changes
Extract expressions -
extract(variant, path, type)referencing a field inside a variant column. RFC-9535 path parse (variant_path.go), bind against the shredded schema (variant_extract.go), cast to the target type (variant_cast.go). Follows JavaExpressions.extract/PathUtil.parse/VariantExpressionUtil.castTo.File pruning - inclusive/strict metrics evaluators use a shredded field's stored bounds for an extract term (
table/evaluators.go); the strict path never prunes on extract. Bounds are split via arrow-goMetadata.SizeBytes()(
table/internal/variant_bounds.go). Follows JavaInclusiveMetricsEvaluator/StrictMetricsEvaluator.Residual filtering (columnar) - surviving extract terms become synthetic derived columns evaluated through the existing arrow-go
computerecord-filter path, not row-by-row (scanTranslatorinvisitors.go,table/variant_residual.go,table/arrow_scanner.go).Reject Variant partition source -
addSpecFieldInternal(partitions.go) rejects a Variant source for any transform. Builds on #1206 (which added Variant toIdentityTransform's deny-list); this path does not consultCanTransform, so it closes the gap for sources added viaAddPartitionFieldBySourceID.Testing
table/variant_shredded_write_test.go).