fix(common): preserve an exact zero through filter selectivity estimation - #23936
Open
asolimando wants to merge 1 commit into
Open
fix(common): preserve an exact zero through filter selectivity estimation#23936asolimando wants to merge 1 commit into
asolimando wants to merge 1 commit into
Conversation
…tion `Precision::with_estimated_selectivity` documented itself as always returning inexact statistics, so `Exact(0)` became `Inexact(0)`. Scaling zero by any selectivity is still exactly zero: filtering an input that provably has no rows cannot produce rows, whatever the predicate is. The consequence is not just a weaker estimate. Because operators widen exact to inexact as statistics flow up a plan, emptiness could not be propagated upwards through statistics at all, it was lost at the first `FilterExec`. `FilterExec` already special-cased this for a contradictory predicate, which returns `Exact(0)` directly, and for column byte sizes via a local `scale_byte_size` helper, so the two disagreed about the same fact depending on the path taken. Preserve `Exact(0)` in `with_estimated_selectivity` itself, which is only called from `FilterExec`, and drop `scale_byte_size` now that it is subsumed. `Inexact(0)` is untouched: an estimate of zero is not a proof of zero.
github-actions
Bot
added
common
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #23936 +/- ##
========================================
Coverage 80.67% 80.68%
========================================
Files 1095 1095
Lines 372460 372576 +116
Branches 372460 372576 +116
========================================
+ Hits 300488 300615 +127
+ Misses 54045 54015 -30
- Partials 17927 17946 +19 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
tschwarzinger
approved these changes
Jul 28, 2026
Member
Author
|
cc:
|
kosiew
approved these changes
Jul 29, 2026
| /// Caps a row-bounded column statistic (a null count or distinct count) at the | ||
| /// filtered row estimate, since a column cannot have more nulls or distinct | ||
| /// values than it has rows. Known counts are demoted to inexact because the | ||
| /// filtered row count is itself an estimate. |
Contributor
There was a problem hiding this comment.
Could we update this comment slightly? It currently says the filtered row count is always an estimate, but this PR now allows filtered_num_rows to be Exact(0). Maybe clarify that counts are normally demoted because filter-derived row bounds are estimates, except when the row bound proves there are zero rows.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
Precision::with_estimated_selectivityalways demoted to inexact, soExact(0)becameInexact(0). Scaling zero by any selectivity is still exactly zero: filtering a provably empty input cannot produce rows.Because operators widen exact to inexact, emptiness could not propagate up a plan through statistics at all, it was lost at the first
FilterExec. That operator already preserved the exact zero for a contradictory predicate and for column byte sizes, so the same fact came out exact or inexact depending on the path taken.This is the same reasoning as #23670, which stops
FileScanConfig::statistics()demoting an exact zero when a filter is present, applied one level up. The two are independent, but together they let a proof of emptiness survive from the scan through the filter.What changes are included in this PR?
with_estimated_selectivityreturns an exact zero unchanged.Inexact(0)is untouched: an estimate of zero is not a proof of zero.scale_byte_sizeinfilter.rsis removed, subsumed by the above.Are these changes tested?
Yes. A unit test for the method, and a
FilterExectest that a satisfiable predicate over an exactly empty input keepsnum_rows,total_byte_size(proving thatscale_byte_sizecan be safely removed) and columnbyte_sizeexact.test_filter_statistics_empty_input_equality_ndv_zeroassertedInexact(0)and now assertsExact(0), which is expected.Are there any user-facing changes?
No breaking changes.
Precision::with_estimated_selectivityis public and keeps its signature, but its documented contract changes: it previously stated it would always return inexact statistics, and it now preserves an exact zero. Callers relying on the old wording will seeExact(0)where they sawInexact(0). This is an improvement and could be arguably considered a bug-fix, so I consider this contract change justified.FilterExecover a provably empty input therefore reports exact statistics, visible inEXPLAINoutput that shows statistics.Disclaimer: I used AI to assist in the code generation, I have manually reviewed the output and it matches my intention and understanding.