Readiness-aware view exploitation: distinguish predicate-pruned empty MVs from unpopulated ones - #122
Merged
xudong963 merged 4 commits intoJul 28, 2026
Conversation
… MVs from unpopulated ones
A materialized view candidate whose scan yields an empty plan is
ambiguous: either the predicate genuinely pruned everything (the empty
result is correct and extremely cheap), or the MV simply has not been
populated yet (trusting the empty plan silently returns wrong results).
Cost functions previously could not tell these apart.
This adds:
- RewriteReadiness (Ready / NotReady / Unknown) and a
Materialized::rewrite_readiness() method (default: Unknown) so table
providers can report their lifecycle state.
- CandidateMetadata (Base | Materialized { table_ref, readiness })
carried per branch via a RewriteContext attached to OneOf/OneOfExec,
aligned with branch order. To make that alignment hold,
OneOf::inputs() now returns branches in stored order (previously it
re-sorted with LogicalPlan::partial_cmp on every call, which could
move the base branch off branches[0] after a rebuild and expose a
candidate's schema from OneOf::schema()); candidates are sorted by
table reference at construction, and the base plan is pinned at
branches[0].
- Two-gate NotReady filtering: NotReady candidates are dropped at
logical rewrite time, and again at physical planning time after a
refresh.
- Physical-time readiness refresh in plan_extension: by the time the
physical planner runs, every candidate's scan() has executed, so
providers whose readiness depends on scan-time side effects (lazy
index loading) are re-consulted. Providers can also opt into
ReadinessAnnotatedExec, a transparent wrapper that carries readiness
captured atomically from the scan's own snapshot, which the refresh
prefers over re-sampling (avoids a TOCTOU race with concurrent index
publication); the wrapper is stripped before the OneOfExec is built.
- CostFn now receives a CostContext (candidate plans + rewrite
metadata) instead of a bare plan iterator, so cost functions can
implement policies like 'trust Ready empty candidates, penalize
Unknown empty ones'.
43 new unit tests cover the readiness types, annotation and stripping,
the two filtering gates, the refresh, and branch-order/metadata
alignment across plan rebuilds.
zhuqi-lucas
marked this pull request as draft
July 27, 2026 09:35
- dependencies.rs: LogicalPlan::Window pattern already covers window_expr via .. - file_metadata.rs: use ? on the else branch instead of explicit return None
The current cargo-deny CLI rejects the singular `license` subcommand:
error: invalid value 'license' for '[WHICH]...'
Use the plural form so the license check runs.
zhuqi-lucas
marked this pull request as ready for review
July 28, 2026 06:39
xudong963
approved these changes
Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
A materialized view candidate whose scan yields an empty plan is ambiguous. Either the predicate genuinely pruned everything, in which case the empty candidate is the correct answer and dramatically cheaper than scanning the base table (a selective filter on a rarely-matching literal), or the MV simply has not been populated yet, in which case trusting the empty plan silently returns wrong results. A cost function today cannot tell these apart, so the safe policy is to penalize every empty candidate, which also throws away the legitimate predicate-pruned wins.
What changes are included in this PR?
RewriteReadiness(Ready/NotReady/Unknown) andMaterialized::rewrite_readiness()(defaultUnknown), letting table providers report lifecycle state. New modulerewrite::readiness.CandidateMetadata(Base|Materialized { table_ref, readiness }) carried per branch via aRewriteContextattached toOneOf/OneOfExec, positionally aligned with branch order.OneOf::inputs()now returns branches in stored order. It previously re-sorted withLogicalPlan::partial_cmpon every call; sincewith_exprs_and_inputsrebuilds branches frominputs()output, any optimizer pass that mutates a branch's shape could move the base plan offbranches[0], at which pointOneOf::schema()exposes a candidate's schema instead of the query's. Candidates are now sorted by table reference at construction (themv_plansHashMap iteration order was also nondeterministic across processes) and the base plan is pinned atbranches[0].NotReadyfiltering: dropped at logical rewrite time, and again at physical planning time after a refresh.plan_extension: by the time the physical planner runs, every candidate'sscan()has executed, so providers whose readiness depends on scan-time side effects (e.g. lazy index loading on first scan) are re-consulted for the definitive value. Providers can additionally opt intoReadinessAnnotatedExec, a transparent wrapper carrying readiness captured atomically from the scan's own snapshot; the refresh prefers the annotation over re-sampling, which closes a TOCTOU race (scan sees an unpopulated source and plans empty, a concurrent load publishes the source, the refresh re-samples and wrongly trusts the stale empty plan). The wrapper is stripped before theOneOfExecis constructed so it never impedes later optimizer passes.CostFnsignature change: cost functions now receive aCostContext(candidate plan iterator +RewriteContext) instead of a bare iterator, so they can implement readiness-aware policies such as "trustReadyempty candidates, keep penalizingUnknownempty ones".Are these changes tested?
43 new unit tests: readiness types and annotation/strip round-trips, both filtering gates, the physical-time refresh (including the scan-transition and all-candidates-NotReady short-circuit cases), and branch-order/metadata alignment across
with_exprs_and_inputsrebuilds.Are there any user-facing changes?
CostFnis a breaking signature change: existing cost functions change fromFn(Box<dyn Iterator<Item = &dyn ExecutionPlan>>)toFn(CostContext), whereCostContext::into_candidate_plans()returns the same iterator.Materializedgains a provided method (rewrite_readiness), default-implemented, so existing implementations are unaffected.