Skip to content

Expose accumulator state to allow prefix scanning - #24035

Open
avantgardnerio wants to merge 2 commits into
apache:mainfrom
avantgardnerio:brent/bwag-finalized-state-observer
Open

Expose accumulator state to allow prefix scanning#24035
avantgardnerio wants to merge 2 commits into
apache:mainfrom
avantgardnerio:brent/bwag-finalized-state-observer

Conversation

@avantgardnerio

@avantgardnerio avantgardnerio commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Expose state of aggregate streams within BWAG so downstream prefix scanning can take place.

API (matches the suggestion verbatim)

// physical-plan/src/windows/bounded_window_agg_exec.rs
pub type FinalizedWindowStateObserver = Arc<
    dyn Fn(usize, &PartitionKey, &[Option<Vec<ScalarValue>>]) -> Result<()>
        + Send + Sync,
>;

impl BoundedWindowAggExec {
    pub fn with_finalized_state_observer(mut self, obs: FinalizedWindowStateObserver) -> Self {}
}

// physical-expr/src/window/window_expr.rs
impl WindowState {
    /// `Accumulator::state()` if this is an aggregate window function, `None` otherwise.
    pub fn aggregate_state(&mut self) -> Result<Option<Vec<ScalarValue>>> {}
}

🤖 Generated with Claude Code

@github-actions github-actions Bot added physical-expr Changes to the physical-expr crates physical-plan Changes to the physical-plan crate labels Jul 31, 2026
@avantgardnerio
avantgardnerio marked this pull request as draft July 31, 2026 18:55
@avantgardnerio avantgardnerio changed the title feat(physical-plan): FinalizedWindowStateObserver on BoundedWindowAggExec Expose accumulator state to allow prefix scanning Jul 31, 2026
@avantgardnerio
avantgardnerio force-pushed the brent/bwag-finalized-state-observer branch from ae37c0b to 65014d5 Compare July 31, 2026 18:58
@avantgardnerio
avantgardnerio requested a review from Dandandan July 31, 2026 18:58
@codecov-commenter

codecov-commenter commented Jul 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.82353% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.85%. Comparing base (dbcb5c0) to head (0767f73).
⚠️ Report is 11 commits behind head on main.

Files with missing lines Patch % Lines
...ysical-plan/src/windows/bounded_window_agg_exec.rs 83.96% 8 Missing and 13 partials ⚠️
datafusion/physical-expr/src/window/window_expr.rs 80.00% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##             main   #24035    +/-   ##
========================================
  Coverage   80.85%   80.85%            
========================================
  Files        1101     1101            
  Lines      374933   375580   +647     
  Branches   374933   375580   +647     
========================================
+ Hits       303166   303692   +526     
- Misses      53671    53781   +110     
- Partials    18096    18107    +11     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@avantgardnerio
avantgardnerio marked this pull request as ready for review August 1, 2026 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-expr Changes to the physical-expr crates physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants