[DO NOT MERGE] Add list length min-max aggregate - #8886
Conversation
Merging this PR will degrade performance by 19.98%
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
Polar Signals Profiling ResultsLatest Run
Previous Runs (2)
Powered by Polar Signals Cloud |
Benchmarks: Vortex queries 📖Verdict: Likely regression (medium confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.276x ❌, 0↑ 2↓)
datafusion / parquet (1.051x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (1.051x ➖, 0↑ 0↓)
duckdb / parquet (1.018x ➖, 0↑ 0↓)
No file size changes detected. |
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
45e424f to
614492d
Compare
Adds an aggregate function
ListLengthMinMaxto calculate the minimum and maximum length of aList-like array. This is just the currentMinMaxaggregate composed with theListLengthscalar function.This will allow us to do several beneficial things if incorporated as a default stat in upcoming pr)
FixedSizeListlayoutlist_length(obviously) and others, since we can identify zones with all-empty lists and prune them (e.g.list_sum > k)What changes are included in this PR?
NOTE: Should come up with framework to do aggregate on derived column instead of just implementing this one-off