Skip to content

Commit 2e3626e

Browse files
authored
minor(test): cover partially ordered aggregate spilling (#23947)
## Which issue does this PR close? - Not closes but part of #13431 ## Rationale for this change `GroupedHashAggregateStream` supports spilling for partially sorted group input, but existing aggregate spill tests only cover unordered (`GroupOrdering::None`) input. The `GroupOrdering::Partial` + `OutOfMemoryMode::Spill` path, including spilling sorted intermediate state and merging it back, does not have direct coverage. ## What changes are included in this PR? added test coverage for the case above ## Are these changes tested? test only change and it can be tested via: ``` cargo test --test sqllogictests -- ordered_aggregate_spill ``` ## Are there any user-facing changes? no test only change
1 parent 2f25454 commit 2e3626e

1 file changed

Lines changed: 52 additions & 0 deletions

File tree

datafusion/sqllogictest/test_files/ordered_aggregate_spill.slt

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,58 @@ Plan with Metrics
176176
03)----AggregateExec: mode=Partial,<slt:ignore>aggr=[sum(t1.v1 * Int64(2)), min(t1.v1 % Int64(2))], ordering_mode=PartiallySorted([0]), metrics=[<slt:ignore>spill_count=0,<slt:ignore>]
177177
<slt:ignore>
178178

179+
# ==================================================================================
180+
# Single mode: with one partition the whole aggregation runs in a `Single` mode
181+
# AggregateExec. min() keeps one intermediate state and avg() keeps two (sum +
182+
# count), so both single- and multi-state accumulators are spilled and merged.
183+
# ==================================================================================
184+
185+
statement ok
186+
SET datafusion.execution.target_partitions = 1
187+
188+
# Reference round: enough memory to aggregate without spilling.
189+
statement ok
190+
SET datafusion.runtime.memory_limit = '10M'
191+
192+
query TT
193+
EXPLAIN ANALYZE
194+
SELECT round(v1, -4), v1 % 5000, min(v1 * 2), avg(v1)
195+
FROM generate_series(20000) AS t1(v1)
196+
GROUP BY round(v1, -4), v1 % 5000
197+
----
198+
Plan with Metrics
199+
01)AggregateExec: mode=Single,<slt:ignore>aggr=[min(t1.v1 * Int64(2)), avg(t1.v1)], ordering_mode=PartiallySorted([0]), metrics=[<slt:ignore>spill_count=0,<slt:ignore>]
200+
<slt:ignore>
201+
202+
query IIIR rowsort
203+
SELECT round(v1, -4), v1 % 5000, min(v1 * 2), avg(v1)
204+
FROM generate_series(20000) AS t1(v1)
205+
GROUP BY round(v1, -4), v1 % 5000
206+
----
207+
60000 values hashing to 872df6cefd51f81820fc5c6e5d7480df
208+
209+
# Spilling round: the same query under a 600 KB limit must spill.
210+
statement ok
211+
SET datafusion.runtime.memory_limit = '600K'
212+
213+
query TT
214+
EXPLAIN ANALYZE
215+
SELECT round(v1, -4), v1 % 5000, min(v1 * 2), avg(v1)
216+
FROM generate_series(20000) AS t1(v1)
217+
GROUP BY round(v1, -4), v1 % 5000
218+
----
219+
Plan with Metrics
220+
01)AggregateExec: mode=Single,<slt:ignore>aggr=[min(t1.v1 * Int64(2)), avg(t1.v1)], ordering_mode=PartiallySorted([0]), metrics=[<slt:ignore>spilled_bytes=<slt:ignore> KB,<slt:ignore>]
221+
<slt:ignore>
222+
223+
# Same result hash as the no-spill round above
224+
query IIIR rowsort
225+
SELECT round(v1, -4), v1 % 5000, min(v1 * 2), avg(v1)
226+
FROM generate_series(20000) AS t1(v1)
227+
GROUP BY round(v1, -4), v1 % 5000
228+
----
229+
60000 values hashing to 872df6cefd51f81820fc5c6e5d7480df
230+
179231
statement ok
180232
RESET datafusion.runtime.memory_limit
181233

0 commit comments

Comments
 (0)