Skip to content

Reorder row groups by GROUP BY keys to reduce aggregate partition state and improve cache locality #21581

Description

@zhuqi-lucas

Is your feature request related to a problem or challenge?

#21317 introduced reordering row groups by statistics for sort pushdown (ORDER BY / TopK queries). The same mechanism could be extended to GROUP BY queries to improve aggregate performance.

As @adriangb and @Dandandan suggested in #21580 (comment):

Ordering by grouping keys can:

  • Reduce cardinality within partitions (partition state can be smaller)
  • Allow for better cache locality (row groups with more equal keys are grouped together)

Describe the solution you'd like

The existing PreparedAccessPlan::reorder_by_statistics method already accepts any LexOrdering — it's generic, not tied to sort pushdown. Extending this for GROUP BY would be:

  1. In the aggregate planner, compute a preferred row group ordering from the grouping keys
  2. Pass it through ParquetSource::sort_order_for_reorder
  3. Existing reorder logic handles the rest
SELECT category, SUM(x) FROM t GROUP BY category

Before: RGs in random order
  → HashAggregate sees mixed categories
  → Hash table stays large (all categories in memory)
  → Cache misses as different categories interleave

After: RGs ordered by category's min statistics
  → HashAggregate sees category-grouped rows
  → Can finalize/flush groups earlier (streaming aggregation opportunity)
  → Better cache locality (same category rows adjacent)

Describe alternatives you've considered

  • Streaming aggregation requires fully sorted input, which is a stronger guarantee. RG-level reorder doesn't guarantee full sort but improves locality within each RG.

Additional context

The infrastructure from #21580 should be directly reusable — this issue is mainly about wiring up the GROUP BY planner to populate sort_order_for_reorder based on grouping keys.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions