Skip to content

Add benchmarks for hashjoin candidate equality filtering - #23980

Merged
neilconway merged 1 commit into
apache:mainfrom
shehab-ali:shehab/benchmarks-skewed-join-key
Jul 30, 2026
Merged

Add benchmarks for hashjoin candidate equality filtering#23980
neilconway merged 1 commit into
apache:mainfrom
shehab-ali:shehab/benchmarks-skewed-join-key

Conversation

@shehab-ali

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

Issue #23237 identifies hash join candidate equality filtering as a potential
performance bottleneck. Before changing that implementation, we need stable
benchmarks that reproduce the relevant workloads and let us measure the effect
of a proposed optimization independently.

The existing hash join benchmark suite covers high-fanout joins, but it does
not isolate these two cases:

  1. A single hot hash bucket in which all build-side rows use the same long
    string key.
  2. A skewed composite-key join in which candidate pairs must be checked across
    multiple key columns.

This PR intentionally adds only the benchmarks. The implementation change for
equal_rows_arr will be submitted separately so that the benchmark workloads
can be reviewed and established independently of the optimization.

What changes are included in this PR?

This PR adds two hash join benchmarks:

  • Q24: single-hot-bucket long string-key join

    • Creates 3,000 build-side rows with the same long string key.
    • Selects matching probe-side rows that use that same key.
    • Causes every selected probe row to fan out across the entire build-side
      bucket.
    • Uses count(*) so the benchmark emphasizes hash match and candidate
      equality processing without materializing and retaining the full joined
      output.
  • Q25: skewed composite-key join

    • Uses an integer key and a long string key.
    • Distributes the build-side integer key over 256 values while all selected
      probe rows use one value.
    • Produces a high-fanout set of candidate matches for one integer-key value.
    • Requires the second string-key column to be checked as part of composite
      join-key equality.
    • Uses count(*) to focus the measurement on the join match path.

Both benchmarks:

  • Assert that the input data is available.
  • Assert that the physical plan contains HashJoinExec.
  • Use the no-statistics configuration and force Partitioned hash join mode.
  • Are registered in the hj benchmark runner with descriptive build/probe
    metadata.
  • Can be used to compare the existing implementation against the follow-up
    optimization for High HashJoin probe latency under high fanout and key skew (Partitioned mode) #23237.

Are these changes tested?

No hash join implementation is changed by this PR.

Are there any user-facing changes?

No.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.75%. Comparing base (1955d5a) to head (2e5933f).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #23980      +/-   ##
==========================================
- Coverage   80.75%   80.75%   -0.01%     
==========================================
  Files        1096     1096              
  Lines      373582   373605      +23     
  Branches   373582   373605      +23     
==========================================
+ Hits       301686   301699      +13     
- Misses      53893    53897       +4     
- Partials    18003    18009       +6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@shehab-ali shehab-ali changed the title Add benchamrks for hashjoin candidate equality filtering Add benchmarks for hashjoin candidate equality filtering Jul 29, 2026
@shehab-ali
shehab-ali marked this pull request as ready for review July 29, 2026 18:32

@neilconway neilconway left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, nicely done.

@neilconway
neilconway added this pull request to the merge queue Jul 30, 2026
Merged via the queue into apache:main with commit eacdf75 Jul 30, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants