fix(spark): support consistent hashing clustering on non-partitioned tables

[ad1happy2go]

Describe the issue this Pull Request addresses

Consistent hashing bucket-index clustering (bucket resizing) fails on non-partitioned tables.

For a non-partitioned table the partition path stored in the clustering group metadata is an empty string (""). SingleSparkJobConsistentHashingExecutionStrategy validated the partition with a "not null or empty" guard and threw IllegalArgumentException: Partition should not be null or empty before any clustering work could run, so split/merge resizing was impossible on non-partitioned tables.

GitHub issue: #18161

Summary and Changelog

• Relax the partition guard in SingleSparkJobConsistentHashingExecutionStrategy (both the merge and split paths) from !StringUtils.isNullOrEmpty(partition) to partition != null. An empty partition path is valid for non-partitioned tables; a genuinely absent metadata key (null) is still rejected.
• Add a parameterized test testResizingNonPartitioned in TestSparkConsistentBucketClustering, mirroring the existing testResizing, covering split and merge resizing on a non-partitioned table across the single-job and multi-job execution strategies and the row-writer on/off paths. A setup(..., boolean nonPartitioned) overload configures the non-partition key generator and an empty partition-path field.

Impact

Consistent hashing clustering now works on non-partitioned tables. No behavior change for partitioned tables — when the partition path is non-empty (every partitioned table), the guard evaluates identically and the code path is unchanged. No public API or config changes.

Risk Level

low

Partitioned-table behavior is byte-for-byte identical (the relaxed guard only differs when the partition path is the empty string). Verified via the new parameterized test and an end-to-end spark-shell run on Spark 4.0.2 against a non-partitioned MOR table with consistent-hashing bucket index: a follow-up write triggered inline split clustering through SingleSparkJobConsistentHashingExecutionStrategy, increasing the bucket count from 2 to 4 with all records preserved.

Documentation Update

none

Contributor's checklist

• Read through contributor's guide
• Enough context is provided in the sections above
• Adequate tests were added if applicable

[lokeshj1703]

@ad1happy2go Thanks for working on this! I have one minor comment below. Also it seems like HUDI-18161 is not accessible anymore. Can we update the reference?

[codecov-commenter]

Codecov Report

❌ Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.24%. Comparing base (782bf65) to head (ce8febd).
⚠️ Report is 15 commits behind head on master.

Files with missing lines	Patch %	Lines
...gleSparkJobConsistentHashingExecutionStrategy.java	0.00%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #18968      +/-   ##
============================================
- Coverage     68.26%   68.24%   -0.02%     
+ Complexity    29486    29478       -8     
============================================
  Files          2542     2542              
  Lines        142580   142541      -39     
  Branches      17781    17798      +17     
============================================
- Hits          97330    97284      -46     
- Misses        37237    37253      +16     
+ Partials       8013     8004       -9     

Flag	Coverage Δ
common-and-other-modules	44.77% <0.00%> (+0.02%)	⬆️
hadoop-mr-java-client	44.75% <ø> (+0.01%)	⬆️
spark-client-hadoop-common	48.06% <0.00%> (+<0.01%)	⬆️
spark-java-tests	48.76% <0.00%> (-0.03%)	⬇️
spark-scala-tests	44.80% <0.00%> (-0.05%)	⬇️
utilities	37.22% <0.00%> (-0.06%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...gleSparkJobConsistentHashingExecutionStrategy.java	91.35% <0.00%> (ø)

... and 77 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[hudi-bot]

CI report:

• ce8febd Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build

[hudi-agent]

🤖 This review was generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.

Thanks for the contribution! This PR relaxes the partition guard in SingleSparkJobConsistentHashingExecutionStrategy from !isNullOrEmpty to != null so consistent-hashing clustering can run on non-partitioned tables (whose partition path is the empty string), and folds non-partitioned coverage into the existing testResizing parameterized test. No correctness issues found. A few style/readability suggestions in the inline comments. Please take a look, and this should be ready for a Hudi committer or PMC member to take it from here. One minor naming nit in the new test helper method; the rest of the changes are clean and well-commented.

cc @yihua

[hudi-agent]

🤖 nit: could you rename a to something like origArgs or baseArgs? The single-letter name is a bit hard to follow when it's indexed three times on the lines below.

_{- AI-generated; verify before applying. React 👍/👎 to flag quality.}