feat(sparksql): support bitmap_bit_position

[felix-windsor]

What problem does this PR solve?

Bolt does not currently register Spark 3.5's bitmap_bit_position, so queries using the bitmap aggregation pipeline cannot execute this scalar step natively.

Issue Number: close #758

Type of Change

• 🐛 Bug fix (non-breaking change which fixes an issue)
• ✨ New feature (non-breaking change which adds functionality)
• 🚀 Performance improvement (optimization)
• ⚠️ Breaking change (fix or feature which changes existing behavior)
• 🔨 Refactoring (no logic changes)
• 🔧 Build/CI or Infrastructure changes
• 📝 Documentation only

Description

• add a Spark-compatible bitmap_bit_position(BIGINT) -> BIGINT scalar function
• register it through the Spark SQL function registry
• preserve NULL propagation and BIGINT cast behavior
• handle INT64_MIN through unsigned magnitude arithmetic, avoiding signed-overflow undefined behavior while matching Java/Spark results
• cover positive, zero, negative, bucket-boundary, integer-limit, NULL, and cast inputs

Performance Impact

• No Impact: this only adds a new scalar function and does not alter existing execution paths.
• Positive Impact: I have run benchmarks.
• Negative Impact: Explained below.

Release Note

Release Note:
- Added Spark SQL `bitmap_bit_position` support.

Validation

• clang-format 14.0.6 --dry-run --Werror on all changed C++ files
• git diff --check
• standalone Clang 17 build with ASan and UBSan exercising the same Spark boundary cases, including INT64_MIN and INT64_MAX
• the native Bolt test target is included for CI; a full local build was not completed because a clean macOS/ARM64 setup requires building the full LLVM/Arrow/Folly dependency graph from source

Checklist (For Author)

• I have added/updated unit tests (ctest).
• I have verified the code with local build (Release/Debug).
• I have run clang-format / linters.
• (Optional) I have run Sanitizers (ASAN/TSAN) locally for complex C++ changes.
• No need to test or manual test.

Breaking Changes

• No
• Yes

[Copilot]

Pull request overview

This PR adds native Spark SQL support for Spark 3.5’s bitmap_bit_position(BIGINT) -> BIGINT, enabling bitmap aggregation pipelines to execute the scalar step in Bolt instead of falling back to Spark.

Changes:

• Implemented BitmapBitPositionFunction with Spark-compatible semantics, including safe handling of INT64_MIN.
• Registered bitmap_bit_position in the Spark SQL function registry.
• Added unit tests covering positive, non-positive, NULL, and casted inputs.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
bolt/functions/sparksql/tests/CMakeLists.txt	Adds the new bitmap scalar function test file to the Spark SQL test target.
bolt/functions/sparksql/tests/BitmapTest.cpp	New unit tests validating bitmap_bit_position behavior across edge cases.
bolt/functions/sparksql/registration/RegisterBitmap.cpp	Registers the new scalar function with the Spark SQL registry.
bolt/functions/sparksql/registration/Register.cpp	Hooks bitmap function registration into the overall Spark SQL registration flow.
bolt/functions/sparksql/registration/CMakeLists.txt	Adds RegisterBitmap.cpp to the Spark functions registration library build.
bolt/functions/sparksql/Bitmap.h	Implements BitmapBitPositionFunction and bitmap bucket-size constant.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.