Feature Category
SQL Function / Operator
Problem / Use Case
Apache Spark 3.5 provides bitmap_bit_position, but Bolt has no native Spark SQL implementation. Queries currently retain Spark's BitmapExpressionUtils.bitmapBitPosition invocation and fall back instead of running in Bolt.
Proposed Solution
Implement Spark-compatible bitmap_bit_position:
bitmap_bit_position(value: BIGINT) -> BIGINT
For Spark's 32,768-bit bucket size, behavior is:
value > 0 => (value - 1) % 32768
value <= 0 => (-value) % 32768
Add the Gluten/Substrait expression mapping and register the function for Spark-compatible Bolt builds. Preserve Spark null behavior and implicit casting to BIGINT.
References / Prior Art
- Apache Spark 3.5
BitmapBitPosition
- Apache Spark 3.5
BitmapExpressionUtils.bitmapBitPosition
- Spark
BitmapExpressionsQuerySuite case bitmap_bit_position
Acceptance Criteria
- Add native Bolt tests for positive, zero, negative, boundary, null, and cast inputs.
- Match Spark 3.5 results.
- Execute the corresponding Gluten query with a native project transformer.
Importance
Medium
Feature Category
SQL Function / Operator
Problem / Use Case
Apache Spark 3.5 provides
bitmap_bit_position, but Bolt has no native Spark SQL implementation. Queries currently retain Spark'sBitmapExpressionUtils.bitmapBitPositioninvocation and fall back instead of running in Bolt.Proposed Solution
Implement Spark-compatible
bitmap_bit_position:For Spark's 32,768-bit bucket size, behavior is:
Add the Gluten/Substrait expression mapping and register the function for Spark-compatible Bolt builds. Preserve Spark null behavior and implicit casting to
BIGINT.References / Prior Art
BitmapBitPositionBitmapExpressionUtils.bitmapBitPositionBitmapExpressionsQuerySuitecasebitmap_bit_positionAcceptance Criteria
Importance
Medium