Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Follow up to #10141.
When the two bitmaps have different lengths, intersect_masks and union_masks (parquet/src/arrow/arrow_reader/selection/boolean.rs) currently:
- slice both inputs to the common prefix,
- compute
& / | over the prefix (one allocation),
- append the prefix and the longer input's tail into a
BooleanBufferBuilder (a second allocation plus two copies).
As suggested by @alamb in #10141 (comment), this can be done with a single allocation: copy the longer buffer once, then apply &= / |= over the common prefix in place.
Describe the solution you'd like
Copy the longer buffer into a MutableBuffer and combine the shorter one into its prefix in place (e.g. via the bitwise helpers in arrow_buffer::buffer), avoiding the intermediate prefix buffer and the builder append copies. Care is needed for non-byte-aligned cases: the inputs may have a non-zero bit offset (e.g. produced by BooleanBuffer::slice), and the split point at the common prefix is not generally byte-aligned.
Describe alternatives you've considered
Keep the current implementation — it is correct and simple, this is purely an allocation/copy optimization.
Additional context
The tail-passthrough semantics (bits beyond the common length are taken from the longer input unchanged, for both intersect and union) are covered by existing unit tests in selection/boolean.rs; those tests should keep passing unchanged.
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Follow up to #10141.
When the two bitmaps have different lengths,
intersect_masksandunion_masks(parquet/src/arrow/arrow_reader/selection/boolean.rs) currently:&/|over the prefix (one allocation),BooleanBufferBuilder(a second allocation plus two copies).As suggested by @alamb in #10141 (comment), this can be done with a single allocation: copy the longer buffer once, then apply
&=/|=over the common prefix in place.Describe the solution you'd like
Copy the longer buffer into a
MutableBufferand combine the shorter one into its prefix in place (e.g. via the bitwise helpers inarrow_buffer::buffer), avoiding the intermediate prefix buffer and the builder append copies. Care is needed for non-byte-aligned cases: the inputs may have a non-zero bit offset (e.g. produced byBooleanBuffer::slice), and the split point at the common prefix is not generally byte-aligned.Describe alternatives you've considered
Keep the current implementation — it is correct and simple, this is purely an allocation/copy optimization.
Additional context
The tail-passthrough semantics (bits beyond the common length are taken from the longer input unchanged, for both intersect and union) are covered by existing unit tests in
selection/boolean.rs; those tests should keep passing unchanged.