Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
764023f
Special form v2 initial implementation
zanmato1984 Mar 21, 2025
4322054
WIP
zanmato1984 Aug 1, 2025
2ac65af
Support selective kernel exec and move dense execution to executor
zanmato1984 Aug 2, 2025
66f85ae
Remove the concept of selection vector awareness
zanmato1984 Aug 2, 2025
ce6d2ff
Cleanup dense things in special form
zanmato1984 Aug 2, 2025
e730560
Remove selection vector span from exec span, and let kernel have a de…
zanmato1984 Aug 4, 2025
976efaa
Fix selection vector span to forward for split exec span
zanmato1984 Aug 4, 2025
f1399e6
Rename
zanmato1984 Aug 4, 2025
217493f
Deep refine special form and executor interfaces
zanmato1984 Aug 6, 2025
1b83450
Abstract dispatch for bind logic
zanmato1984 Aug 6, 2025
0e19231
Fix build benchmark
zanmato1984 Aug 7, 2025
a6491d9
Rename backstep
zanmato1984 Aug 13, 2025
94871c9
Add selection vector span test
zanmato1984 Aug 13, 2025
f5dc732
Small fix on selection span
zanmato1984 Aug 14, 2025
3785d7c
Add selection span iteration test
zanmato1984 Aug 14, 2025
3644aef
Add selective function execution test part 1
zanmato1984 Aug 14, 2025
96e9076
Add real dense execution cases
zanmato1984 Aug 14, 2025
fa7b95a
Split chunked case for dense caller
zanmato1984 Aug 15, 2025
9802cfc
Refine caller test
zanmato1984 Aug 15, 2025
1cbb45c
Change to decent selective kernels
zanmato1984 Aug 18, 2025
52d0629
Reorg existing tests
zanmato1984 Aug 18, 2025
4f4c81b
Add selection argument to function caller
zanmato1984 Aug 18, 2025
dabdf5d
Add partial for sparse and dense cases
zanmato1984 Aug 19, 2025
14a5523
Test finished?
zanmato1984 Aug 19, 2025
e2d865f
More clear naming
zanmato1984 Aug 20, 2025
cd0d5b8
Refine binding
zanmato1984 Sep 3, 2025
69f9d1a
Revert something
zanmato1984 Sep 3, 2025
b718bb8
Fix rebase error: missed constraint
zanmato1984 Sep 5, 2025
624d674
Bind special form test
zanmato1984 Sep 5, 2025
a7096f7
Basic expression tests for special form
zanmato1984 Sep 5, 2025
410a636
Add special form execution test
zanmato1984 Sep 7, 2025
c6d733a
Add more special form test using echo_special
zanmato1984 Sep 7, 2025
0c22f95
Move if_else special form from compute core to compute lib (#58)
zanmato1984 Oct 3, 2025
496c2ed
Make header for conditional stuff so we can test it (#59)
zanmato1984 Oct 4, 2025
bd3dfa5
Merge
zanmato1984 Oct 7, 2025
1a6b828
Port
zanmato1984 Oct 7, 2025
21badec
Fix all scalar execbatch not match selection vector
zanmato1984 Oct 7, 2025
780738e
Some branch mask tests
zanmato1984 Oct 7, 2025
ee0ac7a
Some tests
zanmato1984 Oct 7, 2025
d559fb9
Simplify branch and body masks and complete most tests (#60)
zanmato1984 Oct 9, 2025
1d9dca2
Conditional exec tests
zanmato1984 Oct 9, 2025
4906ef2
More shortcut body mask generation
zanmato1984 Oct 9, 2025
7ba2013
Refine
zanmato1984 Oct 10, 2025
046dde5
Refine
zanmato1984 Oct 10, 2025
84f643b
Refine
zanmato1984 Oct 10, 2025
f9f3031
Pre-specified selection vector tests
zanmato1984 Oct 10, 2025
148c5c8
Fine-grained selection vector check
zanmato1984 Oct 10, 2025
509c9b9
Refine
zanmato1984 Oct 10, 2025
7937232
Reorg
zanmato1984 Oct 11, 2025
b4b7d76
Fix uninitialized seletective exec
zanmato1984 Oct 14, 2025
69a5dbb
WIP if else special tests
zanmato1984 Oct 14, 2025
52627a5
Fix
zanmato1984 Oct 14, 2025
b50a874
Reorder tests
zanmato1984 Oct 14, 2025
efbd809
Refine if else tests
zanmato1984 Oct 15, 2025
0c655b7
WIP
zanmato1984 Oct 15, 2025
73d9be8
Almost done
zanmato1984 Oct 15, 2025
35d6118
Complete the test
zanmato1984 Nov 5, 2025
a399f48
Rewrite benchmark
zanmato1984 Nov 7, 2025
8bf870c
Default params
zanmato1984 Nov 10, 2025
e5b99f8
Add comment of overall rational for special form
zanmato1984 Nov 13, 2025
b1bd847
Refine overall special form comment
zanmato1984 Nov 14, 2025
c98ace8
Comment on special form and executor
zanmato1984 Nov 14, 2025
51546dc
Update branch abstraction doc
zanmato1984 Nov 15, 2025
c4e5d5c
Small refine and more comments
zanmato1984 Nov 15, 2025
89b6b20
DCHECK for the last branch remaining no rows
zanmato1984 Nov 18, 2025
bb89a96
Comments for body mask impls
zanmato1984 Nov 18, 2025
c1dff53
Comment on function backed special form crtp
zanmato1984 Nov 19, 2025
cdd6824
More comment
zanmato1984 Nov 20, 2025
a16d448
Refine and comment
zanmato1984 Nov 20, 2025
18e56fd
More comment
zanmato1984 Nov 20, 2025
91c417c
More comments
zanmato1984 Nov 20, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions cpp/src/arrow/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -802,6 +802,8 @@ if(ARROW_COMPUTE)
compute/row/grouper.cc
compute/row/row_encoder_internal.cc
compute/row/row_internal.cc
compute/special/conditional_special.cc
compute/special/if_else_special.cc
compute/util.cc
compute/util_internal.cc)

Expand Down
2 changes: 2 additions & 0 deletions cpp/src/arrow/compute/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -179,3 +179,5 @@ add_arrow_compute_benchmark(function_benchmark)
add_subdirectory(kernels)

add_subdirectory(row)

add_subdirectory(special)
6 changes: 6 additions & 0 deletions cpp/src/arrow/compute/api.h
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,14 @@
#include "arrow/compute/registry.h" // IWYU pragma: export
#include "arrow/datum.h" // IWYU pragma: export

/// \defgroup expressions Expressions of computation
/// @{
/// @}

#include "arrow/compute/expression.h" // IWYU pragma: export

#include "arrow/compute/api_special.h" // IWYU pragma: export

/// \defgroup execnode-row Utilities for working with data in a row-major format
/// @{
/// @}
Expand Down
52 changes: 52 additions & 0 deletions cpp/src/arrow/compute/api_special.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

#include "arrow/compute/expression.h"

namespace arrow::compute {

/// The concept of "special form" is borrowed from Lisp
/// (https://courses.cs.northwestern.edu/325/readings/special-forms.html). A special form
/// is used to implement evaluation strategies
/// (https://en.wikipedia.org/wiki/Evaluation_strategy) other than the default
/// call-by-value strategy used by Arrow expression evaluation. Velox also uses this term.
///
/// In a call-by-value strategy, all arguments are evaluated before the function
/// invocation. For example, consider a regular function call
/// if_else(a, b, c)
/// Under call-by-value semantics, the expressions `a`, `b`, and `c` are all evaluated
/// before calling `if_else`. This can lead to unintuitive behavior when subexpressions
/// have observable side effects. For instance,
/// if_else(not_equal(a, 0), divide(b, a), 0)
/// should never produce a divide-by-zero error in most programming languages. However,
/// under call-by-value semantics, `divide(b, a)` is evaluated regardless of the
/// condition, so a divide-by-zero error can still occur. To address this, a special form
/// for `if_else` would be needed, namely `if_else_special`, that follows a
/// call-by-name-like evaluation strategy, where, for each row in a batch, only one of the
/// branches is evaluated based on the corresponding value of condition.
///
/// Each API in this file is intended to refer to a concrete special form. In addition to
/// the aforementioned `if_else_special`, the design anticipates variants of conditional
/// constructs such as `case_when_special` and `coalesce_special`, as well as boolean
/// operators with short-circuit semantics, such as `and_special` and `or_special`, some
/// of which may not be implemented yet.

/// @brief Construct an Expression representing an if-else special form.
ARROW_EXPORT
Expression if_else_special(Expression cond, Expression if_true, Expression if_false);

} // namespace arrow::compute
Loading
Loading