GH-50478: [C++][Compute] Support string_view/binary_view in scalar string predicate kernels by fangchenli · Pull Request #50479 · apache/arrow

fangchenli · 2026-07-11T19:51:25Z

Rationale for this change

The scalar string predicate/measurement kernels return NotImplemented for
string_view/binary_view input, so evaluating a string predicate (e.g. a
LIKE filter) on a view array first requires casting the whole column to
utf8/binary, a full copy. This closes that gap so the predicates run
directly on view arrays.

Part of the view-kernel effort (umbrella #44336, tracking #39634); follows
grouper keys (#50224) and cast (#50166). Take/filter (#43010) is handled
separately by #50164.

What changes are included in this PR?

Adds STRING_VIEW/BINARY_VIEW kernels for the input-view, fixed-output subset:

match_substring(_regex), match_like, starts_with, ends_with
find_substring(_regex), count_substring(_regex)
binary_length, utf8_length
string_is_ascii, ascii_is_*, utf8_is_*

Implementation notes:

The match-substring family and the classification predicates read every slot,
so their view paths (an if constexpr branch in MatchSubstringImpl and in
StringPredicateFunctor) iterate with VisitArrayValuesInline, which skips
null slots. This matters for correctness: a null slot's view header is not
validated (ValidateBinaryView skips nulls) and may carry a bogus
buffer_index/offset that decoding would dereference. The base binary/string
fast paths are unchanged.
The applicator-based kernels (find/count/length) already skip nulls, so
they only needed registration entries plus a small StringOffsetType helper
mapping the view types to Int32 output (a view element length is int32-sized).

Kernels that emit strings/lists (utf8_upper/lower/trim*,
replace_substring, split_pattern, utf8_slice_codeunits, …) are out of scope
for a follow-up, since constructing view output is the harder half.

Are these changes tested?

Yes. New TestStringViewPredicates.* cases in scalar_string_test.cc cover empty,
inlined (≤12 byte), out-of-line (>12 byte), sliced, and null values for both
binary_view and utf8_view, asserting results match the plain string/binary
semantics.

A MatchSubstringView / MatchSubstringViewCast benchmark pair quantifies
running the predicate directly on a view array vs. the pre-PR cast-to-utf8
workaround. Release build, ~1M rows of avg-16-byte ASCII, mean of 3 reps:

Benchmark	Time	Throughput
`MatchSubstring` (utf8, direct)	27.7 ms	572 MiB/s
`MatchSubstringView` (utf8_view, direct, this PR)	31.5 ms	503 MiB/s
`MatchSubstringViewCast` (view → cast to utf8 → match, pre-PR workaround)	42.2 ms	378 MiB/s

Evaluating the predicate directly on the view array is ~1.34x faster than
casting to utf8 first, by skipping the full-column copy. Direct view evaluation
is ~14% slower than native utf8 (inherent to the view layout's per-element
decode) but avoids the cast entirely.

Are there any user-facing changes?

Yes. These compute functions now accept string_view/binary_view input. The
compute.rst type-support tables are updated accordingly.

GitHub Issue: [C++][Compute] Support string_view/binary_view in scalar string predicate kernels #50478

github-actions · 2026-07-11T19:51:49Z

⚠️ GitHub issue #50478 has been automatically assigned in GitHub to PR creator.

Copilot

Pull request overview

This PR extends Arrow C++ scalar compute string predicate/measurement kernels to accept string_view/binary_view inputs so predicates can run directly on view arrays (avoiding a cast-and-copy to utf8/binary), and updates docs/tests/benchmarks accordingly.

Changes:

Register view-type kernels for substring/like predicates, find/count, length, and classification predicates.
Add unit tests covering view arrays (including slicing/chunking via the shared test harness) and add benchmarks comparing direct view evaluation vs cast-to-utf8 baseline.
Update C++ compute documentation to reflect new input type support and output type behavior for view inputs.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
docs/source/cpp/compute.rst	Documents added `StringView`/`BinaryView` support for relevant predicate/measurement kernels.
cpp/src/arrow/compute/kernels/scalar_string_utf8.cc	Registers `utf8_length` support for `utf8_view` (Int32 output).
cpp/src/arrow/compute/kernels/scalar_string_internal.h	Adds `utf8_view` registration for unary string predicates (classification/is_*) via `StringPredicateFunctor`.
cpp/src/arrow/compute/kernels/scalar_string_test.cc	Adds `TestStringViewPredicates.*` coverage for view inputs across several kernels.
cpp/src/arrow/compute/kernels/scalar_string_benchmark.cc	Adds benchmarks for `match_substring` on `utf8_view` and cast-to-utf8 workaround; refactors shared dataset generation.
cpp/src/arrow/compute/kernels/scalar_string_ascii.cc	Implements/Registers view support for match-substring predicates and find/count/length kernels, including `StringOffsetType` for view outputs.

…lar string predicate kernels Add STRING_VIEW/BINARY_VIEW support to the scalar string predicate and measurement kernels that read a string and emit a fixed-width type: match_substring(+regex), match_like, starts_with, ends_with, find_substring(+regex), count_substring(+regex), binary_length, utf8_length, string_is_ascii, and the ascii_is_*/utf8_is_* classifiers. The matcher/predicate logic already operates on std::string_view, so the change feeds view elements into the existing logic: MatchSubstringImpl gets a per-element ArrayIterator branch for the view layout (the packed-offset fast path stays for base binary/string), and the applicator-based kernels only need registration entries plus a StringOffsetType helper mapping views to Int32 output. Kernels that emit strings/lists are left for a follow-up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

zanmato1984 · 2026-07-13T21:06:50Z

+TEST(TestStringViewPredicates, MatchSubstringRegex) {
+  MatchSubstringOptions options{"a+"};
+  const auto* input = R"(["", "cat", null, "aaa banana", "concatenation"])";
+  for (const auto& ty : {binary_view(), utf8_view()}) {


Could we add one utf8_view case with ignore_case=true and non-ASCII input here (for example mirroring the existing MatchSubstringIgnoreCase coverage)?

The implementation intentionally registers utf8_view as StringViewType instead of using the generic BinaryViewType dispatch, so it would be good to have a test that would fail if we accidentally lost the UTF-8/Latin1 distinction. The current view-specific tests are mostly ASCII, so they wouldn't catch that regression.

Good call. Added TestStringViewPredicates.MatchSubstringIgnoreCase in 8082d31: it mirrors the existing MatchSubstringIgnoreCase coverage with pattern "aé(", ignore_case=true, and input ["abc", "aEb", "baÉ(", "aé(", "ae(", "Aé("] → [false, false, true, true, false, true]. The uppercase É matching lowercase é only holds because utf8_view folds over the full Unicode range; if it were dispatched through the generic BinaryViewType path (ASCII/Latin1 folding only) that slot would come back false, so the test fails on that regression.

…icates

…g UTF-8/Latin1 distinction Add a match_substring ignore_case=true case on utf8_view with non-ASCII input (uppercase E-acute) mirroring the existing MatchSubstringIgnoreCase coverage. utf8_view is registered as StringViewType rather than the generic BinaryViewType dispatch, so case folding spans the full Unicode range; this test fails if that distinction is ever lost. Requested in review by zanmato1984. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

zanmato1984 · 2026-07-14T18:00:04Z

+  ASSERT_OK_AND_ASSIGN(Datum matched, CallFunction("match_substring", {arr}, &contains));
+  AssertDatumsEqual(Datum(expected), matched);
+  ASSERT_OK_AND_ASSIGN(Datum ascii, CallFunction("string_is_ascii", {arr}));
+  AssertDatumsEqual(Datum(expected), ascii);


Optional test coverage nit: could we also exercise one fixed-width numeric-output kernel in NullSlotWithUnvalidatedHeader, e.g. find_substring or utf8_length? The current regression covers the two boolean-output paths, while several newly registered view kernels go through the applicator path. It looks null-safe too, but this would make the null-slot header invariant explicit across both paths.

Done in 26453db. Extended NullSlotWithUnvalidatedHeader to also call find_substring (→ [0, null]) and utf8_length (→ [2, null]) on the same crafted ["ok", <0xFF null header>] array, so the null-slot header invariant is now asserted on the applicator (ScalarUnaryNotNull) path as well as the two boolean-output paths.

…icates

Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

…eric-output kernel NullSlotWithUnvalidatedHeader covered only the two boolean-output view paths (match_substring, string_is_ascii). Add find_substring and utf8_length so the "do not decode a null slot's unvalidated header" invariant is asserted on the applicator (ScalarUnaryNotNull) path too. Requested in review by zanmato1984. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

zanmato1984

+1. Thanks for working on this!

zanmato1984 · 2026-07-24T23:08:17Z

@pitrou do you want to take another look?

pitrou · 2026-07-23T08:30:30Z

+      ArraySpan* out_arr = out->array_span_mutable();
+      FirstTimeBitmapWriter bitmap_writer(out_arr->buffers[1].data, out_arr->offset,
+                                          input.length);
+      VisitArrayValuesInline<Type>(


Note: I think ideally we would also use VisitArrayValuesInline for other binary types, because the matcher may be expensive and it's better to skip it on null entries.

(for example, if the array has 99% nulls, it would be wasteful to run a regexp matcher on each underlying empty slot)

pitrou · 2026-07-23T13:37:41Z

+// Baseline: the pre-PR workaround of casting the view column to utf8 first. The
+// gap vs. MatchSubstringView is the cast that direct view support avoids.
+static void MatchSubstringViewCast(benchmark::State& state) {


We don't care about benchmarking this IMHO. It was useful to judge this PR's usefulness, but it's not useful for measuring Arrow C++ performance.

pitrou · 2026-07-23T13:38:30Z

+// Register the view kernels for a match-substring-style predicate. Registered per
+// view type (not via GenerateVarBinaryViewBase) so utf8_view -> StringViewType keeps
+// the is_utf8 distinction used by ignore_case/regex folding.


I've opened #50615 as it's frankly a pity to duplicate kernel code just for the is_utf8 flag.

github-actions Bot added Component: C++ Component: Documentation awaiting review Awaiting review labels Jul 11, 2026

fangchenli force-pushed the gh-string-view-predicates branch 4 times, most recently from 4b94187 to 4e3a526 Compare July 12, 2026 02:44

fangchenli marked this pull request as ready for review July 12, 2026 15:16

fangchenli requested a review from pitrou as a code owner July 12, 2026 15:16

Copilot AI review requested due to automatic review settings July 12, 2026 15:16

Copilot started reviewing on behalf of fangchenli July 12, 2026 15:16 View session

Copilot AI reviewed Jul 12, 2026

View reviewed changes

Comment thread cpp/src/arrow/compute/kernels/scalar_string_ascii.cc

Comment thread cpp/src/arrow/compute/kernels/scalar_string_internal.h

fangchenli force-pushed the gh-string-view-predicates branch from 4e3a526 to 19583cc Compare July 12, 2026 16:44

Copilot AI review requested due to automatic review settings July 12, 2026 16:44

Copilot started reviewing on behalf of fangchenli July 12, 2026 16:45 View session

Copilot AI reviewed Jul 12, 2026

View reviewed changes

zanmato1984 reviewed Jul 13, 2026

View reviewed changes

github-actions Bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Jul 13, 2026

fangchenli and others added 2 commits July 13, 2026 21:03

Merge remote-tracking branch 'upstream/main' into gh-string-view-pred…

764dff6

…icates

Copilot AI review requested due to automatic review settings July 14, 2026 04:39

Copilot started reviewing on behalf of fangchenli July 14, 2026 04:40 View session

Copilot AI reviewed Jul 14, 2026

View reviewed changes

zanmato1984 reviewed Jul 14, 2026

View reviewed changes

Merge remote-tracking branch 'upstream/main' into gh-string-view-pred…

098c9ee

…icates

Copilot AI review requested due to automatic review settings July 24, 2026 19:39

Copilot started reviewing on behalf of fangchenli July 24, 2026 19:40 View session

Copilot AI reviewed Jul 24, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 24, 2026 21:12

Copilot started reviewing on behalf of fangchenli July 24, 2026 21:12 View session

Copilot AI reviewed Jul 24, 2026

View reviewed changes

zanmato1984 approved these changes Jul 24, 2026

View reviewed changes

pitrou reviewed Jul 27, 2026

View reviewed changes

Uh oh!

Conversation

fangchenli commented Jul 11, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actions Bot commented Jul 11, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

zanmato1984 Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

fangchenli Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

zanmato1984 Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

fangchenli Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

zanmato1984 left a comment

Choose a reason for hiding this comment

Uh oh!

zanmato1984 commented Jul 24, 2026

Uh oh!

pitrou Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

pitrou Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

pitrou Jul 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

fangchenli commented Jul 11, 2026 •

edited by github-actions Bot

Loading