What is the problem the feature request solves?
RLike::is_match in native/spark-expr/src/predicate_funcs/rlike.rs loops over a StringArray calling regex::Regex::is_match per row into a BooleanBuilder. Arrow's regexp_is_match kernel does the same thing: it is built on the same Rust regex crate, so the documented divergence from Java regex semantics is unchanged, and null propagation matches.
The kernel would also make the array path more robust: it supports Utf8/LargeUtf8/Utf8View, while the current code hard-expects a StringArray and would panic on other string layouts.
Describe the potential solution
Replace the row loop with regexp_is_match(array, &StringArray::new_scalar(pattern), None).
Trade-offs to resolve:
- The struct currently pre-compiles the
Regex once at plan build; the kernel takes a pattern string and compiles per batch. For complex patterns this is a measurable regression; there is no arrow API accepting a pre-built Regex. Either accept per-batch compilation or keep the loop and only fix the downcast robustness.
- The dictionary fast path (run on dictionary values then
take) and the scalar-input branch stay as wrappers.
Additional context
Found during an audit of native code that replicates existing arrow-rs kernels.
What is the problem the feature request solves?
RLike::is_matchinnative/spark-expr/src/predicate_funcs/rlike.rsloops over aStringArraycallingregex::Regex::is_matchper row into aBooleanBuilder. Arrow'sregexp_is_matchkernel does the same thing: it is built on the same Rustregexcrate, so the documented divergence from Java regex semantics is unchanged, and null propagation matches.The kernel would also make the array path more robust: it supports Utf8/LargeUtf8/Utf8View, while the current code hard-
expects aStringArrayand would panic on other string layouts.Describe the potential solution
Replace the row loop with
regexp_is_match(array, &StringArray::new_scalar(pattern), None).Trade-offs to resolve:
Regexonce at plan build; the kernel takes a pattern string and compiles per batch. For complex patterns this is a measurable regression; there is no arrow API accepting a pre-builtRegex. Either accept per-batch compilation or keep the loop and only fix the downcast robustness.take) and the scalar-input branch stay as wrappers.Additional context
Found during an audit of native code that replicates existing arrow-rs kernels.