Skip to content

fix: accept LargeUtf8 and Utf8View patterns in SIMILAR TO planning #23735

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
adriangb merged 3 commits into from
Jul 29, 2026
+43 −7
Merged

fix: accept LargeUtf8 and Utf8View patterns in SIMILAR TO planning #23735

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f4fbbc7
fix: accept LargeUtf8 and Utf8View patterns in SIMILAR TO planning
u70b3 Jul 21, 2026
1906011
test: add NULL pattern cases for SIMILAR TO planning
u70b3 Jul 21, 2026
9e61c37
fix: delegate SIMILAR TO validation to analyzer
u70b3 Jul 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions datafusion/sql/src/expr/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1008,10 +1008,6 @@ impl<S: ContextProvider> SqlToRel<'_, S> {
planner_context: &mut PlannerContext,
) -> Result<Expr> {
let pattern = self.sql_expr_to_logical_expr(pattern, schema, planner_context)?;
let pattern_type = pattern.get_type(schema)?;
if pattern_type != DataType::Utf8 && pattern_type != DataType::Null {
return plan_err!("Invalid pattern in SIMILAR TO expression");
}
let escape_char = match escape_char.map(|v| v.value) {
Some(Value::SingleQuotedString(char)) if char.len() == 1 => {
Some(char.chars().next().unwrap())
Expand Down
46 changes: 43 additions & 3 deletions datafusion/sqllogictest/test_files/type_coercion.slt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -304,7 +304,9 @@ query error does not support zero arguments
SELECT * FROM (SELECT 1) WHERE TRY_CAST(STARTS_WITH() AS INT) = 1;

###################################################################
## SIMILAR TO type coercion (https://github.com/apache/datafusion/issues/22886)
## SIMILAR TO type coercion
## https://github.com/apache/datafusion/issues/22886
## https://github.com/apache/datafusion/issues/23732
###################################################################

# NULL pattern is coerced to a typed NULL and evaluates to NULL instead of panicking
Expand Down Expand Up @@ -349,6 +351,44 @@ SELECT arrow_cast(t.s, 'Dictionary(Int32, Utf8)') SIMILAR TO p.pat FROM t CROSS
----
true

# non-scalar string-like patterns are coerced by the analyzer
query B
SELECT t.s SIMILAR TO arrow_cast(p.pat, 'Utf8View') FROM t CROSS JOIN p;
----
true

query B
SELECT t.s SIMILAR TO arrow_cast(p.pat, 'LargeUtf8') FROM t CROSS JOIN p;
----
true

query B
SELECT t.s NOT SIMILAR TO arrow_cast(p.pat, 'Utf8View') FROM t CROSS JOIN p;
----
false

@adriangb adriangb Jul 21, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add tests for teh null case?

@u70b3 u70b3 Jul 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in add3f8d — added a literal NULL pattern in the table context and a non-scalar Null-typed pattern column (CREATE TABLE pn AS SELECT NULL AS pat), both evaluate to NULL.

query B
SELECT t.s SIMILAR TO arrow_cast(p.pat, 'Dictionary(Int32, Utf8)') FROM t CROSS JOIN p;
----
true

# NULL patterns (literal or Null-typed non-scalar) evaluate to NULL
query B
SELECT t.s SIMILAR TO NULL FROM t;
----
NULL

statement ok
CREATE TABLE pn AS SELECT NULL AS pat;

query B
SELECT t.s SIMILAR TO pn.pat FROM t CROSS JOIN pn;
----
NULL

statement ok
DROP TABLE pn;

statement ok
DROP TABLE t;

Expand All @@ -359,6 +399,6 @@ DROP TABLE p;
query error There isn't a common type to coerce Int64 and Utf8 in SIMILAR TO expression
SELECT 1 SIMILAR TO 'a';

# a non-string pattern is rejected even earlier, during SQL planning
query error Invalid pattern in SIMILAR TO expression
# a non-string pattern is rejected by the analyzer
query error There isn't a common type to coerce Utf8 and Int64 in SIMILAR TO expression
SELECT 'a' SIMILAR TO 1;
Loading