Commit 666bec2
committed
Decouple PPL logical plans from UDT temporal types
Logical RelNode/RexNode trees now use standard Calcite DATE/TIME(9)/
TIMESTAMP(9) types for date columns. A new TemporalUdtRewriteShuttle
runs at the prepare-statement boundary (OpenSearchCalcitePreparingStmt
.implement and OpenSearchRelRunners.run) and rewrites the standard
temporal types back to UDTs (ExprDateType/ExprTimeType/ExprTimeStampType)
just before physical execution, so Linq4j keeps receiving the
VARCHAR-backed representation.
This commit folds in the prior "Decouple Calcite PPL planning from
ExprType" change as well: Calcite-side PPL planning (RelNode/RexNode
code, coercion, type checking, and UDF implementations) operates on
RelDataType throughout instead of bouncing through ExprType.
Type system:
- OpenSearchTypeFactory.convertExprTypeToRelDataType returns standard
TIMESTAMP(9)/DATE/TIME(9) for the corresponding ExprCoreType. IP and
BINARY remain UDT.
- New helper isStandardTemporalType.
- PPLOperandTypes constants renamed DATE_UDT/TIME_UDT/TIMESTAMP_UDT to
DATE_T/TIME_T/TIMESTAMP_T.
- UserDefinedFunctionUtils NULLABLE_*_UDT renamed to NULLABLE_*_T and
repointed to standard temporal RelDataTypes (IP and BINARY constants
unchanged).
- UDTs are recognised via instanceof rather than getExprType(); cloneWith
preserves UDT identity through createTypeWithNullability.
Cast emission and pushdown:
- CalciteRexNodeVisitor.visitCast emits standard temporal types for
AST DATE/TIME/TIMESTAMP cast targets.
- ExtendedRexBuilder.makeCast routes IP separately; standard temporal
targets dispatch to PPLBuiltinOperators.DATE/TIME/TIMESTAMP with the
standard target type so the call's RelDataType stays standard.
- PredicateAnalyzer.isTimestamp/isDate accept both standard SqlTypeName
and UDT, and read literal values via getValueAs(String.class) so
TimestampString/DateString round-trip without ClassCastException.
Coercion + type-checker:
- RelDataType-typed common-type resolver with a CoercionTag widening DAG.
- CoercionUtils' DATE+TIME -> TIMESTAMP resolver emits standard
TIMESTAMP(9).
- PPLTypeChecker gains a temporalKind helper used in typesMatch and
isComparable so standard and UDT temporal pairs match across forms.
- getRelDataTypes(family) returns standard temporal types for
DATETIME/TIMESTAMP/DATE/TIME families (BINARY family stays UDT).
- UDF return-type inference (AddSubDate/Weekday/LastDay/TimestampDiff
/Format/TimestampAdd/Extract/Span/WidthBucket) recognise both
standard and UDT temporal operand types.
Shuttle implementation:
- Atomic rebuild path for Project/Filter/Calc/Aggregate/Values nodes
(Calcite's default copy-then-validate path doesn't work because each
half of the rewrite would briefly violate row-type consistency).
- Filter rebuild preserves variablesSet so correlated subqueries keep
their CorrelationId binding.
- TemporalSchemaRewritable marker interface lets OpenSearch table
scans rewrite their schema in place rather than wrap in a
LogicalProject(CAST), so Calcite Linq4j codegen reads String values
matching what OpenSearchExprValueFactory delivers at runtime.
Other:
- DatetimeUdtNormalizeRule (api/datetime extension) recognises both
UDT and standard temporal RexCalls and forces precision to MAX so
unified-API consumers see TIMESTAMP(9).
- Calcite IT golden files updated to reflect logical plans now showing
TIMESTAMP(9) instead of EXPR_TIMESTAMP VARCHAR (physical plans still
show UDT post-shuttle).
Tests:
- New unit: TemporalUdtRewriteShuttleTest, CalcitePPLLogicalPlanStandardTemporalTest.
- Updated CoercionUtilsTest, OpenSearchTypeFactoryTest. All module unit
tests pass. Full Calcite IT suite (including ExplainIT) green.
Signed-off-by: Peng Huo <penghuo@gmail.com>1 parent 3d4938a commit 666bec2
56 files changed
Lines changed: 1791 additions & 1003 deletions
File tree
- api/src
- main/java/org/opensearch/sql/api/spec
- datetime
- test/java/org/opensearch/sql/api
- spec/datetime
- core/src
- main/java/org/opensearch/sql
- calcite
- plan/rel
- type
- utils
- expression/function
- udf
- binning
- datetime
- ip
- test/java/org/opensearch/sql
- calcite
- plan/rel
- utils
- expression/function
- integ-test/src/test
- java/org/opensearch/sql/calcite/remote
- resources/expectedOutput
- calcite_no_pushdown
- calcite
- opensearch/src
- main/java/org/opensearch/sql/opensearch
- functions
- request
- storage/scan
- test/java/org/opensearch/sql/opensearch/storage/serde
- ppl/src/test/java/org/opensearch/sql/ppl/calcite
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 1 addition & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
14 | 13 | | |
15 | 14 | | |
16 | 15 | | |
| |||
38 | 37 | | |
39 | 38 | | |
40 | 39 | | |
41 | | - | |
| 40 | + | |
42 | 41 | | |
43 | 42 | | |
Lines changed: 1 addition & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
20 | 19 | | |
21 | 20 | | |
22 | 21 | | |
| |||
52 | 51 | | |
53 | 52 | | |
54 | 53 | | |
55 | | - | |
| 54 | + | |
56 | 55 | | |
57 | 56 | | |
58 | 57 | | |
| |||
Lines changed: 0 additions & 49 deletions
This file was deleted.
Lines changed: 0 additions & 58 deletions
This file was deleted.
Lines changed: 0 additions & 13 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
171 | 171 | | |
172 | 172 | | |
173 | 173 | | |
174 | | - | |
175 | | - | |
176 | | - | |
177 | | - | |
178 | | - | |
179 | | - | |
180 | | - | |
181 | | - | |
182 | | - | |
183 | | - | |
184 | | - | |
185 | | - | |
186 | | - | |
187 | 174 | | |
188 | 175 | | |
189 | 176 | | |
| |||
Lines changed: 0 additions & 90 deletions
This file was deleted.
0 commit comments