Skip to content

[#11841] fix ClickHouse partition expression round trip#11878

Open
hutiefang76 wants to merge 1 commit into
apache:mainfrom
hutiefang76:codex/gravitino-11841-clickhouse-partition
Open

[#11841] fix ClickHouse partition expression round trip#11878
hutiefang76 wants to merge 1 commit into
apache:mainfrom
hutiefang76:codex/gravitino-11841-clickhouse-partition

Conversation

@hutiefang76

Copy link
Copy Markdown

What changes were proposed in this pull request?

Fix ClickHouse partition expression round-tripping for day transforms and nested date wrappers:

  • format Gravitino day transforms as toYYYYMMDD(column) instead of toDate(column)
  • parse toYYYYMMDD(column) back to a day transform
  • unwrap a supported toDate(...) wrapper when parsing expressions such as toYYYYMM(toDate(event_time))
  • update the ClickHouse catalog documentation for supported partition functions

Why are the changes needed?

Fixes #11841.

Without this change, a ClickHouse partition expression like toYYYYMM(toDate(event_time)) is parsed with toDate(event_time) as the field name. Also, day transforms are formatted as toDate(...), which does not round-trip cleanly with the ClickHouse day partition form described in the issue.

Does this PR introduce any user-facing change?

Yes. ClickHouse day partition transforms are now generated as toYYYYMMDD(column). Existing toDate(column) partition expressions are still accepted when reading metadata.

How was this patch tested?

  • JAVA_HOME=$(/usr/libexec/java_home -v 17) ./gradlew :catalogs-contrib:catalog-jdbc-clickhouse:test --tests org.apache.gravitino.catalog.clickhouse.operations.TestClickHouseTableOperationsPartitioning -PskipITs
  • JAVA_HOME=$(/usr/libexec/java_home -v 17) ./gradlew :catalogs-contrib:catalog-jdbc-clickhouse:spotlessCheck -PskipITs
  • git diff --check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug report] ClickHouse partition expression round-trip broken for day transforms and nested functions

1 participant