Skip to content

feat: add system table framework with options table#261

Open
suxiaogang223 wants to merge 9 commits intoalibaba:mainfrom
suxiaogang223:codex/system-table-pr1
Open

feat: add system table framework with options table#261
suxiaogang223 wants to merge 9 commits intoalibaba:mainfrom
suxiaogang223:codex/system-table-pr1

Conversation

@suxiaogang223
Copy link
Copy Markdown
Contributor

@suxiaogang223 suxiaogang223 commented Apr 30, 2026

Related to #141.

Summary

  • Add Java-aligned system table identifier parsing for table$system_table and reserved branch form table$branch_xxx$system_table
  • Add a lightweight system table framework with loader, scan, read, schema, and in-memory split support
  • Implement the first metadata system table: table$options
  • Route catalog schema/existence and scan/read paths through the system table framework
  • Keep system table DDL unsupported

Tests

  • Added identifier parsing tests for normal tables, system tables, branch tables, and invalid $ combinations
  • Added catalog tests for table$options existence/schema and system table DDL rejection
  • Added scan/read integration coverage for reading table$options
  • Verified on Fedora:
    • cmake --build build-codex-system-table-pr1 --target paimon-core-test -j$(nproc)
    • ./build-codex-system-table-pr1/release/paimon-core-test --gtest_filter="IdentifierTest.*:FileSystemCatalogTest.TestOptionsSystemTableCatalog:TableReadTest.TestReadOptionsSystemTable"
  • Ran formatting checks:
    • clang-format
    • cmake-format
    • pre-commit clang-format
    • pre-commit cmake-format

@suxiaogang223
Copy link
Copy Markdown
Contributor Author

@lxy-9602 Hi, could you please help review this PR when you have time? This is the first step for system table support, adding the shared framework and the minimal table$options implementation. I’d like to continue with the follow-up system table work after this direction is reviewed. Thanks!🤓

Comment thread src/paimon/core/catalog/identifier.cpp
Comment thread src/paimon/core/catalog/identifier_test.cpp Outdated
Comment thread src/paimon/core/catalog/identifier_test.cpp Outdated
Comment thread src/paimon/core/table/source/split.cpp Outdated
Comment thread src/paimon/core/table/system/system_table.h
Comment thread src/paimon/core/table/system/system_table.cpp
Comment thread src/paimon/core/table/system/system_table_scan.h
Comment thread src/paimon/core/table/system/system_table_schema.h
Comment thread src/paimon/core/table/system/options_system_table.h
Comment thread src/paimon/core/table/system/options_system_table.cpp Outdated
@lxy-9602
Copy link
Copy Markdown
Collaborator

lxy-9602 commented May 4, 2026

Thanks for this great contribution — the code is clean, well-structured, and clearly reflects the effort you've made to bridge C++ and Java differences.

Looking forward to your follow-up updates and more system table support!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants