Skip to content

feat: add global system tables framework under sys database#366

Draft
suxiaogang223 wants to merge 6 commits into
alibaba:mainfrom
suxiaogang223:codex/global-system-tables
Draft

feat: add global system tables framework under sys database#366
suxiaogang223 wants to merge 6 commits into
alibaba:mainfrom
suxiaogang223:codex/global-system-tables

Conversation

@suxiaogang223

Copy link
Copy Markdown
Contributor

Summary

Add global system table support under the sys database, enabling catalog-level metadata queries.

Closes part of #141.

Changes

Infrastructure

  • Add GetOptions() to Catalog interface for catalog-level config access
  • FileSystemCatalog stores and exposes catalog options
  • New GlobalSystemTableLoader with independent registry for sys tables
  • Extend SystemTablePath with is_global flag and TryParsePath to detect sys/ paths
  • FileSystemCatalog handles sys database in ListTables/DatabaseExists/TableExists/LoadTableSchema

Global System Tables

Table Status Columns
sys.catalog_options ✅ Complete key, value
sys.all_table_options ✅ Complete database_name, table_name, key, value
sys.tables ✅ Basic info ready database_name, table_name, table_type, partitioned, primary_key, record_count; file stats columns (file_size_in_bytes, file_count, last_file_creation_time) return null
sys.partitions 🔶 Skeleton Schema defined; returns empty result set

Files Changed

 include/paimon/catalog/catalog.h                   |   5 +
 src/paimon/CMakeLists.txt                          |   1 +
 src/paimon/core/catalog/catalog.cpp                |   2 +-
 src/paimon/core/catalog/file_system_catalog.cpp    |  39 ++-
 src/paimon/core/catalog/file_system_catalog.h      |   5 +-
 .../core/table/system/global_system_tables.cpp     | 370 +++++++++++++++++++++
 .../core/table/system/global_system_tables.h       | 117 +++++++
 src/paimon/core/table/system/system_table.cpp      |  28 +-
 src/paimon/core/table/system/system_table.h        |   2 +
 .../core/catalog/file_system_catalog_test.cpp      |  14 +-
 10 files changed, 573 insertions(+), 10 deletions(-)

Dependency on #309 (codex/system-table-files-manifests-pr4)

The ManifestFile / ManifestList reading infrastructure from #309 is needed to complete:

  • sys.tables: populate file_size_in_bytes, file_count, last_file_creation_time by aggregating manifest entries from the latest snapshot
  • sys.partitions: read manifest entries grouped by partition to output per-partition statistics

Once #309 lands, the remaining TODO items in global_system_tables.cpp can be resolved in a follow-up PR.

🤖 Generated with Claude Code

suxiaogang223 and others added 6 commits June 13, 2026 18:18
- Add GetOptions() to Catalog interface for catalog-level config access
- FileSystemCatalog stores and exposes catalog_options
- New GlobalSystemTableLoader with independent registry for sys tables
- Implement sys.catalog_options, sys.all_table_options, sys.tables
- Stub for sys.partitions (manifest aggregation to follow)
- Extend SystemTablePath with is_global flag
- TryParsePath detects sys/ paths for TableScan/TableRead routing
- FileSystemCatalog handles sys database in ListTables, DatabaseExists,
  TableExists, LoadTableSchema

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant