-
Notifications
You must be signed in to change notification settings - Fork 368
Open
Labels
Description
Search before asking
- I have searched in the issues and found no similar issues.
What would you like to be improved?
IcebergTableMaintainer is currently located in the amoro ams module and directly depends on the DefaultTableRuntime class, resulting in the following issues:
- Unclear module responsibilities: The amoro format iceberg module should include all Iceberg format related implementations, but the implementation of TableMaintainer is coupled with AMS
- Difficulty in code reuse: If other projects need to use Amoro's Iceberg maintenance feature, the entire AMS module must be introduced
- High testing complexity: Testing for maintenance functions requires simulating the entire AMS runtime environment
- Scalability limited: In the future, if we want to support other formats (such as Paimon/Hudi), we will need to repeatedly implement similar AMS integration logic
Before
┌─────────────────────────────────────────────────────────────────┐
│ IcebergTableMaintainer │
│ (amoro-ams) │
└───────────────────────────┬─────────────────────────────────────┘
│
│ dependency
▼
┌─────────────────────────────────────────────────────────────────┐
│ DefaultTableRuntime │
│ (amoro-ams) │
└─────────────────────────────────────────────────────────────────┘
Add TableMaintainerContext interface to the amoro common module as an abstraction layer for maintainer and AMS runtime interaction:
After
┌──────────────────┐
│ amoro-common │
│ - TableMaintainer
│ - TableMaintainerContext
│ - MaintainerMetrics
│ - OptimizingInfo
└────────┬─────────┘
│
│ dependency
▼
┌──────────────────────┐
│ amoro-format-iceberg │ (add dependency: amoro-common)
│ - IcebergTableMaintainer
└────────┬─────────────┘
│
│
▼
┌──────────────────┐
│ amoro-ams │
│ - DefaultTableMaintainerContext
│ - DefaultTableRuntime
└──────────────────┘
How should we improve?
- TableMaintainer interface remove to amoro-common
- Add TableMaintainerContext、MaintainerMetrics、OptimizingInfo interface
- Impl DefaultTableMaintainerContext
- MixedTableMaintainer & IcebergTableMaintainer remove to amoro-iceberg
- Iceberg Maintainer Test from ams move to amoro-iceberg (executor can't remove to amoro-iceberg,execute in ams and tests)
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Subtasks
No response
Code of Conduct
- I agree to follow this project's Code of Conduct