Skip to content

fix(mdt): use native log format for file group initialization#19312

Open
cshuo wants to merge 1 commit into
apache:masterfrom
cshuo:fix_mdt_initialization
Open

fix(mdt): use native log format for file group initialization#19312
cshuo wants to merge 1 commit into
apache:masterfrom
cshuo:fix_mdt_initialization

Conversation

@cshuo

@cshuo cshuo commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Describe the issue this Pull Request addresses

Closes #19311

When native log format is enabled, MDT file-group initialization still creates a legacy inline log containing an empty delete block. Empty file groups therefore do not have the native .log.hfile expected by the file-system view and may not be discovered before receiving records.

Summary and Changelog

  • Create a zero-record native .log.hfile for each empty MDT file group when native log format is enabled.
  • Store the instant time and writer schema in native footer metadata; the native log version is added by the footer metadata utility.
  • Keep the existing empty delete-block initialization for legacy inline log format.
  • Centralize both initialization paths in HoodieMetadataWriteUtils.
  • Add a test that opens the generated HFile and verifies its footer metadata.

Impact

Empty MDT file groups are discoverable by the file-system view when native log format is enabled. There are no public API or configuration changes.

Risk Level

Low. The change is limited to initial empty MDT file-group creation, preserves the legacy inline path, and includes native HFile footer read coverage.

Documentation Update

None. There are no user-facing API or configuration changes.

Contributor's checklist

  • Read through contributor's guide
  • Enough context is provided in the sections above
  • Adequate tests were added if applicable

@cshuo
cshuo marked this pull request as ready for review July 17, 2026 08:42
@hudi-bot

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@github-actions github-actions Bot added the size:M PR with lines of changes in (100, 300] label Jul 17, 2026
@cshuo
cshuo force-pushed the fix_mdt_initialization branch from 2d02cd6 to 1b264dc Compare July 18, 2026 08:00
@cshuo
cshuo force-pushed the fix_mdt_initialization branch from 1b264dc to dd375e0 Compare July 20, 2026 01:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M PR with lines of changes in (100, 300]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Initialize empty MDT file groups with native log files

2 participants