Skip to content

feat(datafusion): support the rename_branch procedure - #942

Open
jackylee-ch wants to merge 3 commits into
apache:mainfrom
jackylee-ch:feat/datafusion-rename-branch-procedure
Open

jackylee-ch wants to merge 3 commits into
apache:mainfrom
jackylee-ch:feat/datafusion-rename-branch-procedure

Conversation

@jackylee-ch

Copy link
Copy Markdown
Contributor

Java exposes sys.rename_branch(table, from_branch, to_branch) to rename a branch. The core BranchManager::rename_branch primitive already existed; only the DataFusion CALL procedure was missing.

This registers rename_branch: it declares the table/from_branch/to_branch parameters so a misspelled argument is rejected, and dispatches to BranchManager, which already validates that the source branch exists, the target name is free, and neither is main.

Tested end-to-end: seed a branch, CALL rename_branch, assert the old name is gone from $branches and the new one is present; renaming a missing branch errors.

@JingsongLi

Copy link
Copy Markdown
Contributor

Requirement fit: SUPPORTED; the SQL procedure provides a real end-to-end branch operation. However, P1: reject path separators in branch names before exposing rename via SQL. BranchManager::validate_branch_name accepts foo/bar. I reproduced this on the PR head: create branches b2 and foo, then run CALL sys.rename_branch(table => 'test_db.t1', from_branch => 'b2', to_branch => 'foo/bar'). The call succeeds, the old branch is gone, but SELECT * FROM t1$branches WHERE branch_name = 'foo/bar' returns zero rows: the directory was moved under branch-foo/bar and is no longer listed as a branch. This can silently hide a branch and its data from normal discovery. Please reject / (and .. path components) at the shared branch-name validation boundary, and add a SQL regression asserting the call fails and the original branch remains visible. The existing normal rename integration test passes; all 14 CI checks are green, but they do not cover this case. I ran the reproducer in a disposable local test directory and restored it afterward.

jackylee-ch added a commit to jackylee-ch/paimon-rust that referenced this pull request Sep 25, 2026
Review follow-up (apache#942): `BranchManager::validate_branch_name` accepted a
rename target containing a path separator such as `foo/bar`, so
`sys.rename_branch` moved the branch to a nested path (`branch-foo/bar`) that
`$branches` never lists back — a silently orphaned branch. Reject '/' and '\\'
in branch names, alongside the existing main/blank/numeric checks.
Java exposes `sys.rename_branch(table, from_branch, to_branch)` to rename a
branch. The core BranchManager::rename_branch primitive already existed;
only the DataFusion procedure was missing.

Register rename_branch: declare its table/from_branch/to_branch parameters
(so a misspelled argument is rejected, like Java's binding) and dispatch to
BranchManager, which validates that the source branch exists, the target
name is free, and neither is main.
Review follow-up (apache#942): `BranchManager::validate_branch_name` accepted a
rename target containing a path separator such as `foo/bar`, so
`sys.rename_branch` moved the branch to a nested path (`branch-foo/bar`) that
`$branches` never lists back — a silently orphaned branch. Reject '/' and '\\'
in branch names, alongside the existing main/blank/numeric checks.
Mirror the create_branch regression for rename: after creating `b1`,
`CALL sys.rename_branch(..., to_branch => 'foo/bar')` must fail with the
path-separator error, leave `b1` in place (still listed by `$branches`),
and not produce a `foo/bar` branch.
@jackylee-ch
jackylee-ch force-pushed the feat/datafusion-rename-branch-procedure branch from 07967cf to af87f57 Compare September 30, 2026 12:04
@jackylee-ch

Copy link
Copy Markdown
Contributor Author

Addressed. BranchManager::validate_branch_name now rejects a target name containing / or \ at the shared validation boundary, so rename_branch (and create_branch/create_branch_from_tag) fail before any snapshot or schema file is moved and the source branch is left intact. Your reproducer now returns the path-separator error instead of silently relocating the branch under branch-foo/bar.

On the .. path component: rejecting / and \ already precludes it. A branch name is placed at the single path segment branch-<name>, so with no separator allowed the name can never introduce a new segment, and .. can only act as a traversal component when it stands alone between separators. A literal name like .. maps to the ordinary directory branch-.., which stays discoverable and cannot escape branch/. I deliberately did not blanket-reject the .. substring, since that would also reject legitimate names such as v1..v2.

Added the SQL regression test_rename_branch_rejects_path_separator: after creating b1, CALL sys.rename_branch(table => 'test_db.t1', from_branch => 'b1', to_branch => 'foo/bar') fails with the path-separator error, b1 is still listed by t1$branches, and no foo/bar branch is produced. The guard fires before the filesystem is touched, so it holds whether or not a foo parent branch already exists. I verified the test is non-vacuous: neutering the check makes the rename return Ok and the test fail with "expected error, got Ok"; restoring it passes.

The validate_branch_name hardening is the same one-line boundary shared with #939 and #941; whichever lands first, the others rebase cleanly by dropping the duplicate commit. Rebased onto current main. procedures (34 tests) and branch_manager (18 tests) pass; clippy -p paimon -p paimon-datafusion --all-targets -D warnings is clean.

@JingsongLi

Copy link
Copy Markdown
Contributor

The path-separator regression is fixed and the existing 34 SQL procedure tests plus 18 branch-manager tests pass. I also verified a tag-born branch retains its data after an ordinary SQL rename and can be queried via t1$branch_b2.

[P1] Validate the source branch name before moving its directory. rename_branch validates the target but only checks filesystem existence for the source. After creating a tag-born b1, CALL sys.rename_branch(table => 'test_db.t1', from_branch => 'b1/schema', to_branch => 'stolen') succeeds: the existing branch-b1/schema subdirectory is moved to branch-stolen, rather than renaming a branch. A fresh table.copy_with_branch("b1") then fails because its schema metadata is missing. I reproduced this on the exact head with a native one-row table and real tag/branch files; the regression fails with original branch readable=false. The source is a malformed logical branch name, but the new SQL operation forwards it to a pre-existing core path without name validation. Reject path separators/control characters before checking source existence or mutating files. A temporary source-name guard preserves the original readable branch.

[P2] Reject target names that readers cannot open. A validation mismatch remains in the newly exposed rename operation (BranchManager::validate_branch_name, crates/paimon/src/table/branch_manager.rs, around lines 70–106). The manager accepts the literal target .., while the catalog's branch-name validator used by Table::copy_with_branch and $branch_... table resolution rejects both . and ... Although branch-.. is physically a single directory segment, it is not a usable branch name through the table API.

Concrete reproduction: create a native table with one row, create tag v1, seed b1 from that tag, then CALL sys.rename_branch(table => 'test_db.t1', from_branch => 'b1', to_branch => '..'). The call succeeds and the original branch disappears, but table.copy_with_branch("..") fails with branch name cannot be '.' or '..'. This moves a readable branch to a name that normal readers cannot open. The core inconsistency predates this wrapper; the new SQL operation needs to enforce the reader's name contract before moving metadata.

Please share the catalog/table name validation at the manager boundary (also rejecting control characters), retaining the manager's additional main/numeric restrictions. My actual rename/read regression fails on this head; a temporary control calling the catalog validator rejects the target before mutation and preserves the original branch and row.

The source traversal probe b1/../../snapshot was rejected by the filesystem backend and preserved the main data; I am not reporting that as a finding.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants