Skip to content

feat(datafusion): support the create_branch procedure - #939

Open
jackylee-ch wants to merge 3 commits into
apache:mainfrom
jackylee-ch:feat/datafusion-create-branch-procedure
Open

jackylee-ch wants to merge 3 commits into
apache:mainfrom
jackylee-ch:feat/datafusion-create-branch-procedure

Conversation

@jackylee-ch

Copy link
Copy Markdown
Contributor

Java exposes sys.create_branch(table, branch[, tag][, ignore_if_exists]) to fork a branch off the current schema or a tag without touching main. The core BranchManager::create_branch / create_branch_from_tag primitives already existed (#290/#482 wired the branch reader and manager); only the DataFusion CALL procedure was missing.

This registers create_branch: it declares the table/branch/tag/ignore_if_exists parameters so a misspelled argument is rejected (matching Java's binding), and dispatches to BranchManager — creating from a tag when one is given, and treating an existing branch as a no-op only when ignore_if_exists is set.

Tested end-to-end: CALL, then assert the branch appears in $branches; recreating errors; recreating with ignore_if_exists is a no-op.

@JingsongLi

Copy link
Copy Markdown
Contributor

Requirement fit: SUPPORTED. Normal branch creation, duplicate handling, and a branch created from a tag worked in local DataFusion SQL tests. However, P1: reject path separators in branch names. The shared BranchManager::validate_branch_name allows b1/hidden. I reproduced this at 697e6c9: after creating b1, CALL sys.create_branch(table => 'test_db.t1', branch => 'b1/hidden') succeeds, but SELECT * FROM t1$branches WHERE branch_name = 'b1/hidden' returns zero rows. The branch is created underneath branch-b1 and is invisible to normal branch listing. Please validate names at the shared manager boundary before writing any schema/tag/snapshot files, and add a SQL regression confirming an invalid name fails without creating files. Original PR test, a temporary from-tag test, formatting and diff checks pass; CI is green but misses this case. Temporary test edits were restored.

jackylee-ch added a commit to jackylee-ch/paimon-rust that referenced this pull request Sep 25, 2026
Review follow-up (apache#939): `BranchManager::validate_branch_name` accepted a
name containing a path separator such as `b1/hidden`, so `sys.create_branch`
created a branch directory at a nested path (`branch-b1/hidden`) that
`$branches` never lists back — a silently orphaned branch. Reject '/' and
'\\' in branch names, alongside the existing main/blank/numeric checks.
Java exposes `sys.create_branch(table, branch[, tag][, ignore_if_exists])`
to fork a branch off the current schema or a tag without touching main. The
core BranchManager primitives already existed; only the DataFusion procedure
was missing (apache#290/apache#482 wired the branch reader and manager, not the CALL).

Register create_branch: declare its table/branch/tag/ignore_if_exists
parameters (so a misspelled argument is rejected, like Java's binding) and
dispatch to BranchManager -- creating from a tag when one is given, and
treating an existing branch as a no-op only when ignore_if_exists is set.
Review follow-up (apache#939): `BranchManager::validate_branch_name` accepted a
name containing a path separator such as `b1/hidden`, so `sys.create_branch`
created a branch directory at a nested path (`branch-b1/hidden`) that
`$branches` never lists back — a silently orphaned branch. Reject '/' and
'\\' in branch names, alongside the existing main/blank/numeric checks.
Add the SQL regression requested in review: after creating `b1`,
`CALL sys.create_branch(..., branch => 'b1/hidden')` must fail with the
path-separator error, leave no nested directory under `branch-b1/`, and keep
`$branches` at the single `b1` row.
@jackylee-ch
jackylee-ch force-pushed the feat/datafusion-create-branch-procedure branch from 4b061cf to fae33f6 Compare September 30, 2026 11:11
@jackylee-ch

Copy link
Copy Markdown
Contributor Author

Addressed. validate_branch_name now rejects / and \ at the shared BranchManager boundary, so create_branch and create_branch_from_tag fail before copying any schema, tag or snapshot file. Added the SQL regression: after creating b1, CALL sys.create_branch(..., branch => 'b1/hidden') fails with the path-separator error, nothing is created under branch-b1/, and $branches still lists only b1. Rebased onto main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants