Skip to content

Resolve staged renames when moving columns in UpdateSchema#3687

Open
Sanjays2402 wants to merge 1 commit into
apache:mainfrom
Sanjays2402:fix-2599-rename-then-move
Open

Resolve staged renames when moving columns in UpdateSchema#3687
Sanjays2402 wants to merge 1 commit into
apache:mainfrom
Sanjays2402:fix-2599-rename-then-move

Conversation

@Sanjays2402

@Sanjays2402 Sanjays2402 commented Jul 19, 2026

Copy link
Copy Markdown

Closes #2599

Rationale for this change

rename_column stages the new column name in self._updates, but move_first/move_before/move_after resolved the target column via self._schema.find_field(...) against the original schema. Moving a column by its new name within the same update context therefore raised ValueError: Cannot move missing column.

Reproduction (reported via AWS Glue, but reproduces fully in-memory with InMemoryCatalog — it is not Glue-specific):

with tbl.update_schema() as update:
    update.rename_column("some_column", "renamed_column")
    update.move_first("renamed_column")   # ValueError: Cannot move missing column: renamed_column

_find_for_move now checks staged renames in self._updates first (respecting _case_sensitive) before falling back to the original schema, matching the Java implementation which resolves staged renames.

Are these changes tested?

Yes. Added tests/table/test_update_schema.py covering rename-then-move by new name for move_first, move_before, and move_after using InMemoryCatalog.

  • Without the fix: all 3 fail with ValueError: Cannot move missing column.
  • With the fix: all 3 pass.

ruff check and ruff format --check are clean on both changed files.

Are there any user-facing changes?

No API changes. This fixes incorrect behavior: renaming a column and then moving it by its new name within the same update_schema() context now works instead of raising ValueError.

Fixes apache#2599. rename_column stages the new name in self._updates, but
move_first/move_before/move_after resolved the target via the original
schema, so moving a column by its new name in the same update context
raised 'Cannot move missing column'. _find_for_move now checks staged
renames first, matching the Java implementation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Possible read-after-write consistency issue with multiple schema migration steps in Iceberg tables on AWS Glue + S3

1 participant