Skip to content

[python] Remove default deletion vectors and dedicated vector files for multimodal tables - #10970

Merged
JingsongLi merged 2 commits into
apache:masterfrom
JingsongLi:codex/python-multimodal-default-options
Nov 3, 2026
Merged

JingsongLi merged 2 commits into
apache:masterfrom
JingsongLi:codex/python-multimodal-default-options

Conversation

@JingsongLi

@JingsongLi JingsongLi commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

Stop automatically setting deletion-vectors.enabled=true and vector.file.format=parquet when creating Python multimodal tables. Both remain available through explicit table options. Update the default-options assertions and documentation, including the deletion-vector requirement for row-level deletes and matched MERGE deletes. Explicitly enable these options in test tables that exercise deletion vectors or dedicated vector files.

Tests

  • PyTorch suite: 79 passed, 65 subtests passed.
  • Ray vector, batch vector, search results, filter exactness, full-text and result metadata suites: 141 passed.
  • Focused multimodal, column enrichment, Lance export, search snapshot, score metadata, batch lookup and global-index regressions passed, including the final partition-scoped index regression.
  • Flake8 for changed Python files and git diff --check passed.

Native CI dependency

The latest Native CI run is blocked by an independent zero-width fixed binary upsert bug in paimon-rust, tracked in apache/paimon-rust#1316 and fixed by apache/paimon-rust#1317. All other selected jobs passed.

The rebuilt native wheel passes all 16 zero-width binary upsert cases, including the four CI failures, and all 534 upsert encoding and binary/text regressions. Native CI builds paimon-rust main; rerun it after the dependency fix is merged.

@leaves12138 leaves12138 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JingsongLi
JingsongLi merged commit de6ac4a into apache:master Nov 3, 2026
20 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants