Skip to content

Parquet, Spark, Flink: partial and multi column variant tests - #17553

Open
nssalian wants to merge 1 commit into
apache:mainfrom
nssalian:variant-multi-column
Open

Parquet, Spark, Flink: partial and multi column variant tests#17553
nssalian wants to merge 1 commit into
apache:mainfrom
nssalian:variant-multi-column

Conversation

@nssalian

@nssalian nssalian commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

Rationale

  • Partial shredding: #15086 raised a concern with partially shredding a ShreddedObject (some fields shredded, the rest left in value). It wasn't a bug, but the path was untested.
  • Multiple variant columns: reading more than one variant column per table wasn't sufficiently covered.

Changes (tests only)

  • Parquet (TestVariantWriters): partial shredding of one and of two variant columns
  • Spark 4.0 / 4.1 (TestSparkVariantRead): multiple variant columns, shredded and unshredded, across the row and vectorized readers.
  • Flink (TestFlinkVariantType, TestFlinkVariantShreddingType): multiple variant columns, shredded and unshredded.
  • The rest is cleanup to use some of the helpers and common artifacts.

@nssalian
nssalian force-pushed the variant-multi-column branch from a07e243 to 562910f Compare August 8, 2026 01:16
@nssalian
nssalian requested a review from huaxingao August 8, 2026 18:24
@nssalian
nssalian marked this pull request as ready for review August 8, 2026 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant