Skip to content

Fix selector strategy#9474

Closed
cetra3 wants to merge 2 commits intoapache:mainfrom
pydantic:fix_sparse_column_error
Closed

Fix selector strategy#9474
cetra3 wants to merge 2 commits intoapache:mainfrom
pydantic:fix_sparse_column_error

Conversation

@cetra3
Copy link
Contributor

@cetra3 cetra3 commented Feb 24, 2026

Which issue does this PR close?

None, but we are seeing this in production

Rationale for this change

There is a mismatch between some configuration options that cause this test to fail.

What changes are included in this PR?

Adds some changes to override_selector_strategy_if_needed

Are these changes tested?

Yes, a new test is written that fails on main

Are there any user-facing changes?

Nope

@github-actions github-actions bot added the parquet Changes to the parquet crate label Feb 24, 2026
@friendlymatthew
Copy link
Contributor

cc @alamb @Jefffrey and maybe @hhhizzz

) -> ReadPlanBuilder {
// override only applies to Auto policy, If the policy is already Mask or Selectors, respect that
let RowSelectionPolicy::Auto { .. } = plan_builder.row_selection_policy() else {
// If the policy is already Selectors, no override is needed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember our design here is to respect the user’s configuration. If they choose mask, they should face the risk that the program may fail, rather than the program implicitly changing the setting to selector.

@cetra3
Copy link
Contributor Author

cetra3 commented Mar 5, 2026

OK I've raised a PR to actually address the underlying cause, but I feel like this whole behaviour is extremely brittle and probably worth a stab at coming up with a better design: #9509

Closing this PR in favour

@cetra3 cetra3 closed this Mar 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parquet Changes to the parquet crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants