Spark: fix delete from branch for canDeleteWhere where it does not resolve to the correct branch

[yingjianwu98]

---

Problem

When WAP (Write-Audit-Publish) is enabled via spark.wap.branch, canDeleteWhere() and deleteWhere() scan different branches:

• canDeleteWhere() scans using this.branch (null → main) because the WAP branch is only a session config, not part of the table identifier
• deleteWhere() resolves the WAP branch before committing

This causes canDeleteWhere() to incorrectly return true (metadata-only delete is possible) based on main's data, while deleteWhere() commits to the
WAP branch where the file has partial matches, resulting in:

ValidationException: Cannot delete file where some, but not all, rows match filter

Example

-- WAP enabled, spark.wap.branch = dev1
INSERT INTO t VALUES (1, 'a'), (2, 'b'), (3, 'c'); -- goes to dev1, main is empty
DELETE FROM t WHERE id = 1;
-- canDeleteWhere scans main (empty) → true → metadata delete
-- deleteWhere commits to dev1 → partial match → ValidationException

Fix

• canDeleteWhere() now resolves the WAP branch via determineReadBranch before scanning. This uses determineReadBranch (not determineWriteBranch)
  because the scan is a read operation, and determineReadBranch correctly handles the case where the WAP branch doesn't exist yet by falling back to
  main.
• Both canDeleteWhere() and deleteWhere() now use local variables for the resolved branch instead of mutating this.branch, avoiding side effects on
  reads and other operations that share the field.

Will work on the backport to other Spark versions once there is consensus from the community.

[rdblue]

From Netflix discussion, this sounds like a correctness bug and release blocker for 1.11.

[stevenzwu]

can we also insert some rows/files into the main branch first? ideally with a row of matching the predicate of id=1 .

[amogh-jahagirdar]

+1

[stevenzwu]

use assertj

assertThat(sql("SELECT * FROM %s VERSION AS OF 'dev1' ORDER BY id", tableName))
.containsExactlyInAnyOrder(...)

[stevenzwu]

@yingjianwu98 can you resolve the merge conflict?

[amogh-jahagirdar]

Thanks @yingjianwu98 for the fix, this does look right to me just had some comments on the tests.

[amogh-jahagirdar]

+1

[amogh-jahagirdar]

I would also add an assertion for the main state as well

[yingjianwu98]

Thanks for the review! I will rebase today or tomorrow and ping back here.

[stevenzwu]

@yingjianwu98 this is marked a blocker for 1.11.0 release. please ping here when the rebase and comments are addressed.

[yingjianwu98]

@stevenzwu @amogh-jahagirdar

Apologize for the delay here. I have addressed the comments, and want to callout the following:

testMetadataDeleteToWapBranchCommitsToWapBranch is trying to reproduce the recent regression where it could delete data from the main branch

testDeleteToWapBranchCanDeleteWhereScansWapBranch is trying to fix an issue that has existed for all the iceberg versions, where the metadata delete condition is always looking at the main branch but not the wap branch.