Core, Spark: Verify that TRUNCATE removes orphaned DVs#16078
Core, Spark: Verify that TRUNCATE removes orphaned DVs#16078nastra wants to merge 1 commit intoapache:mainfrom
Conversation
nastra
force-pushed
the
truncate-with-dvs
branch
2 times, most recently
from
c8668a1 to
ed04dfc
Compare
nastra
force-pushed
the
truncate-with-dvs
branch
from
ed04dfc to
c912617
Compare
| @TestTemplate | ||
| public void truncateWithDVs() throws NoSuchTableException { | ||
| sql( | ||
| "CREATE TABLE %s (id bigint NOT NULL, data string) USING iceberg TBLPROPERTIES ('format-version'='3','write.delete.mode'='merge-on-read')", |
There was a problem hiding this comment.
Would it make sense to parameterize format version to 3 and above?
There was a problem hiding this comment.
it didn't seem worth to me to parameterize the entire test suite in order to run it across multiple format versions as that adds quite some execution overhead. But I can update if people think it's worth it
There was a problem hiding this comment.
I think it makes sense to paramterize unless it's an egregious amount of test time (which I think we should look into separately if it's the case). From a local test on my Mac, run it takes ~20 seconds. I'm mostly looking at it from the perspective of once we make v4 commit path changes which are structurally different we still want to make sure the integration behavior doesn't regress in any manner.
There was a problem hiding this comment.
Not a blocker for me though, just my 2c.
| @TestTemplate | ||
| public void truncateWithDVs() throws NoSuchTableException { | ||
| sql( | ||
| "CREATE TABLE %s (id bigint NOT NULL, data string) USING iceberg TBLPROPERTIES ('format-version'='3','write.delete.mode'='merge-on-read')", |
There was a problem hiding this comment.
I think it makes sense to paramterize unless it's an egregious amount of test time (which I think we should look into separately if it's the case). From a local test on my Mac, run it takes ~20 seconds. I'm mostly looking at it from the perspective of once we make v4 commit path changes which are structurally different we still want to make sure the integration behavior doesn't regress in any manner.
| @TestTemplate | ||
| public void truncateWithDVs() throws NoSuchTableException { | ||
| sql( | ||
| "CREATE TABLE %s (id bigint NOT NULL, data string) USING iceberg TBLPROPERTIES ('format-version'='3','write.delete.mode'='merge-on-read')", |
There was a problem hiding this comment.
Not a blocker for me though, just my 2c.
3 participants
This is just adding some additional testing coverage for
TRUNCATEafter #13222 was introduced, where the row filter is sent asalwaysTrue()