Skip to content

Spark: support CALL older_than function expressions#16058

Open
wangyum wants to merge 4 commits intoapache:mainfrom
wangyum:ICEBERG-16057
Open

Spark: support CALL older_than function expressions#16058
wangyum wants to merge 4 commits intoapache:mainfrom
wangyum:ICEBERG-16057

Conversation

@wangyum
Copy link
Copy Markdown
Member

@wangyum wangyum commented Apr 20, 2026

Why are the changes needed?

CALL ... expire_snapshots should support function-based timestamp arguments (for example, current_timestamp() - INTERVAL 1 DAYS) forms across Spark versions.

What changes were proposed in this pull request?

Spark 3.4 / 3.5

  • Extended Iceberg SQL extension grammar to allow function-call expressions in CALL arguments with optional interval subtraction:
    • functionCallExpression (MINUS intervalLiteral)?
  • Added INTERVAL token/non-reserved handling in the extension grammar.
  • Added parser coverage in TestCallStatementParser for:
    • current_timestamp()
    • current_timestamp() - INTERVAL 2 DAYS
    • mixed-case SQL + lowercase interval usage
    • negative parser boundaries for unsupported interval forms
  • Added procedure coverage in TestExpireSnapshotsProcedure for:
    • current_timestamp() positive path
    • current_timestamp() - INTERVAL 2 DAYS positive path
    • mixed-case/lowercase SQL positive path

Spark 4.0 / 4.1

  • Added procedure coverage in TestExpireSnapshotsProcedure for:
    • current_timestamp() positive path
    • mixed-case SQL + function-call positive path
    • interval subtraction and unsupported interval forms as negative cases
  • Added a helper assertion to validate unsupported older_than expression failures consistently.
  • Kept extension grammar changes aligned with 3.x for cross-version parity.

Closes #16057.

@github-actions github-actions Bot added the spark label Apr 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support dynamic timestamp args in Iceberg CALL

1 participant