Skip to content

feat: persist DuckLake object-table writes with rollback cleanup - #495

Merged
weicao merged 1 commit into
mainfrom
feat/task77-ducklake-object-storage
Sep 7, 2026
Merged

feat: persist DuckLake object-table writes with rollback cleanup#495
weicao merged 1 commit into
mainfrom
feat/task77-ducklake-object-storage

Conversation

@weicao

@weicao weicao commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

Replay accepted object-storage candidate 450fe6f0 onto current main (7a587430). Unique parent is exact main. Does not update PR #489 / task75-object-table.

This PR is the first-cut DuckLake object-table delivery: persist storage selection, dual-protocol writes, rollback compensation, and failure cleanup. Shutdown coordination is already on main via #492.

Usage boundaries (must keep)

  1. MySQL CREATE TABLE is an implicit commit and is not undone by ROLLBACK (local and object tables alike).
  2. One transaction cannot write both a local table and an object table (1105). Unsupported, not omitted.
  3. The local DuckLake catalog must persist. The object bucket stores table data only; losing the catalog cannot be recovered from the bucket alone.

Identity

  • Replay source: 450fe6f0aeee72c7c008b44ea57e8adae0815f52
  • Base: 7a58743079680cf05f35fc1e97f531a0cba5e7f7
  • FlightSQL constructor kept NewSQLiteFlightSQLServerWithProvider
  • No diagnostic inject env / DIAG_RAW dumps
  • Docs: docs/object-storage.md

Product acceptance on 450fe6f0 / image sha256:8c8ed995…cd719 stays; this PR only needs review of the main integration diff. No v0.2.1 / latest change.

@leon-ape leon-ape left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent review of exact head d73a66f039eb075a3633ac89f2661fd0424c2c1d.

  • Unique parent fb299713d6685e3ff63832e9517b6373e0dd1053; GitHub verification valid.
  • Diff vs parent is only docs/object-storage.md: candidate id 450fe6f0 and must not include restored; three usage boundaries remain.
  • Product files vs accepted 450fe6f0 unchanged except the already-reviewed FlightSQL idempotent close. No inject/DIAG_RAW.
  • Reuse FlightSQL Python SUCCESS on fb299713 (job 101608295902); this head does not change that source.

APPROVE binds only this OID. Task #78 product acceptance is not reopened. No v0.2.1 / latest change.

@weicao
weicao enabled auto-merge (squash) September 7, 2026 03:06
@weicao
weicao disabled auto-merge September 7, 2026 03:07
Replay accepted object-storage candidate 450fe6f0 onto current main, plus FlightSQL idempotent ClosePreparedStatement.

Usage boundaries:
1. MySQL CREATE TABLE is an implicit commit and is not undone by ROLLBACK (local and object tables alike).
2. One transaction cannot write both a local table and an object table (1105).
3. The local DuckLake catalog must persist; losing it cannot be recovered from the object bucket alone.

Does not update PR #489 / task75-object-table. Tree matches previous head d73a66f.
@weicao
weicao force-pushed the feat/task77-ducklake-object-storage branch from d73a66f to 6106a18 Compare September 7, 2026 03:27

@leon-ape leon-ape left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent review of exact head 6106a18fa05e32aa3d3d923141d6d68d3e04dbf0.

  • GitHub verification valid.
  • Unique parent 7a58743079680cf05f35fc1e97f531a0cba5e7f7.
  • Tree 65bdc21bbed281a6c19808ab3525183585b7d17b equals previous head d73a66f039eb075a3633ac89f2661fd0424c2c1d.
  • Single commit; not an empty signed follow-up.

APPROVE binds only this OID. Go/Dolt historical reds not re-verified. Task #78 not reopened. No v0.2.1 / latest change.

@weicao
weicao merged commit 93d2cf9 into main Sep 7, 2026
12 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants