Skip to content

Add unified SQL catalog managers (SQLite + Postgres) #33

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
zfarrell merged 18 commits into from
Dec 13, 2025
Merged

Add unified SQL catalog managers (SQLite + Postgres) #33

zfarrell merged 18 commits into from
Dec 13, 2025

Conversation

@zfarrell
Copy link
Contributor

@zfarrell zfarrell commented Dec 12, 2025 •
edited
Loading

  • Replace the DuckDB metadata catalog with a new SQLx-powered abstraction that supports both SQLite and Postgres, including a shared CatalogStore, migration traits, and concrete managers. SQLite is the new default metadata database.
  • Update engine, config handling, and default configs/Docker setup to instantiate the appropriate catalog backend (SQLite by default) and ensure cache/state paths are created consistently.
  • Consolidate catalog tests into a single suite covering both backends with testcontainers, and update the HTTP/integration tests to exercise the new engine wiring.

Resolves #11

@zfarrell
initial pass at sqlite manager implementation
2b6a552
@zfarrell
initial pass at sqlx manager implementation (broken)
c98ffea
@zfarrell
Revert "initial pass at sqlx manager implementation (broken)"
0660f33 
This reverts commit c98ffea.
@zfarrell
try to unify postgres/sqlite managers
265ddec
@zfarrell
centralize migrations
2405171
@zfarrell
consolidate into a single version
a2ae731
@zfarrell
fix tests
4c5aa86
@zfarrell
unify catalog manager test suite
b052000
@zfarrell
remove duckdb catalog manager
ca30a64
@zfarrell
fix issue where cache directory is not created under metadata dir
3cd6d02
@zfarrell
Merge remote-tracking branch 'origin/main' into zac/add-sqlite-manager
80b7bf1
@zfarrell
remove macros feature
137be3c
@zfarrell
add documentation
f2274fc
@zfarrell
clippy
eb31d74
@zfarrell
simplify migrations to use async trait
a210304
@zfarrell
move block_on to manager
8154202
@zfarrell
re-name CatalogStore to CatalogBackend
181d927
@zfarrell
cargo fmt
425354d
@zfarrell zfarrell marked this pull request as ready for review December 13, 2025 20:55
@zfarrell zfarrell merged commit e6e4fbf into main Dec 13, 2025
6 checks passed
@zfarrell zfarrell mentioned this pull request Dec 14, 2025
Closed
zfarrell added a commit that referenced this pull request Jan 6, 2026
@zfarrell
Add unified SQL catalog managers (SQLite + Postgres) (#33)
57ef3d3 
* initial pass at sqlite manager implementation

* initial pass at sqlx manager implementation (broken)

* Revert "initial pass at sqlx manager implementation (broken)"

This reverts commit c98ffea.

* try to unify postgres/sqlite managers

* centralize migrations

* consolidate into a single version

* fix tests

* unify catalog manager test suite

* remove duckdb catalog manager

* fix issue where cache directory is not created under metadata dir

* remove macros feature

* add documentation

* clippy

* simplify migrations to use async trait

* move block_on to manager

* re-name CatalogStore to CatalogBackend

* cargo fmt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Replace duckdb with sqlite for metadata catalog

2 participants

@zfarrell