feat: apso import data — copy table data from Supabase into the rebuilt DB (phase 2)#69
Open
cultron wants to merge 1 commit into
Open
feat: apso import data — copy table data from Supabase into the rebuilt DB (phase 2)#69cultron wants to merge 1 commit into
cultron wants to merge 1 commit into
Conversation
…lt DB Phase 2 of migrating off Supabase: after 'apso import supabase' generates the schema and it's deployed, this copies the rows from the source Postgres into the target database. - src/lib/import/copy-data.ts (pure, fully unit-tested): planCopy() builds a topologically-ordered, FK-aware plan. Mapping mirrors phase 1 exactly (target table = snakeCase(entity); single-col FK -> camelCase <ref>Id via the shared deriveToName/getRelationshipIdField; all other columns identity), then verifies every mapped column against an introspection of the target and drops/reports mismatches. executeCopy() is insert-only (pre-flight aborts if any target table is non-empty), batches inserts, resets serial sequences, and runs in one transaction (rollback on error). - src/lib/import/pg-data.ts: PgSourceReader/PgTargetWriter (lazy pg, SSL on) implementing the injectable SourceReader/TargetWriter seams. - src/commands/import/data.ts: 'apso import data' with source/target conn resolution (flag / SUPABASE_DB_URL|DATABASE_URL and TARGET_DATABASE_URL| APSO_DATABASE_URL / masked prompt), --schema/--tables/--batch-size/ --dry-run/--yes; never logs or persists connection strings. Reuses phase 1's PgIntrospector for both source and target schemas. Verified end-to-end against two live PGlite databases: snake_case source -> camelCase-FK target with JSONB/null preservation, serial sequence reset (next id = 3, no collision), and FK integrity across a join. Full suite 264 tests, lint clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cultron
force-pushed
the
feat/import-supabase-data
branch
from
June 13, 2026 15:44
c28e2db to
bb54fc7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 2 of migrating off Supabase. Phase 1 (#68) generates the
.apsorcand you rebuild the schema on Apso; this command then copies the row data from the source Postgres into the target database.How it works
PgIntrospector(reused as-is).planCopy, pure): topologically order tables by FK (parents first; self-refs ignored, cycles reported + best-effort), map each source column to its target column — exactly mirroring phase 1: target table =snakeCase(entity), single-column FK → camelCase<ref>Id(via the sharedderiveToName/getRelationshipIdField), everything else identity — then verify each mapped column against the target introspection, dropping + reporting any mismatch.executeCopy): insert-only — pre-flight aborts before writing if any target table already has rows; batched inserts; serial sequences reset after load; whole copy in one transaction (rollback on error).Decisions (confirmed)
-t/TARGET_DATABASE_URL/APSO_DATABASE_URL/ prompt) — there's no platform API exposing it. Connection strings are never logged or persisted.Verification
id= 3, no collision), and FK integrity verified across a join.Notes / follow-ups
OFFSET/LIMITordered by the PK — fine for a one-shot migration; a keyset cursor would be better for very large tables under concurrent writes.🤖 Generated with Claude Code