feat(sqlite): add migration planner, runner, and introspection#341
Draft
feat(sqlite): add migration planner, runner, and introspection#341
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
@prisma-next/mongo-runtime
@prisma-next/family-mongo
@prisma-next/sql-runtime
@prisma-next/family-sql
@prisma-next/middleware-telemetry
@prisma-next/mongo
@prisma-next/extension-paradedb
@prisma-next/extension-pgvector
@prisma-next/postgres
@prisma-next/sql-orm-client
@prisma-next/sqlite
@prisma-next/target-mongo
@prisma-next/adapter-mongo
@prisma-next/driver-mongo
@prisma-next/contract
@prisma-next/utils
@prisma-next/config
@prisma-next/errors
@prisma-next/framework-components
@prisma-next/operations
@prisma-next/contract-authoring
@prisma-next/ids
@prisma-next/psl-parser
@prisma-next/psl-printer
@prisma-next/cli
@prisma-next/emitter
@prisma-next/migration-tools
prisma-next
@prisma-next/vite-plugin-contract-emit
@prisma-next/runtime-executor
@prisma-next/mongo-codec
@prisma-next/mongo-contract
@prisma-next/mongo-value
@prisma-next/mongo-contract-psl
@prisma-next/mongo-contract-ts
@prisma-next/mongo-emitter
@prisma-next/mongo-schema-ir
@prisma-next/mongo-query-ast
@prisma-next/mongo-orm
@prisma-next/mongo-pipeline-builder
@prisma-next/mongo-lowering
@prisma-next/mongo-wire
@prisma-next/sql-contract
@prisma-next/sql-errors
@prisma-next/sql-operations
@prisma-next/sql-schema-ir
@prisma-next/sql-contract-psl
@prisma-next/sql-contract-ts
@prisma-next/sql-contract-emitter
@prisma-next/sql-lane-query-builder
@prisma-next/sql-relational-core
@prisma-next/sql-builder
@prisma-next/target-postgres
@prisma-next/target-sqlite
@prisma-next/adapter-postgres
@prisma-next/adapter-sqlite
@prisma-next/driver-postgres
@prisma-next/driver-sqlite
commit: |
Implement the SQLite migration control plane (Milestone 4): Adapter (control-mutation-defaults.ts): - PSL scalar type → SQLite native type mappings - Default function registry (autoincrement, now, uuid, etc.) Adapter (control-adapter.ts): - SqliteControlAdapter with introspect() via sqlite_master + PRAGMA - parseSqliteDefault() normalizer returning ColumnDefault - normalizeSqliteNativeType() for schema verification Target (migrations/): - Planner generating CREATE TABLE, ADD COLUMN, CREATE INDEX ops - DDL builders with SQLite dialect (AUTOINCREMENT, inline FK/UNIQUE) - Runner with BEGIN EXCLUSIVE transactions, control tables (_prisma_marker, _prisma_ledger) using TEXT instead of JSONB - Statement builders with ? params instead of $1 Tests: - 88 adapter tests (introspection, default parsing, codecs) - 41 target tests (DDL builders, planner round-trips, runner lifecycle with real file-backed databases, idempotency, error scenarios matching Postgres test patterns)
Extend the SQLite migration planner with full reconciliation support: - Widening: recreate-table for nullability relaxation, default changes - Destructive: DROP TABLE, DROP INDEX, DROP COLUMN, recreate-table for type changes, nullability tightening, constraint changes - Conflict reporting with granular kinds (typeMismatch, nullabilityConflict, indexIncompatible, foreignKeyConflict) Add e2e migration tests in the e2e package using the contract authoring DSL (defineContract/model/field) covering additive, schema evolution, destructive, widening, and recreate-table operations.
3da2d59 to
b8527ee
Compare
The SQLite recreate-table flow (used for widening/destructive column changes) drops and rebuilds the table. Two problems made this unsafe: - FK-backing indexes were not re-emitted, so verification failed after recreate when the contract declared an FK with index:true. - foreign_keys was left ON during the migration, so DROP TABLE on a referenced parent cascade-deleted rows in child tables. The planner now re-emits FK-backing indexes alongside declared indexes when building recreate-table operations. The runner turns foreign_keys OFF before BEGIN EXCLUSIVE (the pragma is a no-op inside a transaction), runs PRAGMA foreign_key_check before commit, and restores the pragma in an outer finally. A new FOREIGN_KEY_VIOLATION error code surfaces any violations detected by the integrity check. Also fixes the adapter so the test suite can load: controlMutationDefaults was a thunk but the framework expects a plain object (matches postgres); scalarTypeDescriptors was nested under a nonexistent pslTypeDescriptors field; and control-mutation-defaults.ts imported type names that do not exist in family-sql (they live in framework-components). Tests added in test/e2e/framework/test/sqlite/migrations.test.ts: - preserves FK when the child table (holder of the FK) is recreated - preserves FK when the parent (referenced) table is recreated - preserves declared indexes when the table is recreated
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes TML-2168
Intent
Complete the SQLite target's control plane — migration planner, runner, schema introspection, and PSL type mappings — so that
db init/db updatework against SQLite databases using the same contract-driven workflow as Postgres.Change map
SqliteMigrationPlannerthat diffs contract against live schema and emits additive operations (CREATE TABLE, ADD COLUMN, CREATE INDEX, FK-backing indexes)buildCreateTableSql,buildAddColumnSql,buildCreateIndexSqlwith SQLite-specific rendering (AUTOINCREMENT, inline FKs, type affinities, CAST-based defaults)SqliteMigrationRunnerwith exclusive-transaction locking, precheck/execute/postcheck lifecycle, idempotency (skip if postchecks already pass), marker/ledger bookkeeping, post-migration schema verificationBEGIN EXCLUSIVEinstead. Control tables use TEXT instead of JSONB/TIMESTAMPTZ_prisma_markerand_prisma_ledgerwith SQLite-compatible types (TEXT for JSON, TEXT for timestamps viadatetime('now'))SqliteControlAdapter.introspect()queriessqlite_master+ PRAGMAs (table_info,foreign_key_list,index_list,index_info) to buildSqlSchemaIRparseSqliteDefaultnormalizes SQLite's stored defaults (strips parens, mapsdatetime('now')→now(), handles numeric/string/null literals)control-mutation-defaults.tsmaps all PSL scalars to SQLite native types/codecs and implements default function lowering (autoincrement, now, uuid, cuid, ulid, nanoid, dbgenerated)target = "sqlite"in PSL to emit correct contractsSqlControlTargetDescriptornow exposesmigrations.createPlanner(),createRunner(),contractToSchema()The story
The SQLite target already had a working runtime plane (driver, adapter, codecs, query builder, ORM). This commit adds the control plane — the last major piece before SQLite is a fully functional target.
Planner (planner.ts): Walks the contract's tables, columns, indexes, and foreign keys. For each object missing from the live schema, it emits an operation with precheck ("does not exist"), execute (DDL), and postcheck ("now exists"). DDL rendering is delegated to planner-ddl-builders.ts which handles SQLite-specific syntax:
INTEGER PRIMARY KEY AUTOINCREMENTfor auto-increment PKs, inlineFOREIGN KEYconstraints with referential actions,CAST-based default rendering, and safe native-type validation.Runner (runner.ts): Wraps the entire migration in
BEGIN EXCLUSIVE(SQLite's equivalent of advisory locks), creates control tables if needed, checks marker compatibility, applies operations with idempotency support (postchecks satisfied → skip), verifies the resulting schema against the contract, then upserts the marker and records a ledger entry. All within a single transaction — if anything fails, everything rolls back.Introspection (control-adapter.ts): Reads
sqlite_masterfor table names, then for each table queriesPRAGMA table_info(columns, PKs),PRAGMA foreign_key_list(FKs with referential actions), andPRAGMA index_list/index_info(indexes and unique constraints, classified by origin). Produces the standardSqlSchemaIRthat the planner and runner consume.PSL mappings (control-mutation-defaults.ts): Maps PSL scalar types → SQLite affinities (e.g.
Boolean → integerwith boolean codec,DateTime → textwith datetime codec,Json → textwith json codec). Implements all default function lowering handlers with proper argument validation and diagnostic reporting.Behavior changes & evidence
createPlanner()/createRunner()are now callable on the SQLite target descriptorCompatibility / migration / risk
@prisma-next/family-sql,@prisma-next/sql-schema-ir,@prisma-next/sql-errors,@prisma-next/utilsadded to target-sqlite'spackage.json.Non-goals / intentionally out of scope
:memory:database supportsql-lane(legacy) andkysely-laneSQLite support