New migrations system, ported from sqlite-migrate

docs/changelog.rst

@@ -4,6 +4,13 @@
  Changelog
 ===========
 
+.. _v_unreleased:
+
+Unreleased
+----------
+
+- New :ref:`database migrations system <migrations>`, incorporating functionality that was previously provided by the separate `sqlite-migrate <https://github.com/simonw/sqlite-migrate>`__ plugin. Define migration sets using the new :class:`sqlite_utils.Migrations` class and apply them using the ``sqlite-utils migrate`` command or the :ref:`migrations Python API <migrations_python>`.  (:issue:`752`)
+
 .. _v3_39:
 
 3.39 (2025-11-24)
@@ -182,7 +189,7 @@ This release introduces a new :ref:`plugin system <plugins>`. Read more about th
 - Conversion functions passed to :ref:`table.convert(...) <python_api_convert>` can now return lists or dictionaries, which will be inserted into the database as JSON strings. (:issue:`495`)
 - ``sqlite-utils install`` and ``sqlite-utils uninstall`` commands for installing packages into the same virtual environment as ``sqlite-utils``, :ref:`described here <cli_install>`. (:issue:`483`)
 - New :ref:`sqlite_utils.utils.flatten() <reference_utils_flatten>` utility function. (:issue:`500`)
-- Documentation on :ref:`using Just <contributing_just>` to run tests, linters and build documentation. 
+- Documentation on :ref:`using Just <contributing_just>` to run tests, linters and build documentation.
 - Documentation now covers the :ref:`release_process` for this package.
 
 .. _v3_29:

docs/cli-reference.rst

@@ -20,7 +20,7 @@ This page lists the ``--help`` for every ``sqlite-utils`` CLI sub-command.
         "query", "memory", "insert", "upsert", "bulk", "search", "transform", "extract",
         "schema", "insert-files", "analyze-tables", "convert", "tables", "views", "rows",
         "triggers", "indexes", "create-database", "create-table", "create-index",
-        "enable-fts", "populate-fts", "rebuild-fts", "disable-fts"
+        "migrate", "enable-fts", "populate-fts", "rebuild-fts", "disable-fts"
     ]
     refs = {
         "query": "cli_query",
@@ -49,6 +49,7 @@ This page lists the ``--help`` for every ``sqlite-utils`` CLI sub-command.
         "enable-wal": "cli_wal",
         "enable-counts": "cli_enable_counts",
         "bulk": "cli_bulk",
+        "migrate": "cli_migrate",
         "create-database": "cli_create_database",
         "create-table": "cli_create_table",
         "drop-table": "cli_drop_table",
@@ -965,6 +966,44 @@ See :ref:`cli_create_index`.
       -h, --help                 Show this message and exit.
 
 
+.. _cli_ref_migrate:
+
+migrate
+=======
+
+See :ref:`cli_migrate`.
+
+::
+
+    Usage: sqlite-utils migrate [OPTIONS] DB_PATH [MIGRATIONS]...
+
+      Apply pending database migrations.
+
+      Usage:
+
+          sqlite-utils migrate database.db
+
+      This will find the migrations.py file in the current directory or
+      subdirectories and apply any pending migrations.
+
+      Or pass paths to one or more migrations.py files directly:
+
+          sqlite-utils migrate database.db path/to/migrations.py
+
+      Pass --list to see a list of applied and pending migrations without applying
+      them.
+
+      Use --stop-before migration_set:name to stop before a migration. This option
+      can be used multiple times.
+
+    Options:
+      --stop-before TEXT  Stop before applying this migration. Use set:name to
+                          target a migration set.
+      --list              List migrations without running them
+      -v, --verbose       Show verbose output
+      -h, --help          Show this message and exit.
+
+
 .. _cli_ref_enable_fts:
 
 enable-fts

docs/cli.rst

@@ -1058,6 +1058,37 @@ That will look for SpatiaLite in a set of predictable locations. To load it from
 
     sqlite-utils create-database empty.db --init-spatialite --load-extension /path/to/spatialite.so
 
+.. _cli_migrate:
+
+Running migrations
+==================
+
+The ``migrate`` command applies pending Python migrations to a database. For the full migration file format and Python API, see :ref:`migrations`.
+
+.. code-block:: bash
+
+    sqlite-utils migrate creatures.db path/to/migrations.py
+
+If you omit the migration path it will search the current directory and subdirectories for files called ``migrations.py``:
+
+.. code-block:: bash
+
+    sqlite-utils migrate creatures.db
+
+Use ``--list`` to list applied and pending migrations without running them:
+
+.. code-block:: bash
+
+    sqlite-utils migrate creatures.db --list
+
+Use ``--stop-before`` to stop before a named migration. The option can be passed more than once, and can target a specific migration set using ``migration_set:migration_name``:
+
+.. code-block:: bash
+
+    sqlite-utils migrate creatures.db path/to/migrations.py \
+      --stop-before creatures:add_weight \
+      --stop-before sales:drop_index
+
 .. _cli_inserting_data:
 
 Inserting JSON data

docs/index.rst

@@ -36,6 +36,7 @@ Contents
    installation
    cli
    python-api
+   migrations
    plugins
    reference
    cli-reference

docs/migrations.rst

@@ -0,0 +1,171 @@
+.. _migrations:
+
+=====================
+ Database migrations
+=====================
+
+``sqlite-utils`` includes a migration system for applying repeatable changes to SQLite database files.
+
+A migration is a Python function that receives a :class:`sqlite_utils.Database` instance and then executes Python code to modify that database - creating or transforming tables, adding indexes, inserting rows, or any other operation supported by SQLite.
+
+Migrations are grouped into named sets using the :class:`sqlite_utils.Migrations` class, and each applied migration is recorded in the ``_sqlite_migrations`` table in that database.
+
+This means you can run the migrate operation multiple times and it will only apply migrations that have not previously been recorded.
+
+.. _migrations_define:
+
+Defining migrations
+===================
+
+Ordered migration sets are defined by first creating a :class:`sqlite_utils.Migrations` object.
+
+Individual migrations are Python functions that are then registered with that migration set. Each migration function is passed a single argument that is a :ref:`sqlite_utils.Database <reference_db_database>` instance.
+
+The name passed to ``Migrations("creatures")`` identifies that set of migrations. Use a name that is unique for your project, since multiple migration sets can be applied to the same database.
+
+Here is a simple example of a ``migrations.py`` file which creates a table, then adds an extra column to that table in a second migration:
+
+.. code-block:: python
+
+    from sqlite_utils import Database, Migrations
+
+    migrations = Migrations("creatures")
+
+    @migrations()
+    def create_table(db):
+        db["creatures"].create(
+            {"id": int, "name": str, "species": str},
+            pk="id",
+        )
+
+    @migrations()
+    def add_weight(db):
+        db["creatures"].add_column("weight", float)
+
+.. _migrations_python:
+
+Applying migrations in Python
+=============================
+
+Once you have a ``Migrations(name)`` collection with one or more migrations registered to it, you can execute them in Python code like this:
+
+.. code-block:: python
+
+    db = Database("creatures.db")
+    migrations.apply(db)
+
+Running ``migrations.apply(db)`` repeatedly is safe. Migrations that already have a matching ``migration_set`` and ``name`` row in ``_sqlite_migrations`` will be skipped.
+
+Migration functions are applied in the order that they were registered. The function name is used as the migration name unless you pass one explicitly:
+
+.. code-block:: python
+
+    @migrations(name="001_create_table")
+    def create_table(db):
+        db["creatures"].create({"id": int, "name": str}, pk="id")
+
+When you apply a set of migrations you can stop part way through by specifying a ``stop_before=`` migration name:
+
+.. code-block:: python
+
+    migrations.apply(db, stop_before="add_weight")
+
+Applying migrations using the CLI
+=================================
+
+Run migrations using the ``sqlite-utils migrate`` command:
+
+.. code-block:: bash
+
+    sqlite-utils migrate creatures.db path/to/migrations.py
+
+The first argument is the database file. The remaining arguments can be paths to migration files or directories containing migration files.
+
+If you omit migration paths, ``sqlite-utils`` searches the current directory and subdirectories for files called ``migrations.py``:
+
+.. code-block:: bash
+
+    sqlite-utils migrate creatures.db
+
+You can also pass a directory. Every ``migrations.py`` file in that directory tree will be considered:
+
+.. code-block:: bash
+
+    sqlite-utils migrate creatures.db path/to/project/
+
+Running the command repeatedly is safe. Migrations that already have a matching ``migration_set`` and ``name`` row in ``_sqlite_migrations`` will be skipped.
+
+Listing migrations
+==================
+
+Use ``--list`` to show applied and pending migrations without running them:
+
+.. code-block:: bash
+
+    sqlite-utils migrate creatures.db --list
+
+Example output:
+
+.. code-block:: output
+
+    Migrations for: creatures
+
+      Applied:
+        create_table - 2026-06-09 17:23:12.048092+00:00
+        add_weight - 2026-06-09 17:23:12.051249+00:00
+
+      Pending:
+        add_age
+
+Stopping before a migration
+===========================
+
+When applying migrations using the CLI, you can stop before a named migration:
+
+.. code-block:: bash
+
+    sqlite-utils migrate creatures.db path/to/migrations.py --stop-before add_weight
+
+This applies any pending migrations before ``add_weight`` and leaves ``add_weight`` and later migrations pending. An unqualified migration name matches in any migration set.
+
+You can also target a specific migration set using ``migration_set:migration_name``. This is useful if a migrations file contains more than one migration set, or if multiple sets use the same migration name:
+
+.. code-block:: bash
+
+    sqlite-utils migrate creatures.db path/to/migrations.py \
+      --stop-before creatures:add_weight \
+      --stop-before sales:drop_index
+
+The ``--stop-before`` option can be passed more than once.
+
+Verbose output
+==============
+
+Use ``--verbose`` or ``-v`` to show the schema before and after migrations are applied, plus a unified diff when the schema changes:
+
+.. code-block:: bash
+
+    sqlite-utils migrate creatures.db --verbose
+
+Migrating from sqlite-migrate
+=============================
+
+This system uses the same migration table format as the older `sqlite-migrate <https://github.com/simonw/sqlite-migrate>`__ package. To use existing migration files directly with ``sqlite-utils``, update their import from ``sqlite_migrate`` to ``sqlite_utils``:
+
+.. code-block:: python
+
+    from sqlite_utils import Migrations
+
+    migration = Migrations("creatures")
+
+    @migration()
+    def create_table(db):
+        db["creatures"].create({"id": int, "name": str}, pk="id")
+
+Python API
+==========
+
+.. autoclass:: sqlite_utils.migrations.Migrations
+   :members:
+   :undoc-members:
+   :exclude-members: _Migration, _AppliedMigration

sqlite_utils/__init__.py

@@ -2,5 +2,6 @@
 from .hookspecs import hookimpl
 from .hookspecs import hookspec
 from .db import Database
+from .migrations import Migrations
 
-__all__ = ["Database", "suggest_column_types", "hookimpl", "hookspec"]
+__all__ = ["Database", "Migrations", "suggest_column_types", "hookimpl", "hookspec"]