Skip to content

feat(migrations): expose the SQL migrations as an embed.FS - #138

Merged
hsinatfootprintai merged 1 commit into
mainfrom
feat/expose-migrations-as-embed-fs
Aug 14, 2026
Merged

feat(migrations): expose the SQL migrations as an embed.FS#138
hsinatfootprintai merged 1 commit into
mainfrom
feat/expose-migrations-as-embed-fs

Conversation

@hsinatfootprintai

Copy link
Copy Markdown
Contributor

Asked for by FootprintAI/grandturks#1004, which is Phase 1 of that repo's docs/architecture/database-schema-provisioning.md.

Why

These .sql files are plain files in the module, and Go's embed cannot reach into a dependency — a consumer that imports restcol can see the package but not the files beside it. So a downstream service wanting to apply these migrations had exactly one option: copy them.

FootprintAI/grandturks#987 did that, into deploy/demo/restcol-migrations/ behind a drift test. It was the right stopgap for a deployment that was broken, but a copy of a file versioned elsewhere drifts — and silently in the direction that matters, because the copy keeps working while the pinned version moves underneath it.

Exporting the FS makes "the version pinned" and "the migrations applied" the same fact by construction.

What changed

migrations/embed.go — one //go:embed *.sql and an exported EmbeddedMigrations, the shape grandturks' own components/notification/broker/migrations already uses. Consumers read it with golang-migrate's iofs source driver:

src, err := iofs.New(migrations.EmbeddedMigrations, ".")
m, err := migrate.NewWithSourceInstance("iofs", src, databaseURL)
err = m.Up()

No new dependencies — embed is stdlib, and nothing here imports golang-migrate.

Tests

They are about coverage, not SQL. A migration on disk but not in the embed is invisible to every consumer, and nothing else in the build would say so:

  • TestEmbeddedMigrationsCoverEveryFileOnDisk — walks the directory and the embed FS and compares both the file set and the bytes. Verified it actually fails: dropping a .sql into a subdirectory produces extra/0003_x.up.sql is on disk but not in EmbeddedMigrations rather than passing quietly.
  • TestEveryMigrationHasBothDirections — an up without a down is a migration that cannot be rolled back, discovered when someone needs to roll it back.
  • TestNoEmbeddedMigrationIsEmpty — an empty file advances the version table without changing the schema, which is worse than a failure.

None of them needs a database.

Note on the existing suite

go test ./... is red on this branch — and equally red on a clean main, which I checked by stashing this change: integrationtest, pkg/app, and the three pkg/storage/* suites panic in AutoMigrate on a nil PostgresDb when no Postgres is configured. Pre-existing and unrelated to this PR, but worth stating plainly rather than reporting "tests pass": they don't, they just don't fail because of this. go build ./..., go vet ./migrations/, gofmt and go test ./migrations/ are all clean.

Once this merges, grandturks bumps its pin and deletes deploy/demo/restcol-migrations/ plus its drift test (that half is grandturks#1005).

These .sql files are plain files in the module, and Go's embed cannot reach
into a dependency - a consumer that imports restcol can see the package but
not the files beside it. So a downstream service wanting to apply these
migrations had only one option: copy them.

FootprintAI/grandturks#987 did exactly that, into
deploy/demo/restcol-migrations/ behind a drift test. It was the right
stopgap for a broken deployment, but a copy of a file versioned elsewhere
drifts - and silently in the direction that matters, since the copy keeps
working while the pinned version moves underneath it.

Exporting the FS makes "the version pinned" and "the migrations applied"
the same fact by construction. Consumers read it with golang-migrate's
iofs source driver.

The tests are about coverage rather than SQL: a migration on disk but not
in the embed is invisible to every consumer, and nothing else in the build
would say so. TestEmbeddedMigrationsCoverEveryFileOnDisk walks both and
compares, so a .sql file that lands outside the embed pattern - in a
subdirectory, say - fails instead of being silently skipped. Verified it
does fail that way before trusting it.

Asked for by FootprintAI/grandturks#1004.
@hsinatfootprintai
hsinatfootprintai merged commit 5d0e42c into main Aug 14, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant