diff --git a/.changeset/eql-3-0-5-migration.md b/.changeset/eql-3-0-5-migration.md new file mode 100644 index 000000000..32bdea778 --- /dev/null +++ b/.changeset/eql-3-0-5-migration.md @@ -0,0 +1,42 @@ +--- +'@cipherstash/stack-prisma': minor +--- + +Move the bundled EQL v3 migrations to **eql-3.0.5**, which renames the SQL +function `eql_v3.ste_vec_contains` to `eql_v3.jsonb_document_contains`. The +operators are unchanged (`@>` / `<@` on `public.eql_v3_json_search` behave +exactly as before) — only callers that invoke the function **by name** are +affected, which in practice means Supabase / PostgREST, since PostgREST calls +functions rather than operators. + +Two artefacts carry the new bundle: + +- A new upgrade edge, `20260814T0000_upgrade_eql_v3_3_0_5`, carrying the + invariant `cipherstash:upgrade-eql-v3-bundle-3.0.5-v1`. Databases already + running an earlier bundle re-install through this edge on the next + `prisma-next migrate`, exactly as they did for 3.0.2 and 3.0.4. +- The baseline install migration `20260601T0100_install_eql_v3_bundle`, whose + baked bundle moves to 3.0.5 and which gains a fourth no-SQL carrier op for + the new invariant. Fresh databases therefore land on 3.0.5 from the single + all-additive genesis edge, keeping `db init` (additive-only policy) working. + +**Action required.** The baseline's bytes — and so its `migrationHash` — have +changed. If your project already has a `migrations/cipherstash/` directory +generated against `@cipherstash/stack-prisma@1.0.0`, delete that directory and +re-run `prisma-next migration plan` (or `migrate`); the seed phase regenerates +it byte-identical to the shipped artefacts. Your database keeps its markers, so +already-applied invariants are not re-run — the only new work is the 3.0.5 +upgrade edge. + +**Why the baseline was re-emitted rather than left frozen.** These artefacts are +content-addressed and normally append-only: an EQL bump ships as a new upgrade +directory and published directories are never rewritten. That rule cannot be +followed here without a second `from: null` genesis edge, because no upgrade +edge can ever be walked by `db init` — every upgrade edge is a self-edge, and +the integrity checker requires a self-edge to carry a `data`-class op, which +`db init`'s additive-only policy refuses. A fresh database must therefore +collect every head-ref invariant from the genesis edge it walks. The +append-only alternative would duplicate the full ~2.6 MB bundle into a new +genesis edge on every EQL release, permanently; re-emitting was taken instead +while 1.0.0 was two weeks old with negligible adoption, and is a decision to be +re-argued on adoption numbers rather than repeated by default. diff --git a/.changeset/supabase-skill-eql-305.md b/.changeset/supabase-skill-eql-305.md new file mode 100644 index 000000000..a16a55495 --- /dev/null +++ b/.changeset/supabase-skill-eql-305.md @@ -0,0 +1,9 @@ +--- +'stash': patch +--- + +Update the bundled agent skills for eql-3.0.5. `skills/stash-supabase` +re-states the PostgREST query-domain limitations against 3.0.5 (unchanged in +substance — the typed `eql_v3.query_*` operand requirement still stands), and +`skills/stash-postgres` drops a claim that the CLI pins `@cipherstash/eql` to +an exact version, which stopped being true when EQL moved in-tree. diff --git a/.github/actionlint.yaml b/.github/actionlint.yaml index 21fa08b11..633dcb406 100644 --- a/.github/actionlint.yaml +++ b/.github/actionlint.yaml @@ -10,3 +10,9 @@ self-hosted-runner: labels: - blacksmith-4vcpu-ubuntu-2404 + # Every job in the imported EQL suite. Sixteen of them, and the size is + # deliberate upstream: the sharded SQLx matrix compiles the full Rust + # dependency tree, so the runner is chosen for the compile rather than for + # the tests. Kept as its own label rather than renamed to the 4vcpu one — + # rehoming those jobs is a cost decision, not a lint fix. + - blacksmith-16vcpu-ubuntu-2204 diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 57a53d23c..bd1c83c0d 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -149,6 +149,69 @@ updates: update-types: - version-update:semver-major + # ── Cargo (packages/eql — the second Rust workspace) ─────────── + # The EQL subtree brought a second Cargo workspace: five members + # (crates/eql-{domains,codegen,tests-macros,bindings} plus tests/sqlx) and + # its own Cargo.lock. It needs its own entry because Dependabot's cargo + # `directory:` is a single workspace root, not a glob — the protect-ffi entry + # above cannot reach it. The lockfile-to-ecosystem check in + # e2e/tests/supply-chain.e2e.test.ts asserts coverage per ECOSYSTEM, so it + # was already satisfied by that entry and would NOT have caught this gap; + # the symptom would simply have been that no PR ever arrived. + - package-ecosystem: cargo + # The workspace root, where Cargo.toml and Cargo.lock live. Not `/`, and + # not a crate directory — same trap as the protect-ffi entry documents. + directory: /packages/eql + # Monthly, matching protect-ffi and for the same reason: validating a + # Cargo.lock bump here means the sharded SQLx matrix in test-eql.yml, which + # compiles the full dependency tree and runs against live Postgres with + # CipherStash credentials. Security fixes are unaffected — those are driven + # by alerts, not by `schedule`. + # + # No `day:`, for the reason recorded above: it applies to `interval: weekly` + # and would be configuration that does nothing. + schedule: + interval: monthly + cooldown: + default-days: 7 + open-pull-requests-limit: 3 + labels: + - dependencies + - supply-chain + commit-message: + prefix: "chore" + include: scope + groups: + cargo-minor-patch: + patterns: + - "*" + update-types: + - minor + - patch + ignore: + # `cipherstash-client = "=0.42.0"` in tests/sqlx/Cargo.toml — the SAME + # exact pin, at the SAME version, as packages/protect-ffi. That is not a + # coincidence and it is the reason the subtree was imported: the two now + # share one release train, and a Dependabot PR that moved one workspace + # and not the other would reintroduce precisely the skew the absorption + # removed. Bump both, manually, in step with the npm catalog. + # + # Same caveat as above: `ignore` suppresses security PRs too. osv-scanner + # is the compensating control and already reaches this lockfile + # (`--recursive ./`). + - dependency-name: "cipherstash-client" + - dependency-name: "cts-common" + - dependency-name: "stack-auth" + - dependency-name: "stack-profile" + # Published from this workspace by release-plz. A Dependabot PR proposing + # a registry version for a crate we release here would fight the lockstep + # version hook (scripts/sync-lockstep-versions.mjs). + - dependency-name: "eql-bindings" + # Major bumps are reviewed and applied manually, not by Dependabot. + - dependency-name: "*" + update-types: + - version-update:semver-major + # ── GitHub Actions ───────────────────────────────────────────── - package-ecosystem: github-actions directory: / diff --git a/.github/workflows/bench-eql.yml b/.github/workflows/bench-eql.yml new file mode 100644 index 000000000..b49787ffa --- /dev/null +++ b/.github/workflows/bench-eql.yml @@ -0,0 +1,119 @@ +name: "Bench EQL" + +# Ported from cipherstash/encrypt-query-language with the subtree. See +# `.github/workflows/test-eql.yml` for the full account of what the move costs; +# this file needed the same four path rewrites plus two changes of its own, +# noted at the steps that carry them. +# +# Runs the slow benchmark / regression / scale SQLx tests gated behind the +# `bench` cargo feature. Not on pull requests — those use the fast `test-eql` +# workflow. +# Triggers: +# - push to main (catches regressions before release) +# - nightly schedule (additional smoke) +# - manual workflow_dispatch (PR triage) +on: + push: + branches: + - main + # Repo-root relative, so every entry gained the subtree prefix. Unprefixed, + # `src/**/*.sql` and `tests/sqlx/**/*` match nothing under this repository's + # root and the bench would simply stop running on pushes — silently, since + # a workflow that never triggers reports nothing at all. + paths: + - ".github/workflows/bench-eql.yml" + - "packages/eql/src/**/*.sql" + - "packages/eql/tests/sqlx/**/*" + - "packages/eql/tasks/**/*" + - "packages/eql/crates/**" + - "packages/eql/Cargo.toml" + - "packages/eql/Cargo.lock" + + schedule: + # 02:00 UTC daily + - cron: "0 2 * * *" + + workflow_dispatch: + +env: + FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true" + MISE_VERBOSE: "1" + +permissions: + contents: read + +defaults: + run: + shell: bash {0} + # Every `run:` here is written against the EQL root. + working-directory: packages/eql + +jobs: + bench: + name: "Bench EQL (Postgres 17)" + runs-on: blacksmith-16vcpu-ubuntu-2204 + timeout-minutes: 60 + + env: + POSTGRES_VERSION: "17" + + steps: + # SHA-pinned, where upstream used floating major tags. The rest of the + # imported suite already pins by SHA; a mutable tag on a job that holds + # live CipherStash credentials means the code running there can change + # without a commit here. Same three pins as test-eql.yml — keep them in + # step. (checkout moves v4 -> v6 with this, matching that file.) + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + + # Fast pre-flight: fail in seconds if a secret was rotated or cleared, + # before the rust-cache restore and a bench run budgeted at 60 minutes. + # This is a scheduled job, so nobody is watching it start — the difference + # between failing here and failing in `test:bench` is a legible nightly + # failure versus "Auth strategy error: Not authenticated" an hour in. + - uses: ./.github/actions/require-cs-secrets + with: + workspace-crn: ${{ vars.CS_WORKSPACE_CRN }} + client-id: ${{ vars.CS_CLIENT_ID }} + client-key: ${{ secrets.CS_CLIENT_KEY }} + client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }} + + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + cache: true + # mise reads config from cwd and its PARENTS, so an action running at + # the repo root never sees packages/eql/mise.toml. + working_directory: packages/eql + + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + with: + workspaces: packages/eql + shared-key: sqlx-tests + + - name: Setup database + run: | + mise run postgres:up postgres-${POSTGRES_VERSION} --extra-args "--detach --wait" + + - name: Run bench tests + # CS_* scoped to THIS step only (least privilege): test:bench -> test:sqlx:prep + # -> fixture:generate:all encrypts via cipherstash-client and needs BOTH a + # ZeroKMS auth credential (CS_CLIENT_ACCESS_KEY + CS_WORKSPACE_CRN) AND a client + # key (CS_CLIENT_ID + CS_CLIENT_KEY); without them it fails "Auth strategy error: + # Not authenticated". Kept off job scope so checkout/mise/rust-cache actions + # never see them. + # + # CS_WORKSPACE_CRN and CS_CLIENT_ID read from `vars.`, not `secrets.` — + # this repo keeps the two non-sensitive ones there. Reading them from + # `secrets.` yields the empty string rather than an error. + env: + CS_CLIENT_ACCESS_KEY: ${{ secrets.CS_CLIENT_ACCESS_KEY }} + CS_WORKSPACE_CRN: ${{ vars.CS_WORKSPACE_CRN }} + CS_CLIENT_ID: ${{ vars.CS_CLIENT_ID }} + CS_CLIENT_KEY: ${{ secrets.CS_CLIENT_KEY }} + run: | + export active_rust_toolchain=$(rustup show active-toolchain | cut -d' ' -f1) + rustup component add --toolchain ${active_rust_toolchain} rustfmt clippy + mise run --output prefix test:bench --postgres ${POSTGRES_VERSION} diff --git a/.github/workflows/fta-v3.yml b/.github/workflows/fta-v3.yml index fc2dff955..827612d73 100644 --- a/.github/workflows/fta-v3.yml +++ b/.github/workflows/fta-v3.yml @@ -82,10 +82,10 @@ jobs: # reflow trips this step without a real complexity increase, re-measure # (`npx fta src --format table`) before assuming the code got worse. - name: Analyze stack (eql/v3) complexity - run: pnpm --filter @cipherstash/stack run analyze:complexity + run: pnpm exec turbo run analyze:complexity --filter @cipherstash/stack - name: Analyze stack-drizzle complexity - run: pnpm --filter @cipherstash/stack-drizzle run analyze:complexity + run: pnpm exec turbo run analyze:complexity --filter @cipherstash/stack-drizzle - name: Analyze stack-supabase complexity - run: pnpm --filter @cipherstash/stack-supabase run analyze:complexity + run: pnpm exec turbo run analyze:complexity --filter @cipherstash/stack-supabase diff --git a/.github/workflows/macro-expand-eql.yml b/.github/workflows/macro-expand-eql.yml new file mode 100644 index 000000000..5496b5620 --- /dev/null +++ b/.github/workflows/macro-expand-eql.yml @@ -0,0 +1,95 @@ +name: "Macro expand EQL" + +# Ported from cipherstash/encrypt-query-language with the subtree. See +# `.github/workflows/test-eql.yml` for what the move costs (three path rewrites +# here) and `scripts/__tests__/eql-suite-ci.test.mjs` for why a workflow left in +# `packages/eql/.github/` runs nowhere. +# +# Regenerates the matrix `cargo expand` snapshots (one per reachable +# `scalar_matrix!` arm: integer = [eq, ord], text = [eq, ord, search], boolean = +# [storage]) and fails if any has drifted from its committed copy. This is a +# body-level fidelity backstop for the matrix macros — the name-inventory +# snapshot (test-eql.yml `matrix-coverage` job) catches add/remove of whole +# arms; this catches changes *inside* the generated bodies. +# +# Non-blocking by design: it is NOT a required PR check. `cargo expand` needs a +# nightly toolchain, so it is isolated off the PR path. +# - nightly schedule (the backstop that flags a forgotten local regen) +# - manual workflow_dispatch +# +# GAP (intended): there is no `pull_request` trigger, so a change that only +# touches macro *bodies* (no arm add/remove) can merge without ever running +# here and will first surface as a red nightly run afterwards. Accept this — the +# expand lane needs nightly and stays off the PR critical path by design. +# +# The pinned nightly date lives in ONE place: the `cargo +nightly-...` invocation +# in the `test:matrix:expand` mise task. The install step below DERIVES the date +# from mise.toml (grep), so there is nothing to keep in lockstep — bump it once in +# mise.toml. The snapshot then only moves when the macro moves, not when nightly +# reformats its expansion. +on: + schedule: + # 03:00 UTC daily + - cron: "0 3 * * *" + + workflow_dispatch: + +env: + FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true" + MISE_VERBOSE: "1" + +defaults: + run: + shell: bash {0} + # Every `run:` here is written against the EQL root: `mise.toml` for the + # nightly grep, and `tests/sqlx/snapshots` for the drift diff. + working-directory: packages/eql + +permissions: + contents: read + +jobs: + macro-expand: + name: "Macro expand drift (nightly)" + runs-on: blacksmith-16vcpu-ubuntu-2204 + timeout-minutes: 30 + + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + cache: true + working_directory: packages/eql + + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + with: + workspaces: packages/eql/tests/sqlx + shared-key: sqlx-tests + + # Derive the pinned nightly date from mise.toml (single source of truth — + # the `cargo +nightly-...` invocation in the `test:matrix:expand` task) so + # there is nothing to bump in lockstep here. cargo-expand is likewise pinned + # once in mise.toml's [tools] (`cargo:cargo-expand`) and installed by the + # mise-action step above, so its version is single-sourced too — no + # hardcoded version lives in this workflow. It drives the rustfmt pass, so + # an unpinned version could drift the snapshot even with a frozen macro + + # nightly. The snapshot then only moves when the macro moves. + - name: Install pinned nightly toolchain + run: | + NIGHTLY=$(grep -oE 'nightly-[0-9]{4}-[0-9]{2}-[0-9]{2}' mise.toml | head -1) + test -n "$NIGHTLY" || { echo "could not find pinned nightly in mise.toml"; exit 1; } + rustup toolchain install "$NIGHTLY" --profile minimal --component rustfmt + + - name: Regenerate and verify the matrix expansion snapshots + run: | + mise run test:matrix:expand + git diff --exit-code -- \ + tests/sqlx/snapshots/integer_expanded.rs \ + tests/sqlx/snapshots/text_expanded.rs \ + tests/sqlx/snapshots/boolean_expanded.rs \ + || { echo "Expansion snapshot stale — run 'mise run test:matrix:expand' (needs the pinned nightly) and commit."; exit 1; } diff --git a/.github/workflows/prisma-next-e2e.yml b/.github/workflows/prisma-next-e2e.yml index ae61fda56..6e6a5a12e 100644 --- a/.github/workflows/prisma-next-e2e.yml +++ b/.github/workflows/prisma-next-e2e.yml @@ -149,7 +149,7 @@ jobs: run: pnpm exec turbo run build --filter @cipherstash/stack-prisma - name: Emit example contract - run: pnpm --filter @cipherstash/prisma-example emit + run: pnpm exec turbo run emit --filter @cipherstash/prisma-example - name: Start E2E Postgres container working-directory: examples/prisma diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index ae03d708a..3c4245dcc 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -278,6 +278,15 @@ jobs: uses: changesets/action@v1.9.0 with: publish: pnpm run release + # LOAD-BEARING, and it fails OPEN if removed. Without `version:` the + # action runs its own built-in `changeset version` and never invokes + # the root `version` script — so `scripts/sync-lockstep-versions.mjs` + # would not run, npm would bump while + # packages/eql/crates/eql-bindings/Cargo.toml and the bundled SQL + # assets kept the old version, and the first symptom would be a + # published crate disagreeing with the SQL bundle it ships. + # Asserted by scripts/__tests__/release-version-hook.test.mjs. + version: pnpm run version commitMode: 'github-api' env: # No NPM_TOKEN — publishing authenticates via npm OIDC trusted diff --git a/.github/workflows/test-eql.yml b/.github/workflows/test-eql.yml new file mode 100644 index 000000000..b6cdf71db --- /dev/null +++ b/.github/workflows/test-eql.yml @@ -0,0 +1,750 @@ +name: "Test EQL" + +# The EQL suite, ported from cipherstash/encrypt-query-language when the subtree +# landed at `packages/eql`. It arrived under `packages/eql/.github/workflows/` — +# a directory GitHub never reads — so between the import and this file it ran +# NOWHERE. That is the same failure the protect-ffi absorption hit, and +# `scripts/__tests__/eql-suite-ci.test.mjs` is what stops it recurring. +# +# Four things changed in the port, all of them because the tree moved one level +# down. Each is silent when wrong: +# +# * `defaults.run.working-directory` — every `run:` here is written against +# the EQL root (`mise run …`, `release/*.sql`, `tests/sqlx/snapshots`). +# Without it they resolve against the monorepo root and fail with paths that +# look like typos. +# * `working_directory:` on each mise-action step — mise reads config from the +# current directory and its PARENTS, so an action running at the repo root +# never sees `packages/eql/mise.toml`. It would install nothing and leave the +# config untrusted, and the first `mise run` fails with "Config files … are +# not trusted", which reads as a toolchain problem rather than a path one. +# `defaults.run` does not reach a `uses:` step, so this is separate. +# * `workspaces:` on each rust-cache step — the Cargo workspace is nested now. +# * artifact `path:`/`path:` on upload/download — those are workspace-root +# relative and `defaults.run` does not touch them either. +# +# The credentials also changed shape. Upstream read all four from `secrets.`; +# this repo keeps the two non-sensitive ones in `vars.` (see +# `.github/actions/require-cs-secrets`), so `CS_WORKSPACE_CRN` and +# `CS_CLIENT_ID` are `vars.` here. Reading them from `secrets.` would not error +# — it yields the empty string, and the suite would fail at client construction +# in every shard instead of saying what was missing. +# +# NB: NO path filter at on: level. A workflow skipped by a path/branch filter +# leaves its required checks stuck Pending and blocks merge. Relevance is +# computed by the `changes` job and applied per-job via `if:` instead. +# +# NB: NO `push:` trigger, and that is not the same as "runs rarely". This runs +# on `pull_request` — every PR, unfiltered, with relevance applied per job — and +# on `merge_group`. Under a required merge queue, push-to-main validation is +# redundant: the queue already validated the exact merge commit, and branch +# protection blocks direct pushes. (Resolves the design's ambiguous "light jobs +# only on push:main" sanity net by dropping the trigger entirely.) +on: + pull_request: {} + merge_group: {} # required pre-merge gate; runs the full matrix + workflow_dispatch: {} # manual runs use the PR shape (PG17 x 4 shards) + +env: + FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true" + MISE_VERBOSE: "1" + # CI compile-time tuning (CI-only; local dev keeps full debuginfo + incremental). + # Clean CI builds never reuse incremental state, so it only bloats target/ and + # the rust-cache up/download. line-tables-only keeps readable panic backtraces + # for failing tests at a fraction of full-debuginfo compile cost. nextest's + # `test` profile inherits these from `dev`. + CARGO_INCREMENTAL: "0" + CARGO_PROFILE_DEV_DEBUG: "line-tables-only" + +defaults: + run: + shell: bash {0} + # Every `run:` in this workflow is written against the EQL root. This is the + # one-line form of what would otherwise be ~30 per-step `working-directory:` + # lines, each of which could be forgotten independently. + working-directory: packages/eql + +permissions: + contents: read + +# PRs cancel superseded runs; the merge queue must NOT cancel — a cancelled +# merge_group run never reports a final status and ejects the PR from the queue. +concurrency: + group: test-eql-${{ github.event_name }}-${{ github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +jobs: + # Runs on EVERY event and MUST always succeed (never skipped, never failed) — + # downstream heavy jobs `needs: [changes]`, and a skipped/failed `changes` + # would either skip the merge-queue matrix or deadlock `ci-required`. + changes: + name: "Detect relevant changes" + runs-on: blacksmith-16vcpu-ubuntu-2204 + outputs: + relevant: ${{ steps.r.outputs.relevant }} + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + + # Diff ONLY on pull_request, where a base ref is well-defined. On + # merge_group/workflow_dispatch the base ref is absent and the filter errors/empties. + - id: f + if: github.event_name == 'pull_request' + uses: dorny/paths-filter@d1c1ffe0248fe513906c8e24db8ea791d46f8590 # v3 + with: + # Every path except the workflow itself gained the `packages/eql/` + # prefix with the subtree. dorny/paths-filter matches against + # repo-root-relative paths, so the unprefixed globs would match + # `packages/stack/src/**` and friends instead — firing the whole + # matrix on changes that cannot affect EQL, and (worse) NOT firing on + # EQL's own `src/`, since nothing at the repo root is called that. + filters: | + relevant: + - ".github/workflows/test-eql.yml" + - "packages/eql/src/**" + - "packages/eql/sql/**" + - "packages/eql/tests/**" + - "packages/eql/tasks/**" + - "packages/eql/crates/**" + - "packages/eql/Cargo.toml" + - "packages/eql/Cargo.lock" + - "packages/eql/mise.toml" + + # Explicit default (not `|| 'true'`, which trips GitHub's inconsistent + # treatment of the string 'false'). merge_group/workflow_dispatch never + # read this value — their downstream `if:` branch ignores `relevant` — but + # default true is the safe value regardless. + - id: r + run: | + if [ "${{ github.event_name }}" = "pull_request" ]; then + echo "relevant=${{ steps.f.outputs.relevant }}" >> "$GITHUB_OUTPUT" + else + echo "relevant=true" >> "$GITHUB_OUTPUT" + fi + + # Pure bash; no checkout/toolchain. Derives the PG-version + shard fan-out + # from the event: PR -> PG17 x 4 shards; merge queue -> PG 14-17 x 2 shards. + setup: + name: "Compute matrix" + runs-on: blacksmith-16vcpu-ubuntu-2204 + # Opts out of the workflow-level `working-directory: packages/eql`. This job + # deliberately does not check out — it is pure bash deriving a matrix — so + # `packages/eql` does not exist on the runner and every `run:` step fails + # before its first line, with `chdir: No such file or directory`. Guarded by + # `scripts/__tests__/workflow-working-directory.test.mjs`. + defaults: + run: + working-directory: . + outputs: + pg-versions: ${{ steps.cfg.outputs.pg }} + shard-total: ${{ steps.cfg.outputs.shard_total }} + shards: ${{ steps.cfg.outputs.shards }} + steps: + - id: cfg + run: | + if [ "${{ github.event_name }}" = "merge_group" ]; then + echo 'pg=[14,15,16,17]' >> "$GITHUB_OUTPUT" + echo 'shard_total=2' >> "$GITHUB_OUTPUT" + echo 'shards=[1,2]' >> "$GITHUB_OUTPUT" + else + echo 'pg=[17]' >> "$GITHUB_OUTPUT" + echo 'shard_total=4' >> "$GITHUB_OUTPUT" + echo 'shards=[1,2,3,4]' >> "$GITHUB_OUTPUT" + fi + + # Compile the test binaries ONCE. Runs in the queue and on workflow_dispatch + # always, and on PRs only when relevant files changed (docs-only PRs never pay + # the ~4-min compile). + build-archive: + name: "Build test archive" + needs: [changes] + # This repo is PUBLIC and accepts fork PRs (the approval policy only gates + # first-time contributors). build-archive is the sole holder of the CS_* + # credentials below, so it must never run on a fork PR. The trailing clause + # restricts it to push / workflow_dispatch / same-repo branch PRs; the + # downstream test/validate shards `needs:` it, so they skip on fork PRs too. + if: >- + (github.event_name == 'merge_group' + || github.event_name == 'workflow_dispatch' + || (github.event_name == 'pull_request' && needs.changes.outputs.relevant == 'true')) + && (github.event_name != 'pull_request' + || github.event.pull_request.head.repo.full_name == github.repository) + runs-on: blacksmith-16vcpu-ubuntu-2204 + env: + # test:sqlx:archive depends on test:sqlx:prep, which copies the built EQL + # into migrations/, applies it to a live Postgres, and regenerates the + # per-type fixtures — both are include_str!'d into the test binaries at + # COMPILE time, so they must exist before `cargo nextest archive`. Fixture + # generation needs a live PG with EQL installed (the postgres:up step + # below) plus CS_* creds. The job-level `if:` above keeps those creds off + # fork-PR runs. + POSTGRES_VERSION: "17" + CS_CLIENT_ACCESS_KEY: ${{ secrets.CS_CLIENT_ACCESS_KEY }} + CS_WORKSPACE_CRN: ${{ vars.CS_WORKSPACE_CRN }} + CS_CLIENT_ID: ${{ vars.CS_CLIENT_ID }} + CS_CLIENT_KEY: ${{ secrets.CS_CLIENT_KEY }} + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + + # Fast pre-flight: fail in seconds if a secret was rotated or cleared, + # before the rust-cache restore and the ~4-minute nextest archive build. + # Ordering is asserted by scripts/__tests__/ffi-binding-step-order.test.mjs. + - uses: ./.github/actions/require-cs-secrets + with: + workspace-crn: ${{ vars.CS_WORKSPACE_CRN }} + client-id: ${{ vars.CS_CLIENT_ID }} + client-key: ${{ secrets.CS_CLIENT_KEY }} + client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }} + + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + cache: true + working_directory: packages/eql + + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + with: + workspaces: packages/eql + shared-key: sqlx-tests + # The sole saver of the shared cache: this job compiles the full heavy + # dep tree, so it must own the `sqlx-tests` key. All other jobs set + # `save-if: false` so a fast-finishing light job can't win the save race + # and overwrite the key with a deps-less target/. + + - name: Setup database (Postgres 17) + run: | + mise run postgres:up postgres-${POSTGRES_VERSION} --extra-args "--detach --wait" + + - name: Build EQL + archive test binaries + run: | + mise run test:sqlx:archive + + # Ship the built release artifacts: build_validation_tests read + # cipherstash-encrypt{,-uninstall}.sql from ../../release at RUN time + # (std::fs, not embedded), and release/ is gitignored so the shard checkout + # has none of them. `mise run build` (via prep) produced them in build-archive. + # + # `path:` is workspace-root relative — `defaults.run.working-directory` + # does not reach a `uses:` step — hence the prefix. upload-artifact v4 + # roots the archive at the least common ancestor of everything it matched, + # which for these two is `packages/eql`, so the artifact still contains + # `nextest.tar.zst` and `release/*.sql` at its top level and the shard's + # `download-artifact` puts them back where the archive expects them. + - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 + with: + name: nextest-archive + path: | + packages/eql/nextest.tar.zst + packages/eql/release/*.sql + retention-days: 1 + if-no-files-found: error + + # Sharded sqlx suite. No longer needs [schema, codegen] (gate removed) — + # shards start right after build-archive. + test: + name: "Shard PG${{ matrix.postgres-version }} ${{ matrix.shard }}/${{ needs.setup.outputs.shard-total }}" + needs: [changes, setup, build-archive] + if: >- + github.event_name == 'merge_group' + || github.event_name == 'workflow_dispatch' + || (github.event_name == 'pull_request' && needs.changes.outputs.relevant == 'true') + runs-on: blacksmith-16vcpu-ubuntu-2204 + strategy: + fail-fast: false + matrix: + postgres-version: ${{ fromJSON(needs.setup.outputs.pg-versions) }} + shard: ${{ fromJSON(needs.setup.outputs.shards) }} + env: + POSTGRES_VERSION: ${{ matrix.postgres-version }} + SHARD: ${{ matrix.shard }} + SHARD_TOTAL: ${{ needs.setup.outputs.shard-total }} + # No CS_* here: the shard runs the prebuilt archive (fixtures + migration + # embedded by build-archive), so it does not regenerate fixtures and needs + # no credentials. It only needs the live Postgres (below) for sqlx::test's + # per-test scratch databases and the release/*.sql from the artifact. + steps: + # Checkout path MUST be identical to build-archive so the archive's + # workspace remap lines up (design: archive<->commit coupling). + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + cache: true + working_directory: packages/eql + + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + with: + workspaces: packages/eql + shared-key: sqlx-tests + save-if: false + + # Extracts to the workspace root by default, which would drop + # `nextest.tar.zst` and `release/` two levels above where the archive's + # workspace remap and the build_validation_tests expect them. + - uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 + with: + name: nextest-archive + path: packages/eql + + - name: Setup database (Postgres ${{ matrix.postgres-version }}) + run: | + mise run postgres:up postgres-${POSTGRES_VERSION} --extra-args "--detach --wait" + + - name: Run shard ${{ matrix.shard }}/${{ needs.setup.outputs.shard-total }} + run: | + mise run test:sqlx:partition + + # docs:validate + Clean-DB v3 install smoke. Both are version-relevant, so + # they follow the event's PG set (PG17 on PR; 14-17 in the queue). Moved out + # of the old per-version test job so they run ONCE per version, not per shard. + validate: + name: "Validate (Postgres ${{ matrix.postgres-version }})" + needs: [changes, setup] + if: >- + github.event_name == 'merge_group' + || github.event_name == 'workflow_dispatch' + || (github.event_name == 'pull_request' && needs.changes.outputs.relevant == 'true') + runs-on: blacksmith-16vcpu-ubuntu-2204 + strategy: + fail-fast: false + matrix: + postgres-version: ${{ fromJSON(needs.setup.outputs.pg-versions) }} + env: + POSTGRES_VERSION: ${{ matrix.postgres-version }} + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + cache: true + working_directory: packages/eql + + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + with: + workspaces: packages/eql + shared-key: sqlx-tests + save-if: false + + - name: Setup database (Postgres ${{ matrix.postgres-version }}) + run: | + mise run postgres:up postgres-${POSTGRES_VERSION} --extra-args "--detach --wait" + + # Source-only doc checks (coverage + required-tags) moved to the dedicated + # `docs-static` job so they run exactly once, not per-Postgres. This step + # keeps only the DB-backed SQL-syntax validation, which genuinely needs the + # per-version Postgres. + - name: Validate documented SQL syntax (Postgres ${{ matrix.postgres-version }}) + run: | + mise run docs:validate:documented-sql + + - name: Clean-DB v3 install smoke (Postgres ${{ matrix.postgres-version }}) + run: | + mise run clean && mise run build + mise run test:clean_install_v3 + + schema: + name: "JSON Schema validation" + needs: [changes] + if: >- + github.event_name == 'merge_group' + || github.event_name == 'workflow_dispatch' + || (github.event_name == 'pull_request' && needs.changes.outputs.relevant == 'true') + runs-on: blacksmith-16vcpu-ubuntu-2204 + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + cache: true + working_directory: packages/eql + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + with: + workspaces: packages/eql + shared-key: sqlx-tests + save-if: false + - name: Validate v2.2 / v2.3 payload schemas + run: | + mise run test:schema + + rust-crates: + name: "Rust workspace crates" + needs: [changes] + if: >- + github.event_name == 'merge_group' + || github.event_name == 'workflow_dispatch' + || (github.event_name == 'pull_request' && needs.changes.outputs.relevant == 'true') + runs-on: blacksmith-16vcpu-ubuntu-2204 + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + cache: true + working_directory: packages/eql + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + with: + workspaces: packages/eql + shared-key: sqlx-tests + save-if: false + # `mise run test:crates` runs `cargo fmt --check` at the workspace root, + # which covers tests/sqlx (a workspace member). This subsumes the old + # standalone `test:lint` step that the removed per-version test job ran. + - name: Compile, lint and test the Rust workspace crates + run: | + export active_rust_toolchain=$(rustup show active-toolchain | cut -d' ' -f1) + rustup component add --toolchain ${active_rust_toolchain} rustfmt clippy + mise run test:crates + + # Freshness gate for the eql-types codegen output: regenerate the + # TypeScript bindings and JSON Schemas and fail if the checked-in + # copies differ. Reuses the toolchain from the step above. + - name: Verify eql-types bindings and schemas are fresh + run: | + mise run types:check + + # Publish gate for eql-bindings (the one crate we ship to crates.io via + # release-plz). `--dry-run` packages + compiles the crate exactly as + # crates.io would, catching publish-blockers — missing `license`/metadata, + # a real path dependency without a version — on the PR rather than at + # release time. No token needed. `--allow-dirty` tolerates any files the + # preceding regenerate-and-diff steps leave in the working tree. + - name: Verify eql-bindings packages cleanly for crates.io + run: | + cargo publish -p eql-bindings --dry-run --allow-dirty + + codegen: + name: "Encrypted-domain codegen" + needs: [changes] + if: >- + github.event_name == 'merge_group' + || github.event_name == 'workflow_dispatch' + || (github.event_name == 'pull_request' && needs.changes.outputs.relevant == 'true') + runs-on: blacksmith-16vcpu-ubuntu-2204 + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + cache: true + working_directory: packages/eql + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + with: + workspaces: packages/eql + shared-key: sqlx-tests + save-if: false + - name: Verify generator parity (golden) + run: | + mise run codegen:parity + - name: Verify schema split parity (SQL owned_schemas vs Rust consts) + run: | + mise run test:schemas:parity + + self-contained-v3: + name: "eql_v3 self-containment" + needs: [changes] + if: >- + github.event_name == 'merge_group' + || github.event_name == 'workflow_dispatch' + || (github.event_name == 'pull_request' && needs.changes.outputs.relevant == 'true') + runs-on: blacksmith-16vcpu-ubuntu-2204 + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + cache: true + working_directory: packages/eql + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + with: + workspaces: packages/eql + shared-key: sqlx-tests + save-if: false + - name: Build EQL + run: mise run clean && mise run --force build + - name: Assert eql_v3 is self-contained + run: mise run test:self_contained_v3 + - name: Assert the installer contains every ordered file + run: mise run test:installer_complete + - name: Symbol-order cross-check (v3) + run: mise run test:symbol_order_v3 + - name: Build-ordering helper unit tests + run: mise run test:build_ordering_helpers + + matrix-coverage: + name: "Matrix coverage inventory" + needs: [changes] + if: >- + github.event_name == 'merge_group' + || github.event_name == 'workflow_dispatch' + || (github.event_name == 'pull_request' && needs.changes.outputs.relevant == 'true') + runs-on: blacksmith-16vcpu-ubuntu-2204 + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + cache: true + working_directory: packages/eql + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + with: + workspaces: packages/eql + shared-key: sqlx-tests + save-if: false + - name: Verify the matrix test-name inventory + # The workflow-wide `shell: bash {0}` default drops GitHub's implicit + # `-eo pipefail`, so without fail-fast this multi-command step only + # reports the LAST command's exit code and a failing inventory task is + # silently swallowed. Restore fail-fast for this step. + run: | + set -euo pipefail + mise run test:matrix:inventory + mise run test:matrix:inventory:jsonb_entry + mise run test:matrix:inventory:ope + mise run test:v3-jsonb:inventory + git add -N tests/sqlx/snapshots + git diff --exit-code -- tests/sqlx/snapshots \ + || { echo "Coverage inventory stale — run the relevant inventory task and commit."; exit 1; } + - name: Verify catalog-surface coverage + run: mise run test:matrix:catalog-coverage + + splinter: + name: "Supabase splinter" + needs: [changes] + if: >- + github.event_name == 'merge_group' + || github.event_name == 'workflow_dispatch' + || (github.event_name == 'pull_request' && needs.changes.outputs.relevant == 'true') + runs-on: blacksmith-16vcpu-ubuntu-2204 + env: + POSTGRES_VERSION: "17" + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + cache: true + working_directory: packages/eql + - name: Setup database + run: | + mise run postgres:up postgres-${POSTGRES_VERSION} --extra-args "--detach --wait" + - name: Build and install EQL + run: | + mise run clean && mise run --output prefix --force build + cat release/cipherstash-encrypt.sql \ + | docker exec -i postgres-${POSTGRES_VERSION} \ + psql -v ON_ERROR_STOP=1 \ + postgresql://cipherstash:password@localhost/cipherstash -f- + - name: Run splinter + run: | + mise run --output prefix test:splinter --postgres ${POSTGRES_VERSION} + + # Source-only SQL documentation validation (coverage + required Doxygen tags). + # Deliberately NOT relevance-gated: it runs on EVERY pull_request — including + # docs-only PRs that skip the heavy jobs — so documentation is always + # validated. DB-free and creds-free (the psql-backed syntax check stays in the + # per-version `validate` job). + docs-static: + name: "SQL doc validation" + runs-on: blacksmith-16vcpu-ubuntu-2204 + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + cache: true + working_directory: packages/eql + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + with: + workspaces: packages/eql + shared-key: sqlx-tests + save-if: false + - name: Validate SQL doc coverage + required tags + run: | + mise run docs:validate:source + + - name: Assert user-facing docs are free of the removed eql_v2 surface + run: | + mise run test:docs_v3_grep + + - name: Assert public files contain no private issue identifiers + run: | + mise run test:public_identifiers + + # Every `known_failure` marker must name a real, OPEN issue, and its constant + # must actually be referenced by a test. The Rust half of the contract is + # self-expiring (the marker FAILS once the wrapped assertion starts passing); + # this is the half that stops a suppression outliving a closed issue. + # + # Credential-free and DB-free — it only reads the registry and asks GitHub for + # issue state, so it runs on every PR rather than hiding behind the e2e job. + known-failures: + name: "known-failure markers" + runs-on: blacksmith-16vcpu-ubuntu-2204 + permissions: + contents: read + issues: read + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + cache: true + working_directory: packages/eql + # The gate detects drift by cross-checking a loose declaration count against + # a strict parse, so an over-accepting parser defeats it silently. Runs + # first, and needs no token: a broken parser makes the gate below + # meaningless, not merely wrong. + - name: The gate's strict ISSUE_ parser is sound + run: | + mise run test:known-failures:parser + + - name: Every suppressed test names an open issue + env: + # `gh` needs a token to read issue state; the default job token is + # sufficient (issues: read, same repo). + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + mise run test:known-failures + + # Markdown anchor links. DB-free, credential-free and fast, and deliberately + # NOT relevance-gated: its inputs are the docs themselves, so gating it on the + # `relevant` filter (src/**, crates/**) would skip it on exactly the docs-only + # PRs it exists to check. + doc-anchors: + name: "doc anchor links" + runs-on: blacksmith-16vcpu-ubuntu-2204 + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + cache: true + working_directory: packages/eql + - name: Every intra-document anchor link resolves + run: | + mise run test:doc-anchors + + # The e2e (fresh-encryption) property suite. Encrypts random values through + # ZeroKMS at run time, so it needs CS_* creds and is PG-version-independent — + # one PG17 run, never the matrix. Compiles the `proptest-e2e`-gated binaries + # (which the default-feature sharded archive excludes) and runs only the + # e2e oracle. Like build-archive, it holds CS_* and so carries the same + # fork-PR guard to keep the secrets off fork runs. + e2e: + name: "e2e property suite (fresh encryption)" + needs: [changes, setup] + if: >- + (github.event_name == 'merge_group' + || github.event_name == 'workflow_dispatch' + || (github.event_name == 'pull_request' && needs.changes.outputs.relevant == 'true')) + && (github.event_name != 'pull_request' + || github.event.pull_request.head.repo.full_name == github.repository) + runs-on: blacksmith-16vcpu-ubuntu-2204 + env: + POSTGRES_VERSION: "17" + CS_CLIENT_ACCESS_KEY: ${{ secrets.CS_CLIENT_ACCESS_KEY }} + CS_WORKSPACE_CRN: ${{ vars.CS_WORKSPACE_CRN }} + CS_CLIENT_ID: ${{ vars.CS_CLIENT_ID }} + CS_CLIENT_KEY: ${{ secrets.CS_CLIENT_KEY }} + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + # Fast pre-flight: fail in seconds if a secret was rotated or cleared, + # before the rust-cache restore and the proptest-e2e compile. + - uses: ./.github/actions/require-cs-secrets + with: + workspace-crn: ${{ vars.CS_WORKSPACE_CRN }} + client-id: ${{ vars.CS_CLIENT_ID }} + client-key: ${{ secrets.CS_CLIENT_KEY }} + client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }} + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + cache: true + working_directory: packages/eql + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + with: + workspaces: packages/eql + shared-key: sqlx-tests + save-if: false + - name: Setup database (Postgres 17) + run: | + mise run postgres:up postgres-${POSTGRES_VERSION} --extra-args "--detach --wait" + - name: Run e2e property suite + run: | + mise run test:sqlx:e2e + + # The ONE required status check. Stable name on every event, so branch + # protection never references an event-dependent leaf name (which would + # deadlock). Passes iff every needed job is success or skipped. Treating + # skipped as pass is intentional: heavy jobs are legitimately skipped on + # docs-only PRs, and a genuine failure is still caught because the FAILING + # source job is itself in `needs` and reports failure. + ci-required: + name: "ci-required" + needs: [changes, setup, build-archive, test, validate, schema, rust-crates, + codegen, self-contained-v3, matrix-coverage, splinter, docs-static, + known-failures, doc-anchors, e2e] + if: always() + runs-on: blacksmith-16vcpu-ubuntu-2204 + # Same opt-out as `setup`, and this is the worse of the two to get wrong: + # this job is the required check. It reads `needs.*.result` and checks out + # nothing, so under the workflow-level `working-directory` it fails for a + # reason unrelated to any job it is reporting on — a red required check that + # says nothing about the suite. + defaults: + run: + working-directory: . + steps: + - name: Assert all required jobs passed or were skipped + run: | + results='${{ join(needs.*.result, ' ') }}' + echo "needed results: $results" + for r in $results; do + case "$r" in + success|skipped) ;; + *) echo "gate fail: a needed job reported '$r'"; exit 1 ;; + esac + done + echo "ci-required: all needed jobs passed or were skipped" diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 435c113ef..fcc766b6b 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -128,14 +128,14 @@ jobs: uses: ./.github/actions/build-ffi-binding - name: Type tests (stack) - run: pnpm --filter @cipherstash/stack run test:types + run: pnpm exec turbo run test:types --filter @cipherstash/stack # The v3 domain catalog lives in the test kit, and its # `satisfies Record` is what forces a new SDK # domain to be covered. That check only fires under `tsc`, so without this # step a domain added to stack would slip through untested. - name: Type tests (test-kit — enforces v3 domain coverage) - run: pnpm --filter @cipherstash/test-kit run test:types + run: pnpm exec turbo run test:types --filter @cipherstash/test-kit # The adapter packages carry their own `.test-d.ts` type-contract guards # (the M1 client-surface / #622 erasure guards, the Supabase key-set @@ -143,19 +143,29 @@ jobs: # were never wired into CI, so type-level regressions in them went # undetected — run them explicitly. - name: Type tests (stack-drizzle) - run: pnpm --filter @cipherstash/stack-drizzle run test:types + run: pnpm exec turbo run test:types --filter @cipherstash/stack-drizzle - name: Type tests (stack-supabase) - run: pnpm --filter @cipherstash/stack-supabase run test:types + run: pnpm exec turbo run test:types --filter @cipherstash/stack-supabase # prisma-next's operator-capability gating is proven by `.test-d.ts` # `@ts-expect-error` assertions (an unsupported operator on a column # must be a compile error). Those only fire when tsc processes them, # so the package `typecheck` must run — and be required — here (#684). - # Its tsconfig resolves stack subpaths to SOURCE, so no build step is - # needed first. + # Its tsconfig resolves stack subpaths to SOURCE — but NOT + # `@cipherstash/eql`, which it reaches through the package `exports` map + # at `./dist/sql.d.ts`. That used to be a registry tarball with `dist/` + # already in it, so a bare `pnpm --filter` resolved it whatever CI had + # built. The EQL subtree import made it a workspace package whose `dist/` + # is a build output, and this step became the first in the job to need + # something built — failing `TS2307` on three files. + # + # Through turbo, so `typecheck`'s `dependsOn: ["^build"]` builds + # `@cipherstash/eql` first. This is the trap + # `workflow-turbo-build-deps.test.mjs` was written for; the step was on + # its grandfathered KNOWN_BARE list, and the import is what collected. - name: Typecheck (prisma-next — enforces v3 operator-capability gating) - run: pnpm --filter @cipherstash/stack-prisma run typecheck + run: pnpm exec turbo run typecheck --filter @cipherstash/stack-prisma # `packages/bench` is a live importer of `@cipherstash/stack` and # `@cipherstash/stack-drizzle`, but it has no `test` script (its suites @@ -171,7 +181,7 @@ jobs: # nothing here caught the three `auth.AutoStrategy` resolution errors that # sat in `main` until #771. Gate it so they cannot come back silently. - name: Typecheck (wizard) - run: pnpm --filter @cipherstash/wizard run typecheck + run: pnpm exec turbo run typecheck --filter @cipherstash/wizard # `examples/*` are standalone apps outside the `./packages/*` filter that # root `build`/`test` use, so nothing in CI compiled them. `examples/basic` @@ -246,6 +256,13 @@ jobs: - name: Lint — no references to deleted package directories run: pnpm run lint:package-paths + # `eql-bindings` emits EQL payloads; `@cipherstash/eql` carries the SQL + # that stores them. Both live here now and release at one lockstep + # version. A registry pin on either lets them drift apart — it compiles, + # it passes every suite, and it fails in a customer's database. + - name: Lint — EQL resolves in-tree, not from a registry + run: pnpm run lint:eql-pins + # TEMPORARY — delete with the script in the phase-4 cutover PR. # The seven protect-ffi packages live here but still publish from # cipherstash/protectjs-ffi. A changeset naming any of them bumps all diff --git a/biome.json b/biome.json index 63fb55273..987de3c3d 100644 --- a/biome.json +++ b/biome.json @@ -10,6 +10,12 @@ "!packages/protect-ffi/lib", "!packages/protect-ffi/target", "!packages/protect-ffi/src/eql-v3-types", + "!packages/eql/crates/eql-bindings/bindings", + "!packages/eql/crates/eql-bindings/schema", + "!packages/eql/packages/eql/src/generated", + "!packages/eql/release", + "!packages/eql/target", + "!packages/eql/docs/api", "!**/*.grit", "!**/*.generated.ts", "!**/contract.json", diff --git a/docs/plans/2026-08-13-eql-monorepo-absorption.md b/docs/plans/2026-08-13-eql-monorepo-absorption.md new file mode 100644 index 000000000..2f2a7f4ea --- /dev/null +++ b/docs/plans/2026-08-13-eql-monorepo-absorption.md @@ -0,0 +1,387 @@ +# EQL monorepo absorption — implementation plan + +**Goal:** Build, version and publish the EQL subsystem — the `@cipherstash/eql` npm package, the `eql-bindings` crate, the SQL install bundle, the API docs, the `postgres-eql` Docker image and the dbdev extension — from this repository instead of `cipherstash/encrypt-query-language`, and eliminate the EQL version skew between the Rust that emits payloads and the SQL that stores them. + +**Architecture:** A verbatim-prefix `git subtree` import at `packages/eql/`, so every repo-root-relative path inside EQL keeps resolving. The npm package therefore lands at `packages/eql/packages/eql` and needs its own pnpm-workspace glob — the `packages/protect-ffi/platforms/*` precedent. Publishing moves surface by surface, each inert behind a guard until its own cutover, as `scripts/lint-no-ffi-changeset.mjs` does for the FFI packages today. + +**Tech Stack:** pnpm 10.33.2 workspaces + catalogs, Turborepo 2.10.4, Changesets 2.31, release-plz + crates.io OIDC, Rust/Cargo (a second nested workspace), SQLx + cargo-nextest, mise (a second nested `mise.toml`), Doxygen + Python 3.13, Docker/GHCR, Vitest 3.2.7, Biome 2.5.3. + +--- + +## Global constraints + +Inherited from the protect-ffi absorption, all still binding: + +- **Root `pnpm test` must never invoke cargo.** `turbo test --filter './packages/*'` reaches every package; a cargo process on that path is a Rust toolchain on every contributor's machine. EQL's `codegen:parity` drift gate and the whole SQLx suite are cargo, so they need the same `test:cargo` split protect-ffi has, enforced the same way. +- **Publish workflows must never restore the GitHub Actions cache** (`scripts/lint-no-workflow-caching.mjs`). EQL's own release workflows already respect this — EQL is where that rule came from — but the reusable `_build-sql.yml` / `_build-docs.yml` need re-checking against this repo's target list once they move. +- **npm and crates.io trusted publishing both validate against the entry-point workflow filename**, and both require the manifest's repository field to match the publishing repository exactly. Seven EQL manifests plus `release-plz.toml` name `cipherstash/encrypt-query-language` today. +- **Do not write `pnpm run