ci: faster, cheaper, consistently named CI

.agents/skills/ship/SKILL.md

@@ -47,7 +47,7 @@ When the user runs `/ship`:
   - Run `/db-migrate` to review the migration for zero-downtime safety (expand/contract phasing, backward-compatibility with the deployed app version).
   - `(cd packages/db && bunx drizzle-kit generate && git status --porcelain ./migrations)` must print nothing (CI's schema/migration sync step).
   - `bun run check:migrations origin/staging` must pass (staging is the PR base). Do not silence a flagged statement with a `-- migration-safe:` annotation unless `/db-migrate` confirmed the old code no longer depends on it; otherwise split the destructive change into a later deploy.
-6. **Run pre-ship checks** from the repo root before staging. This has two phases: first **regenerate** every committed artifact so generated files never drift into a CI failure (this is what catches things like `agent-stream-docs` going stale after a `models.ts` edit), then run the **full audit suite** CI's `Lint and Test` job enforces. Both phases parallelize — but only across commands that write **disjoint** outputs — and a bare `wait` swallows child exit codes, so both phases below explicitly collect each job's status and abort ship if any failed.
+6. **Run pre-ship checks** from the repo root before staging. This has two phases: first **regenerate** every committed artifact so generated files never drift into a CI failure (this is what catches things like `agent-stream-docs` going stale after a `models.ts` edit), then run the **full audit suite** CI's `lint` job enforces. Both phases parallelize — but only across commands that write **disjoint** outputs — and a bare `wait` swallows child exit codes, so both phases below explicitly collect each job's status and abort ship if any failed.
 
   **Phase A — regenerate the always-in-repo committed artifacts (parallel), then let step 7 stage whatever changed.** Regenerate only the generators whose inputs live entirely in this repo and that any ordinary code change can drift — `agent-stream-docs:generate` (derives from the provider model registry), `docs-manifest:generate` (derives from docs page paths), and `skills:sync` (derives from `.agents/skills/**`). They write disjoint outputs (`apps/docs/…/agent.mdx`, `apps/sim/lib/mothership/generated/docs-manifest.ts`, and `.claude/skills` links), so they parallelize safely, and each is idempotent (a no-op when already in sync):
   ```bash
@@ -66,7 +66,7 @@ When the user runs `/ship`:
 
   **Do NOT blanket-run the domain generators here.** `mship:generate` (`generate-mship-contracts.ts`) is an **umbrella** that drives all nine mothership contract generators (`mship-contracts`, `billing-protocol-contract`, `mship-tools`, the four `trace-*`, `metrics-contract`, `vfs-snapshot-contract`) and biome-formats `apps/sim/lib/mothership/generated/` — never run it *and* its constituents (they write the same files and corrupt each other in parallel), and never run it on an ordinary ship: it reads an **external** copilot-contract source that isn't checked out in most worktrees, so it hard-fails with `ENOENT` and would abort ship for an unrelated reason. `generate:pi-model-catalog` (under `apps/sim`) likewise regenerates from the installed Pi package, not repo source. `scripts/generate-docs.ts` rewrites the integration docs and client-safe catalog; run it when this PR changes their block/icon/landing-content inputs or when `integration-catalog:check` reports drift, then review its broad generated diff. Only when **this PR's diff actually touches** a domain generator's input do you regenerate it deliberately and run its matching `:check` (`bun run mship:check` / the individual `*:check`) — with the external source present.
 
-  **Phase B — run lint + every audit CI enforces, in parallel, and abort ship if any fails.** Before running the commands, compare this list with `.github/workflows/test-build.yml`; when CI adds an audit, run it and update this skill instead of trusting a stale snapshot. The env-flag audit is currently an inline workflow block rather than a package script: when `apps/sim/lib/core/config/env-flags.ts` changed, run that current workflow block verbatim instead of copying a second version into this skill. Run `bun run lint` first (it autofixes formatting and mutates files, so don't parallelize it with the read-only audits), then run the base-sensitive block-registry check, then fan the independent audits out and collect exit codes:
+  **Phase B — run lint + every audit CI enforces, in parallel, and abort ship if any fails.** Before running the commands, compare this list with `.github/workflows/checks.yml`; when CI adds an audit, run it and update this skill instead of trusting a stale snapshot. The env-flag audit is currently an inline workflow block rather than a package script: when `apps/sim/lib/core/config/env-flags.ts` changed, run that current workflow block verbatim instead of copying a second version into this skill. Run `bun run lint` first (it autofixes formatting and mutates files, so don't parallelize it with the read-only audits), then run the base-sensitive block-registry check, then fan the independent audits out and collect exit codes:
   ```bash
   # autofix formatting first (mutating; not parallel-safe with the audits). Gate its exit too —
   # a non-zero lint (unfixable errors) must abort before the audits run, not be ignored.

.claude/rules/sim-testing.md

@@ -32,11 +32,11 @@ contracts, and demonstrated regressions.
 
 | Suffix | Needs | Run with | In CI |
 |--------|-------|----------|-------|
-| `*.test.ts(x)` | nothing; global mocks from `vitest.setup.ts` | `vitest run` | Lint and Test job |
-| `*.integration.ts` | real PostgreSQL (`TEST_DATABASE_URL`), optionally Redis (`TEST_REDIS_URL`) | `vitest run --mode integration` | `PostgreSQL integration` job, by glob |
+| `*.test.ts(x)` | nothing; global mocks from `vitest.setup.ts` | `vitest run` | `test` jobs (sharded) |
+| `*.integration.ts` | real PostgreSQL (`TEST_DATABASE_URL`), optionally Redis (`TEST_REDIS_URL`) | `vitest run --mode integration` | `integration` jobs, by glob (sharded per provisioning path) |
 | `*.live.test.ts` | provider APIs, hosted sandboxes, local runtimes, or sibling checkouts | `vitest run --mode live <file>` (apps/sim) | never |
 | `apps/desktop/e2e/*.spec.ts` | the packaged Electron app | Playwright | desktop E2E workflow |
-| `apps/sim/scripts/test-*-e2e.ts` | a running app over HTTP | its `package.json` script when one exists (`bun run test:scim:e2e`; `test:workflow-version-compare:e2e` adds `--no-env-file`), else `bun scripts/test-<suite>-e2e.ts` from apps/sim | End-to-end over real HTTP job |
+| `apps/sim/scripts/test-*-e2e.ts` | a running app over HTTP | its `package.json` script when one exists (`bun run test:scim:e2e`; `test:workflow-version-compare:e2e` adds `--no-env-file`), else `bun scripts/test-<suite>-e2e.ts` from apps/sim | `e2e` jobs (`.github/scripts/http-e2e.sh`) |
 
 - A unit test lives next to its source: `feature.ts` → `feature.test.ts`. No network, no database,
   no real timers.

.cursor/rules/sim-testing.mdc

@@ -30,11 +30,11 @@ contracts, and demonstrated regressions.
 
 | Suffix | Needs | Run with | In CI |
 |--------|-------|----------|-------|
-| `*.test.ts(x)` | nothing; global mocks from `vitest.setup.ts` | `vitest run` | Lint and Test job |
-| `*.integration.ts` | real PostgreSQL (`TEST_DATABASE_URL`), optionally Redis (`TEST_REDIS_URL`) | `vitest run --mode integration` | `PostgreSQL integration` job, by glob |
+| `*.test.ts(x)` | nothing; global mocks from `vitest.setup.ts` | `vitest run` | `test` jobs (sharded) |
+| `*.integration.ts` | real PostgreSQL (`TEST_DATABASE_URL`), optionally Redis (`TEST_REDIS_URL`) | `vitest run --mode integration` | `integration` jobs, by glob (sharded per provisioning path) |
 | `*.live.test.ts` | provider APIs, hosted sandboxes, local runtimes, or sibling checkouts | `vitest run --mode live <file>` (apps/sim) | never |
 | `apps/desktop/e2e/*.spec.ts` | the packaged Electron app | Playwright | desktop E2E workflow |
-| `apps/sim/scripts/test-*-e2e.ts` | a running app over HTTP | its `package.json` script when one exists (`bun run test:scim:e2e`; `test:workflow-version-compare:e2e` adds `--no-env-file`), else `bun scripts/test-<suite>-e2e.ts` from apps/sim | End-to-end over real HTTP job |
+| `apps/sim/scripts/test-*-e2e.ts` | a running app over HTTP | its `package.json` script when one exists (`bun run test:scim:e2e`; `test:workflow-version-compare:e2e` adds `--no-env-file`), else `bun scripts/test-<suite>-e2e.ts` from apps/sim | `e2e` jobs (`.github/scripts/http-e2e.sh`) |
 
 - A unit test lives next to its source: `feature.ts` → `feature.test.ts`. No network, no database,
   no real timers.

.github/actions/cache-mount/action.yml → .github/actions/cache/action.yml

@@ -1,4 +1,4 @@
-name: Cache Mount
+name: cache
 description: Mount a build cache directory using Blacksmith sticky disks, or the GitHub Actions cache when running on GitHub-hosted runners.
 
 inputs:

.github/actions/docker-build/action.yml → .github/actions/image/action.yml

@@ -1,4 +1,4 @@
-name: Docker Build and Push
+name: image
 description: Set up a buildx builder and build/push an image, using Blacksmith's builder or the upstream Docker actions on GitHub-hosted runners.
 
 inputs:
@@ -35,7 +35,7 @@ inputs:
     required: false
 
 # Registry logins must precede this action. provenance/sbom stay off: attestation
-# manifests break `imagetools create` retagging in promote-images.
+# manifests break `imagetools create` retagging in the `promote` job of ci.yml.
 runs:
   using: composite
   steps:

.github/actions/setup-workspace/action.yml → .github/actions/setup/action.yml

@@ -1,15 +1,23 @@
-name: Setup Workspace
+name: setup
 description: Install the pinned Bun and Node toolchain, mount the dependency (and optionally Turbo) caches, and install workspace dependencies.
 
 inputs:
   provider:
-    description: The CI_PROVIDER repo variable, forwarded to cache-mount.
+    description: The CI_PROVIDER repo variable, forwarded to the cache action.
     required: false
     default: ''
   turbo-cache-key:
     description: Suffix for a Turbo cache mounted at ./.turbo. Empty skips the mount. Jobs that write Turbo entries need distinct suffixes, or last-writer-wins commits evict each other's entries.
     required: false
     default: ''
+  node-version:
+    description: Node version to install. Empty keeps the runner's preinstalled Node, for jobs that only ever ran Bun.
+    required: false
+    default: '24'
+  registry-url:
+    description: npm registry to write an .npmrc for, so `npm publish` reads NODE_AUTH_TOKEN. Empty writes none.
+    required: false
+    default: ''
 
 # Cache keys are scoped by event name, and fork PRs get their own namespace on
 # top: untrusted fork runs must never share a cache with push runs (whose caches
@@ -30,27 +38,29 @@ runs:
         bun-version: 1.4.2
 
     - name: Setup Node
+      if: inputs.node-version != ''
       uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
       with:
-        node-version: 24
+        node-version: ${{ inputs.node-version }}
+        registry-url: ${{ inputs.registry-url }}
 
     - name: Mount Bun cache
-      uses: ./.github/actions/cache-mount
+      uses: ./.github/actions/cache
       with:
         provider: ${{ inputs.provider }}
         key: ${{ github.repository }}-bun-cache-${{ github.event_name }}${{ github.event.pull_request.head.repo.fork && '-fork' || '' }}
         path: ~/.bun/install/cache
 
     - name: Mount node_modules
-      uses: ./.github/actions/cache-mount
+      uses: ./.github/actions/cache
       with:
         provider: ${{ inputs.provider }}
         key: ${{ github.repository }}-node-modules-${{ github.event_name }}${{ github.event.pull_request.head.repo.fork && '-fork' || '' }}-${{ hashFiles('bun.lock') }}
         path: ./node_modules
 
     - name: Mount Turbo cache
       if: inputs.turbo-cache-key != ''
-      uses: ./.github/actions/cache-mount
+      uses: ./.github/actions/cache
       with:
         provider: ${{ inputs.provider }}
         key: ${{ github.repository }}-${{ inputs.turbo-cache-key }}-${{ github.event_name }}${{ github.event.pull_request.head.repo.fork && '-fork' || '' }}

.github/dependabot.yml

@@ -0,0 +1,19 @@
+version: 2
+
+# GitHub Actions only: every `uses:` is pinned to a commit SHA, and without this the pins drift
+# apart (checkout had three different SHAs across workflows). One grouped PR a week keeps every
+# workflow and composite action on the same version of each action.
+updates:
+  - package-ecosystem: github-actions
+    directories:
+      - /
+      - /.github/actions/*
+    schedule:
+      interval: weekly
+    target-branch: staging
+    groups:
+      actions:
+        patterns:
+          - '*'
+    commit-message:
+      prefix: ci

.github/scripts/http-e2e.sh

@@ -0,0 +1,157 @@
+#!/usr/bin/env bash
+# Runs one end-to-end suite group over real HTTP, each against its own `next dev` app.
+#
+# Usage: http-e2e.sh <scim|cli|stop-after|desktop-inbox>   (run from apps/sim)
+#
+# The job provides DATABASE_URL, BETTER_AUTH_SECRET and ENCRYPTION_KEY; each group sets the rest of
+# its app's environment here. Reports and server logs land in $RUNNER_TEMP/e2e.
+#
+# The first request cold-compiles the app under Turbopack, which takes 42-150s on CI runners, so
+# the readiness deadline only has to catch a hung boot: an exited server fails immediately, and
+# either way the server log tail lands in the job log.
+#
+# Each app starts from an empty Turbopack dev cache: a cache written under other NEXT_PUBLIC_*
+# values, by a server that `next dev` SIGKILLs 100ms after SIGTERM, can panic Turbopack or wedge a
+# route compile on restore. It runs in its own session under an E2E_APP tag, and stop-session.sh
+# returns only once every process in that session or carrying that tag has exited (Next's
+# telemetry flush runs detached and still writes .next/dev).
+set -euo pipefail
+
+group=${1:?usage: http-e2e.sh <scim|cli|stop-after|desktop-inbox>}
+report_dir="$RUNNER_TEMP/e2e"
+ready_timeout_seconds=300
+mkdir -p "$report_dir"
+
+server_pid=''
+app_tag=''
+server_log=''
+status_log=''
+
+finish() {
+  local status=$?
+  if [ -n "$server_pid" ]; then
+    bash "$GITHUB_WORKSPACE/.github/scripts/stop-session.sh" "$server_pid" "$app_tag" || status=1
+    wait "$server_pid" 2>/dev/null || true
+    if [ -n "$status_log" ]; then
+      awk '/^ (GET|POST|PUT|PATCH|DELETE|HEAD) \/api\// { print }' "$server_log" > "$status_log"
+    fi
+    if [ "$status" -ne 0 ]; then
+      tail -n 200 "$server_log"
+    fi
+  fi
+  exit "$status"
+}
+trap finish EXIT
+
+# start_app <name> <port> <label> [record-http-status]
+start_app() {
+  local name=$1 port=$2 label=$3
+  server_log="$report_dir/$name-next.log"
+  if [ "${4:-}" = record-http-status ]; then
+    status_log="$report_dir/$name-http-status.log"
+  fi
+  app_tag="$name-$GITHUB_RUN_ID-$GITHUB_RUN_ATTEMPT-$$"
+  export NEXT_PUBLIC_APP_URL="http://127.0.0.1:$port"
+  export BETTER_AUTH_URL="$NEXT_PUBLIC_APP_URL"
+  export DISABLE_TELEMETRY=true NEXT_TELEMETRY_DISABLED=1
+  rm -rf .next/dev
+  E2E_APP="$app_tag" setsid node ../../node_modules/next/dist/bin/next dev --hostname 127.0.0.1 \
+    --port "$port" > "$server_log" 2>&1 &
+  server_pid=$!
+
+  local started=$SECONDS
+  until curl --fail --silent --max-time 10 "$NEXT_PUBLIC_APP_URL/api/health" > /dev/null; do
+    if ! kill -0 "$server_pid" 2>/dev/null; then
+      echo "::error::Local $label app exited during startup."
+      exit 1
+    fi
+    if [ $((SECONDS - started)) -ge "$ready_timeout_seconds" ]; then
+      echo "::error::Local $label app did not become ready within $ready_timeout_seconds seconds."
+      exit 1
+    fi
+    sleep 2
+  done
+  echo "Local $label app ready after $((SECONDS - started))s"
+}
+
+case "$group" in
+  scim)
+    export NEXT_PUBLIC_FORCE_HOSTED=true
+    export BILLING_ENABLED=true NEXT_PUBLIC_BILLING_ENABLED=true
+    export ENTERPRISE_ENABLED=true NEXT_PUBLIC_ENTERPRISE_ENABLED=true
+    export SCIM_ENABLED=true NEXT_PUBLIC_SCIM_ENABLED=true
+    export SSO_ENABLED=true NEXT_PUBLIC_SSO_ENABLED=true
+    export ORGANIZATIONS_ENABLED=true NEXT_PUBLIC_ORGANIZATIONS_ENABLED=true
+    export INTERNAL_API_SECRET=scim-http-ci-local-secret-at-least-32-characters
+    export DB_TX_TRIPWIRE=throw
+    export NEXT_PUBLIC_CHAT_DISABLED=true
+    start_app scim 3017 SCIM record-http-status
+    SCIM_E2E_BASE_URL="$NEXT_PUBLIC_APP_URL" \
+    SCIM_E2E_DATABASE_URL="$DATABASE_URL" \
+    SCIM_E2E_AUTH_SECRET="$BETTER_AUTH_SECRET" \
+    SCIM_E2E_REPORT_PATH="$report_dir/scim-e2e-report.json" \
+      bun run test:scim:e2e
+    VERSION_COMPARE_E2E_BASE_URL="$NEXT_PUBLIC_APP_URL" \
+    VERSION_COMPARE_E2E_DATABASE_URL="$DATABASE_URL" \
+    VERSION_COMPARE_E2E_AUTH_SECRET="$BETTER_AUTH_SECRET" \
+    VERSION_COMPARE_E2E_REPORT_PATH="$report_dir/version-compare-http-report.json" \
+      bun run test:workflow-version-compare:e2e
+    ;;
+
+  # The search suites serve their own fixtures in-process and need no app. They share this group
+  # because they finish in seconds. Self-hosted without billing: hosted billing admits runs through
+  # Redis, which the CLI app is not given.
+  cli)
+    for search in google-content lucid zoom google-meet; do
+      report_var="SEARCH_$(echo "$search" | tr 'a-z-' 'A-Z_')_REPORT_PATH"
+      env NEXT_PUBLIC_APP_URL=http://127.0.0.1:3040 NEXT_PUBLIC_FORCE_HOSTED=false \
+        "$report_var=$report_dir/search-$search.json" \
+        bun "scripts/test-search-$search-e2e.ts"
+    done
+    export NEXT_PUBLIC_FORCE_HOSTED=false
+    export INTERNAL_API_SECRET=cli-http-ci-local-secret-at-least-32-characters
+    start_app cli 3018 CLI
+    CLI_LATENCY_E2E_BASE_URL="$NEXT_PUBLIC_APP_URL" \
+    CLI_LATENCY_E2E_DATABASE_URL="$DATABASE_URL" \
+    CLI_LATENCY_E2E_RUNS=3 \
+    CLI_LATENCY_E2E_WARMUP=1 \
+    CLI_LATENCY_E2E_REPORT_PATH="$report_dir/cli-run-latency-report.json" \
+      bun run test:cli-run-latency:e2e
+    ;;
+
+  # Self-hosted: hosted billing admits a run only through a Redis usage reservation.
+  stop-after)
+    export NEXT_PUBLIC_FORCE_HOSTED=false
+    export INTERNAL_API_SECRET=stop-after-http-ci-local-secret-at-least-32-characters
+    export DB_TX_TRIPWIRE=throw
+    export NEXT_PUBLIC_CHAT_DISABLED=true
+    start_app stop-after 3018 workflow record-http-status
+    STOP_AFTER_E2E_BASE_URL="$NEXT_PUBLIC_APP_URL" \
+    STOP_AFTER_E2E_DATABASE_URL="$DATABASE_URL" \
+    STOP_AFTER_E2E_REPORT_PATH="$report_dir/stop-after-http-report.json" \
+      bun run test:workflow-stop-after:e2e
+    ;;
+
+  # The desktop background executor's protocol: device registration, the SSE doorbell over Redis
+  # pub/sub, presence, leased claims, Stop and isolation. The only group whose app gets Redis.
+  desktop-inbox)
+    export REDIS_URL=redis://127.0.0.1:6379
+    export NEXT_PUBLIC_FORCE_HOSTED=false
+    export MSHIP_DESKTOP_BACKGROUND_EXECUTOR=true
+    export COPILOT_TOOL_PERMISSIONS_ENABLED=true
+    export INTERNAL_API_SECRET=desktop-inbox-http-ci-local-secret-at-least-32-characters
+    export DB_TX_TRIPWIRE=throw
+    start_app desktop-inbox 3019 'desktop executor' record-http-status
+    DESKTOP_INBOX_E2E_BASE_URL="$NEXT_PUBLIC_APP_URL" \
+    DESKTOP_INBOX_E2E_DATABASE_URL="$DATABASE_URL" \
+    DESKTOP_INBOX_E2E_REDIS_URL="$REDIS_URL" \
+    DESKTOP_INBOX_E2E_AUTH_SECRET="$BETTER_AUTH_SECRET" \
+    DESKTOP_INBOX_E2E_REPORT_PATH="$report_dir/desktop-inbox-http-report.json" \
+      bun run test:desktop-inbox:e2e
+    ;;
+
+  *)
+    echo "::error::Unknown end-to-end group: $group" >&2
+    exit 2
+    ;;
+esac