ci: faster, cheaper, consistently named CI by waleedlatif1 · Pull Request #8750 · simstudioai/sim

waleedlatif1 · 2026-10-07T17:54:09Z

What

A cleanup of the GitHub Actions setup. It makes PR checks faster and cheaper, collapses duplicated setup, aligns versions, and gives every workflow, job and composite action a short name. What gets tested and how deploys are gated are unchanged.

Speed and cost (`checks.yml`)

Job	Before	After
PostgreSQL integration	2 × ~15 min on 8 vCPU	`integration (push\|migrate, n/4)`: 8 shards on 4 vCPU, 3–7 min each
End-to-end over real HTTP	4 apps in series, ~8 min	`e2e (scim\|cli\|stop-after\|desktop-inbox)` in parallel, ≤4.7 min
Lint and Test	one serial job, ~6.5 min	`lint` 1.1 min + `test (1/2)`, `test (2/2)`
—	—	`ci`: one aggregate status for every check

Both provisioning paths still run the whole integration suite. I built a database each way and compared them: migrations add triggers, CHECK constraints and NOT VALID FKs that db:push does not create.
The E2E boot/wait/stop shell, copied 4 times, now lives once in .github/scripts/http-e2e.sh.
desktop-live is now the longest PR job. About 10 min: a cold Next compile plus tests that deliberately wait out tool budgets. Untouched here.

Names

Workflow files:

Before After

test-build checks

migrations migrate

deploy-trigger-dev trigger-dev

docs-embeddings docs

companion-pr-check companion

stickydisk-gc disk-gc

ci.yml, helm.yml and codeql.yml keep their paths, because the path is part of the Sigstore signer identity and the code-scanning analysis key.
Composite actions: setup-workspace → setup, cache-mount → cache, docker-build → image.
Job and workflow display names: short lowercase IDs that match the job ID, e.g. ci / checks / integration (push, 1/4), ci / image-amd64 (app), ci / promote, ci / trigger-promote.

Duplicates and consistency

Shared setup: 9 hand-written Bun cache + install blocks now use ./.github/actions/setup. It gains two inputs: node-version (empty keeps the runner's Node, as those jobs had) and registry-url (for npm publish). This also ends restores of node_modules built from a different lockfile.
Runner expression: anchored once per file with YAML anchors.
One pinned SHA per action:

Action Was Now

checkout v4 (helm), v5 (codeql), v6 v6

setup-node v4 (desktop), v6 v6

cache v4 (desktop-release), v5 v5

softprops/action-gh-release v1 (Node 16) v3.0.3

Redis is 8.2 everywhere.
Secrets: secrets: inherit is replaced with only the secrets each callee reads. checks reads none. The dev migration receives only DEV_DATABASE_URL.
macOS desktop-e2e: timeouts added (there were none, so a hang billed up to 6 h), plus a cache for the Electron, electron-builder and Playwright downloads.
Publish workflows:
- Values in run: blocks moved to env:.
- Concurrency on the npm and PyPI publishers.
- persist-credentials: false on checkouts that never push.
Dependabot: added for github-actions only, as one grouped weekly PR to staging.
actionlint: pinned and checksum-verified, in the lint job.

Fix

PyPI version check. It matched substrings, so 0.1.1 counted as published once 0.1.10 existed, and it read a PyPI outage as "not published". It now looks up the exact version and fails on anything other than 200 or 404.

Not changed, on purpose

The npm publish workflows stay separate files. They only run on pushes to main/staging/dev, so a PR can't exercise them, and an accidental trigger on main publishes a new latest. They got the shared setup and the fixes above, but not a restructure.
Concurrency-group names and the companion sticky-comment marker. Renaming them could let an in-flight run overlap a new one, or orphan existing comments.
Vitest fsModuleCache. Measured on apps/sim shard 1/2: 140 s with no cache, 157 s cold, 118 s warm. The two test shards share one node_modules disk where the last writer wins, so one would usually run cold.

After merge (needs a repo admin)

Add ci / checks / ci as the one required status check in the staging and main rulesets. Today the rulesets require no status checks, so a red PR can merge.

Verify

actionlint 1.7.12 clean on all workflows, and shellcheck clean on the new script
Loaded the old and new ci.yml, checks.yml and helm.yml with a YAML parser and compared every job's runner, if, needs, permissions, timeout and outputs: only the intended renames differ
bun run lint:check, check:audits (58), test:scripts (354 tests, including the ones that run the real promote/prune/migrate shell from the workflows), check:skills
First commit's CI run: all 19 jobs green
This commit's CI run
First staging push after merge: watch migrate, trigger-upload, promote, trigger-promote, publish-sim-cli, publish-sim-setup

vercel · 2026-10-07T17:54:14Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs	Skipped		Oct 7, 2026 7:40pm UTC

cubic-dev-ai

All reported issues were addressed across 5 files

_{Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic}

greptile-apps · 2026-10-07T18:00:18Z

Confidence Score: 5/5

[High risk] Renames CI workflows and refactors build infrastructure.

The PR appears safe to merge; the latest documentation changes match the workflows.

Summary

This PR splits CI work into parallel jobs, shares setup and HTTP test scripts, and shortens workflow names.

Keeps both database setup paths and adds one combined ci check.
Narrows passed secrets, adds publisher concurrency, and fixes the PyPI version lookup.
Since the previous review, only desktop release documentation changed. Its names and release order match the workflows.
No new actionable issues or repository-rule violations were found. No previous findings were supplied.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Integration["integration: two paths, four shards each"] --> Gate["ci"]
  E2E["e2e: four parallel groups"] --> Gate
  DesktopChanges["desktop-changes"] --> DesktopLive["desktop-live when needed"]
  DesktopChanges --> Gate
  DesktopLive --> Gate
  Lint["lint"] --> Gate
  Test["test: two shards"] --> Gate
  Build["build"] --> Gate

_{Reviews (3) · Last reviewed commit: "docs(desktop): point the release notes a..." · Reviewed by Greptile}

… a ci gate The PR critical path was the PostgreSQL integration suite (~15 min), run in full on an 8 vCPU runner once per provisioning path. Its files run one at a time, so the runner sat mostly idle. - integration: each provisioning path (push, migrate) is split into 4 Vitest shards on 4 vCPU runners. Both paths keep the full suite: migrations add triggers, checks and NOT VALID constraints that db:push does not, so the schemas differ. - e2e: the four next-dev groups (scim, cli, stop-after, desktop-inbox) run as a matrix, each on its own database. The boot/wait/stop shell lives once in http-e2e.sh. - lint: lint, audits, type-check and schema sync split off from the tests. - test: apps/sim unit tests sharded 2 ways; shard 1 also runs root scripts and the other workspaces. Each shard has its own Turbo cache disk. - ci: one aggregate job that fails unless every check passed (skipped is allowed), so a ruleset can require a single stable check. Deploy gating is unchanged: migrate still requires the whole Test and Build workflow.

Naming - Workflow files: test-build -> checks, migrations -> migrate, deploy-trigger-dev -> trigger-dev, docs-embeddings -> docs, companion-pr-check -> companion, stickydisk-gc -> disk-gc. ci.yml, helm.yml and codeql.yml keep their paths: they sign or analyze, and the path is part of the signer identity and code-scanning key. - Composite actions: setup-workspace -> setup, cache-mount -> cache, docker-build -> image. - Every workflow and job display name is a short lowercase id, matching the job id, with any matrix value in parentheses: ci / checks / integration (push, 1/4), image-amd64 (app). Duplicates - Nine hand-written Setup Bun + actions/cache + bun install blocks use the setup action. It gains node-version (empty keeps the runner's Node, as those jobs had) and registry-url (npm publishing). This also ends restoring node_modules from another lockfile through the `${runner.os}-bun-` prefix restore key. - The runner expression is anchored once per file and aliased after. Consistency - One SHA per action: checkout v6 (helm was on v4, codeql on v5), setup-node v6 (desktop on v4), cache v5 (desktop-release on v4), softprops/action-gh-release v3.0.3 (was v1, Node 16). Redis 8.2 everywhere. - secrets: inherit replaced by the secrets each callee reads; the checks workflow reads none. The dev migration gets only DEV_DATABASE_URL, and staging/production never see it. - Timeouts on the three macOS desktop-e2e jobs (none before, so a hang billed 6 hours), and a cache for their Electron, electron-builder and Playwright downloads. - Publish workflows: values moved from ${{ }} in run blocks to env, concurrency on the npm/PyPI publishers, persist-credentials: false on checkouts that never push. - Dependabot for github-actions (one grouped weekly PR), and actionlint in the lint job. Fixes - PyPI version check matched substrings (0.1.1 "existed" once 0.1.10 did) and treated a PyPI outage as "not published". It now asks PyPI for the exact version and fails on anything but 200 or 404. Job wiring (runner, if, needs, permissions, timeout, outputs) is unchanged: verified by loading the old and new ci.yml, checks.yml and helm.yml and comparing every job.

waleedlatif1 · 2026-10-07T19:27:50Z

@greptile

waleedlatif1 · 2026-10-07T19:27:51Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-07T19:27:58Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

1 issue found across 29 files

Confidence score: 5/5

In .github/workflows/desktop-e2e.yml, the canary job shares its cache key with e2e, so its cache save is skipped. Give the canary a distinct cache key if you expect it to save its Electron download.

Prompt for AI agents (unresolved issues)


Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/workflows/desktop-e2e.yml">

<violation number="1" location=".github/workflows/desktop-e2e.yml:85">
P3: The canary job shares its cache key with the `e2e` job, so its cache save is silently skipped: `bun add --no-save electron@latest` leaves bun.lock unchanged and `e2e` already created `desktop-downloads-${{ runner.os }}-<lock hash>` on the prior PR run. Every weekly canary run re-downloads electron@latest (~120 MB), so the new download caching never helps this job. Give the canary its own key segment (e.g. `...-${{ runner.os }}-canary-${{ hashFiles('bun.lock') }}` while keeping the `desktop-downloads-${{ runner.os }}-` restore-keys prefix) so each job saves and restores its own entries.</violation>
</file>

_{Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic}

waleedlatif1 · 2026-10-07T19:40:56Z

@greptile

waleedlatif1 · 2026-10-07T19:40:57Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-07T19:41:03Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 30 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Turn on auto-fix | Re-trigger cubic}

vercel Bot deployed to Preview October 7, 2026 17:56 View deployment

cubic-dev-ai Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread .github/workflows/checks.yml Outdated

waleedlatif1 changed the title ~~ci: shard integration and unit tests, run e2e groups in parallel, add a ci gate~~ ci: faster, cheaper, consistently named CI Oct 7, 2026

vercel Bot deployed to Preview October 7, 2026 18:26 View deployment

waleedlatif1 added 3 commits October 7, 2026 12:27

ci: run the ci gate on cancelled runs so a cancelled head never passes

569288f

waleedlatif1 force-pushed the chore/ci-cleanup branch from 8ba3d02 to 569288f Compare October 7, 2026 19:27

vercel Bot deployed to Preview October 7, 2026 19:30 View deployment

cubic-dev-ai Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread .github/workflows/desktop-e2e.yml

Comment thread .github/workflows/desktop-release.yml

docs(desktop): point the release notes at the renamed workflow and jobs

26c3b19

vercel Bot temporarily deployed to Preview October 7, 2026 19:40 Inactive

cubic-dev-ai Bot reviewed Oct 7, 2026

View reviewed changes

waleedlatif1 merged commit 9ab24e9 into staging Oct 7, 2026
55 checks passed

waleedlatif1 deleted the chore/ci-cleanup branch October 7, 2026 20:38

This was referenced Oct 7, 2026

v0.9.15: mship improvements, memory improvements, nextjs bump, plane integration #8748

Open

ci: duration-balanced integration shards, warm desktop-live, CI on every PR #8763

Open

Before	After
`test-build`	`checks`
`migrations`	`migrate`
`deploy-trigger-dev`	`trigger-dev`
`docs-embeddings`	`docs`
`companion-pr-check`	`companion`
`stickydisk-gc`	`disk-gc`

Action	Was	Now
`checkout`	v4 (helm), v5 (codeql), v6	v6
`setup-node`	v4 (desktop), v6	v6
`cache`	v4 (desktop-release), v5	v5
`softprops/action-gh-release`	v1 (Node 16)	v3.0.3

Conversation

waleedlatif1 commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Speed and cost (checks.yml)

Names

Duplicates and consistency

Fix

Not changed, on purpose

After merge (needs a repo admin)

Verify

Uh oh!

vercel Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

greptile-apps Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

waleedlatif1 commented Oct 7, 2026 •

edited

Loading

Speed and cost (`checks.yml`)

vercel Bot commented Oct 7, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

greptile-apps Bot commented Oct 7, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading