ci: duration-balanced integration shards, warm desktop-live, CI on every PR by waleedlatif1 · Pull Request #8763 · simstudioai/sim

waleedlatif1 · 2026-10-07T21:10:18Z

What

Follow-up to #8750. Three changes, none of which change what a check verifies.

1. Integration shards balanced by duration (`vitest.shared.ts`)

Before: Vitest's --shard sorts files by path hash and gives each shard an equal file count. recovery-storm.integration.ts alone takes ~217 s, about 21% of the suite, and the hash put it beside 41 other files. Shards ranged from 183 to 384 s.
Change: DurationBalancedSequencer overrides only shard(). It packs files longest-first into the lightest shard, using per-file seconds from the committed vitest.integration-durations.json. Ordering within a shard is still Vitest's own sort().
Same split on every shard: each shard computes the full split from the same file list and weights, breaking ties by path and then by lowest shard index. Files stay disjoint and all are covered, which I checked against the real file lists (166 + 17): no duplicates, none missing, and the same result with the input order reversed.
New or stale files: a new file gets an average weight, and a stale weight only costs balance.
Result (measured weights): the apps/sim shards come out at 254 s each, so the slowest integration job drops from ~7.4 to ~5.3 min at the same runner cost.
Refreshing weights: gh run download <id> -p 'integration-reports-*' -D <dir>, then bun run scripts/update-integration-durations.ts <dir>.

2. desktop-live: keep the Turbopack dev cache (`checks.yml`)

The job's ~4 min route warm-up is a cold Turbopack compile, and next.config.ts already enables the dev filesystem cache (measured there at 5.4× faster restarts).

Cache disk: apps/sim/.next/dev now lives on a sticky disk. It is keyed by event and fork (like the other caches) and by the installed Next version, so an upgrade starts from empty.
Clean stop: the app is stopped with SIGINT before stop-session.sh runs. next dev SIGKILLs its server 100 ms after SIGTERM, which throws away a cache write in progress.
Corrupt cache: if Turbopack reports a corrupt cache during startup, the job clears the cache and restarts once from empty. It also clears the cache after the run if the log shows corruption or a panic. A plain test failure keeps the shared cache.
Cache size: dev:cache:cap keeps the cache under the 20 GB cap the dev scripts already use.
Parallel boot: the apps now boot while the apt install and the Electron bundle run (~35 s).
Expected: ~2.5–3.5 min off the job once the cache is warm. The first run on this PR is cold by design.

3. CI runs on every pull request (`ci.yml`)

The paths-ignore for docs and markdown is gone. checks / ci is about to become the required status check, and a PR whose run never starts never reports it, so it could not merge. None of the last 200 PRs merged to staging was docs-only, so the cost is nil.

Verify

actionlint and shellcheck (the desktop-live script) clean; lint:check, check:audits (58), test:scripts (357), type-check
Sequencer split checked on the real file lists; two real packages/db integration shards run against Postgres with it
This PR's CI: every shard green, plus integration shard times
Re-run desktop-live on the same commit to measure the warm cache

vercel · 2026-10-07T21:10:20Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Oct 7, 2026 9:52pm UTC

cubic-dev-ai

All reported issues were addressed across 5 files

_{Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic}

greptile-apps · 2026-10-07T21:17:02Z

Confidence Score: 5/5

[High risk] Rewires CI workflow triggers and test sharding logic.

The PR appears safe to merge; no blocking issue remains.

What we checked:

Integration files stay covered: Each file is added to exactly one shard. Sorting by duration and path makes the split repeatable across the separate CI jobs.
Cache pruning keeps the mount: The size cap targets the nested cache/turbopack directory. It runs before the server starts and leaves the mount intact.

Summary

This PR balances integration shards using saved file durations, keeps the desktop test’s Turbopack cache between runs, and starts CI for docs-only PRs.

The shard split assigns each file once and keeps Vitest’s existing sorting.
Desktop cleanup stops app processes before clearing a broken cache.
All three earlier, unnumbered findings are fixed: cache cleanup uses an absolute path, worker cleanup always runs, and the refresh script uses createLogger.
No new actionable issues were found.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Files[Integration files] --> Weights[Read saved durations]
  Weights --> Sort[Sort longest first]
  Sort --> Assign[Assign to lightest shard]
  Assign --> Run[Vitest sorts and runs each shard]

_{Reviews (3) · Last reviewed commit: "ci: a malformed durations file names its..." · Reviewed by Greptile}

waleedlatif1 · 2026-10-07T21:39:07Z

@greptile

waleedlatif1 · 2026-10-07T21:39:08Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-07T21:39:14Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 5 files

_{Reply with feedback, questions, or to request a fix.

Turn on auto-fix | Re-trigger cubic}

…ery PR - integration: a custom Vitest sequencer packs `--shard` files by measured duration from a committed `vitest.integration-durations.json` (refreshed with scripts/update-integration-durations.ts) instead of by path hash. On the measured run the apps/sim shards go from 183-384s to 254s each, so the slowest job drops from ~7.4 to ~5.3 min at the same cost. Every shard computes the same partition, so files stay disjoint and covered. - desktop-live: Turbopack's dev cache is kept on a sticky disk keyed by event, fork and the installed Next version, so the spec's warm-up restores instead of cold-compiling. The app is stopped with SIGINT so the cache write completes, a cache Turbopack reports as corrupt is dropped and the app restarted from empty, and the apps boot while apt and the Electron bundle run. - ci.yml: no paths-ignore on pull_request. `checks / ci` is going to be the required check, and a pull request whose run never starts never reports it.

… floor shard weights, log through @sim/logger

…count as unknown

waleedlatif1 · 2026-10-07T21:49:40Z

@greptile

waleedlatif1 · 2026-10-07T21:49:41Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-07T21:49:48Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 5 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Turn on auto-fix | Re-trigger cubic}

vercel Bot deployed to Preview October 7, 2026 21:13 View deployment

cubic-dev-ai Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread vitest.integration-durations.json

Comment thread .github/workflows/checks.yml Outdated

Comment thread .github/workflows/checks.yml

greptile-apps Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread .github/workflows/checks.yml Outdated

Comment thread .github/workflows/checks.yml Outdated

Comment thread scripts/update-integration-durations.ts Outdated

vercel Bot deployed to Preview October 7, 2026 21:41 View deployment

cubic-dev-ai Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread vitest.shared.ts Outdated

waleedlatif1 added 3 commits October 7, 2026 14:49

ci: clear the dev cache by absolute path, always run session cleanup,…

d521bda

… floor shard weights, log through @sim/logger

ci: a malformed durations file names its fix, and non-finite weights …

74af417

…count as unknown

waleedlatif1 force-pushed the chore/ci-speed branch from a237a66 to 74af417 Compare October 7, 2026 21:49

vercel Bot deployed to Preview October 7, 2026 21:52 View deployment

cubic-dev-ai Bot reviewed Oct 7, 2026

View reviewed changes

waleedlatif1 merged commit d8cb659 into staging Oct 7, 2026
37 checks passed

waleedlatif1 deleted the chore/ci-speed branch October 7, 2026 21:56

This was referenced Oct 7, 2026

v0.9.15: mship improvements, memory improvements, nextjs bump, plane and ramp integrations, outbox hardening #8748

Merged

improvement(ci): check out same-repo pull requests through a git mirror #8768

Merged

Conversation

waleedlatif1 commented Oct 7, 2026

What

1. Integration shards balanced by duration (vitest.shared.ts)

2. desktop-live: keep the Turbopack dev cache (checks.yml)

3. CI runs on every pull request (ci.yml)

Verify

Uh oh!

vercel Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

waleedlatif1 commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 7, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

1. Integration shards balanced by duration (`vitest.shared.ts`)

2. desktop-live: keep the Turbopack dev cache (`checks.yml`)

3. CI runs on every pull request (`ci.yml`)

vercel Bot commented Oct 7, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

greptile-apps Bot commented Oct 7, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading