Repository navigation
Reduce GitHub Actions jobs and runner queue time across the organization #128
Description
Activity
Bootstrap verification: merged #127 triggered https://github.com/Cratis/Workflows/actions/runs/36861584784 at 2026-10-01 12:24:53 UTC; it succeeded. Spot checks confirm release-intent and release-notes callers on Arc, Chronicle and Fundamentals, with recent passing runs for both gates. Public repository rulesets (including inherited rulesets) and main branch protections do not require verify / verify. Direct organization ruleset listing is unavailable to the current token (admin:org missing), so that limitation remains explicit. Package cleanup remains blocked by the fine-grained PAT_WORKFLOWS credential; the existing focused issue #103 now has the exact failure and admin action, avoiding a duplicate issue.
Remaining implementations are prepared as local commits: Q3 public caller paths/concurrency (the shared five-minute timeout already exists), current SHA pins, ignored-repository release callers, SDK-pin installation, and Q11 job-level usage reporting with explicit missing-private-coverage warnings. No pushes or PRs yet: the complete package-updater offline suite exceeded its first 120-second execution limit, and subsequent bounded phase admission could not obtain a build/test slot; other repositories have not passed their full local gates. YAML validation, reporting fixtures, release gate/script tests, cleanup tests and the Dapr workflow-boundary check passed. This is incomplete, not ready-to-merge work. No D1–D10 change or schedule restoration was made. Cleanup permission remediation is tracked in #103.
Decisions on the remaining CI-cost items (made 2026-10-01, delegated by the owner):
- D1: add a second, small
cratis-arc-smallrunner scale set for gate and guard jobs (release notes, release intent, work-record checks, PR gate checks) instead of doublingcratis-arc. About two thirds of jobs do under a minute of work, so they stop competing with builds for the 3 large slots. - D2: delete Studio's PR Gate. It cannot block merges on the Free plan, and Publish's
verify-pr-checksalready enforces the same rule at merge time. - D3: merge Studio Publish's small serial jobs into one job. Keep the
auditrun on the merged tree. - D4: run Studio Integration Specs on a
run-integrationlabel, on changes underSource/Integrations/**, and nightly, following Chronicle's pattern. - D5: keep Direct's Debug and Release builds. The saving is small and both configurations catch different problems.
- D6: Chronicle Health runs hourly instead of every 15 minutes.
- D7: drop the push-to-main build where pull request runs cover it (Strategy Validate, Chronicle.Wolverine CI, Chronicle.Dapr Build, Ensemble Build); Ensemble's daily build only runs when there are new commits.
- D8: Update Packages runs weekly on a per-repository staggered minute.
- D9: done for Arc (#2943, docs-only CodeQL skip).
- D10: small private checks move to
cratis-arc-small(D1) rather than to hosted runners.
- D1: add a second, small
The remaining public-repository CI follow-ups are pushed with one
no-releasePR per branch, all assigned to woksin:- Workflows work-record/report follow-ups: Improve work-record checks and Actions usage reporting #129
- Workflows caller SDK pin: Honor caller SDK pins in package updates #130
- Arc.Kotlin: Update pinned release checks and normalize Dependabot labels Arc.Kotlin#272
- Ante: Refresh shared release checks and Dependabot labels Ante#158
- Components: Refresh shared release checks and Dependabot labels Components#391
- Orleans: Refresh shared release checks and Dependabot labels Orleans#48
- Stage: Refresh shared release checks and Dependabot labels Stage#192
- Templates: Refresh shared release checks and Dependabot labels Templates#70
- Chronicle.Elixir: Refresh shared release checks and Dependabot labels Chronicle.Elixir#97
- Chronicle.Python: Refresh shared release checks and Dependabot labels Chronicle.Python#35
- Arc.TypeScript: Refresh shared release checks and Dependabot labels Arc.TypeScript#149
The complete updater suite passed once via pi-phase (
--timeout 300 --queue-timeout 600): 64 tests in 131.204 seconds, exit 0; no retries or direct fallback were needed. SDK-pin passing tests and other unchanged passing preparation checks were reused. YAML-only caller branches use the requested actionlint/diff-check gate. Main was fetched for every branch; Ante merged origin/main and its resulting YAML-only diff passed both checks again.Verification details and limitations are in PR comments, not release-note bodies. The first post-push CI snapshot has no failing checks, but several builds remain pending; this is not an all-green or merge-ready claim. No PR was merged, and no schedules, secrets, publication settings, or D1–D10 work were changed.
Completed: all 13 listed PRs and 10 of 11 follow-ups are merged. Decisions D2-D9 delivered: Studio pr-gate removed; Studio#1495 publish job merge plus integration-spec timing; chronicle-health runs hourly at 17 * * * *; push-to-main builds removed in Strategy, Wolverine, Dapr and Ensemble; #131 covers D8; #2943 covers D9. Remaining items live in their own repos: D1/D10 runner scale set in Cratis/Infrastructure#100 and Cratis/Infrastructure#102 (Infrastructure#117 raised arcMaxRunners 3 to 6, partly covering D1), and Cratis/Chronicle.Python#35.
Waiting for runners, not money, is the real cost of GitHub Actions across the organization. Private repositories bill about 42 hosted minutes a week, which the Free plan's included minutes cover, so the bill is effectively $0. What hurts is queue time, and this issue tracks the set of pull requests that cut the number of jobs and runaway runs behind it.
Measured drivers
Measured over 2026-09-24 to 2026-10-01.
cratis-arcsaturation. The scale set allows 3 runners at a time (arcMaxRunners: "3",arcMinRunners: "0"), and every job starts a fresh pod that takes about 1.5-2 minutes to begin. Across the organization's private repositories that is about 4,470 jobs a week, and about 66% of them do less than a minute of work. Demand (about 10,670 job-minutes plus about 5,800 minutes of pod start-up) is roughly 16,500 slot-minutes a week against about 30,000 in theory, mostly in working hours, so the pool is saturated during the day.cratis-arc. It still took a pod, about 660 times a week, and added 1.5-2 minutes serially to every run that used it.*.mdfiles and.ai-work/, yet it ran on every pull request and every push to main: about 570 pods a week oncratis-arc. In public repositories the same guard takes about 10 minutes of wall-clock time for a roughly 10-second job, which points to queueing for hosted runners, and the Free plan caps the organization at 20 hosted jobs running at once.Pull requests
Together they remove the runner-selection job, run the work-record guard only for markdown and
.ai-work/changes, skip CodeQL for documentation-only pull requests, drop duplicate push-to-main builds, add job timeouts, collapse bursts of dependency updates, and pause or stagger failing scheduled updates.Follow-up decisions
Raising
arcMaxRunners, or adding a second small scale set for gate and guard jobs, is the largest remaining lever for queue time and is not part of these pull requests.