feat(ci): record per-run LEM actuals via workflow_run#3766
feat(ci): record per-run LEM actuals via workflow_run#3766EffortlessSteven wants to merge 1 commit intomainfrom
Conversation
Adds .github/workflows/ci-actuals.yml. Triggers on workflow_run completed
for each canonical PR-time workflow (CI Core, Feature Matrix, GPU CI,
Compatibility, Property, Test Telemetry, ripr, PR Plan), pulls the job
list from the Actions API, and emits a `ci-actuals.json` artifact:
{
"schema_version": 1,
"workflow": "CI (Core)",
"workflow_run_id": 25438135897,
"conclusion": "success",
"event": "pull_request",
"head_branch": "...",
"head_sha": "...",
"totals": { "wall_min": 12.4, "lem": 12.4, "jobs": 5 },
"jobs": [
{
"name": "Build & Test (ubuntu-latest)",
"status": "completed",
"conclusion": "success",
"started_at": "...",
"completed_at": "...",
"wall_min": 6.55,
"runner_label": "ubuntu-22.04",
"runner_mult": 1.0,
"lem": 6.55
},
...
]
}
Runner multipliers mirror policy/ci-budget.toml (Linux=1, macOS=10,
Windows=2). Skipped / cancelled jobs are excluded from the total but
listed in the job array for diagnostics.
The workflow also writes a step summary with totals and the 15 top jobs
by LEM, so engineers see actuals as soon as a run completes.
Stage: visibility only. We are NOT enforcing budget yet — we need a few
weeks of actuals before promoting to soft / hard enforcement (PR O).
Out of scope:
- Aggregating actuals across runs into a durable dashboard.
- Comparing estimated LEM (PR Plan) to observed LEM in-line on the PR.
- Hard budget enforcement (PR O).
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
2 participants
Summary
Adds
.github/workflows/ci-actuals.yml. Triggers onworkflow_runcompletion for each canonical PR-time workflow, pulls the job list from the Actions API, and emits aci-actuals.jsonartifact with per-job wall time + LEM and a per-run total.This is PR N in the multi-PR CI roadmap.
Schema (v1)
{ "schema_version": 1, "workflow": "CI (Core)", "workflow_run_id": 25438135897, "conclusion": "success", "event": "pull_request", "head_branch": "...", "head_sha": "...", "totals": { "wall_min": 12.4, "lem": 12.4, "jobs": 5 }, "jobs": [ { "name": "Build & Test (ubuntu-latest)", "wall_min": 6.55, "runner_mult": 1.0, "lem": 6.55, ... }, ... ] }Runner multipliers mirror
policy/ci-budget.toml(Linux=1, macOS=10, Windows=2). Skipped/cancelled jobs are excluded from the total but listed injobsfor diagnostics.Workflows watched
CI (Core),Feature Matrix,GPU CI Matrix,Compatibility Tests,Property-Based TestsTest Telemetry(PR M),ripr (static exposure)(PR J),PR Plan(PR H + xtask in PR K)Stage: visibility only
We do not enforce budget here. The existing CI baseline does not yet have enough actuals to enforce learned thresholds safely. The path is:
policy/ci-budget.tomlthresholdsfull-ci/ci-budget-overridelabel is presentTest plan
ci-actuals-<run_id>artifact appears.Out of scope
https://claude.ai/code/session_01S2yTnEYJcA3G2CyZn9bY4v
Generated by Claude Code