Repository navigation
ci: analyse every CodeQL language on every pull request, group and push - #353
Merged
Merged
Conversation
The code scanning rule holds a commit until it has a result in every category main has, so #347, which changed no Rust and got no Rust analysis, couldn't join the merge queue. A policy now fails when codeql.yml's matrix isn't a fixed list of languages.
botandbang
enabled auto-merge
November 1, 2026 16:16
Coverage Report for apps/web
File CoverageNo changed files found.SizeCompared with main. KiB are 1,024 bytes.
|
bnb
approved these changes
Nov 1, 2026
Coverage Report for apps/localtoolbox
File CoverageNo changed files found.SizeCompared with main. KiB are 1,024 bytes.
|
Rust coverageFrom run 19. Both are held to the thresholds in .cargo/config.toml.
|
This was referenced Nov 1, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
#352 ran CodeQL's Rust and TypeScript analyses only on commits that changed that code. The "Protect Main" ruleset's code scanning rule holds a pull request or merge queue group until its commit has a result in every category main has. So #347, a web-only pull request, got no Rust analysis, and adding it to the queue failed with "Code scanning is still expecting 1 result from CodeQL for 3de1fc5 or 1df9ec4". The same blocks every pull request that leaves out Rust or TypeScript (#329, #338, #342, #345, #347, bnb/lighthouse-history).
ci.yml's CodeQL call runs every language (actions, javascript-typescript, rust) on every pull request into main, merge queue group and push to main, whatever changed. Drafts and pull requests into other branches still skip it. Main's pushes analyse every language too, since pull requests' alerts are compared with main's analysis.codeql_rustandcodeql_javascriptfilters, thecodeql_languagesoutput, andcodeql.yml'slanguagesinput. Its matrix is a fixed list again.tools/policy/ruleset.rego. When the ruleset has a code scanning rule, every job incodeql.ymlhas to list its languages as a fixed matrix, not an expression, with tests..github/CONTRIBUTING.mdand.github/NEW_PROJECT.md.Rust's analysis is build-mode none, about 15 to 19 minutes on Linux, so this gives back part of #352's CodeQL saving on pull requests that don't touch Rust. A cheaper option for later: upload main's last Rust SARIF for the pull request's commit when no Rust changed. It would be accurate, since the Rust is identical, but it needs a real run to check that the rule accepts it.
After this merges, each blocked pull request needs a run that includes it: merge main into it, or close and reopen it once its merge ref contains this commit.
How it was tested
tools/policy.sh: 84/84 Rego tests at full coverage, 51/51 conftest checks. The new rule's test fails when the rule is broken on purpose.zizmor --offlineare clean.codeql=truefor pull requests into main (Rust-only, Markdown-only, theall-oslabel, a 3000-file one), merge groups and pushes, andfalsefor a draft and for a pull request into another branch.Checklist
docs/and--helpstill say what the code does