ci: rebase merge-gate required jobs on measured failure data - #4178
Draft
Leiyks wants to merge 1 commit into
Draft
ci: rebase merge-gate required jobs on measured failure data#4178Leiyks wants to merge 1 commit into
Leiyks wants to merge 1 commit into
Conversation
Re-require six compile globs, verify windows, and System Tests:* (a CI outage, not flakiness). Newly exempt 13 families with measured failures. Prune 88 globs with no evidence behind them (zero failures, single-failure, or unreachable jobs). Net: 149 -> 74 patterns; gating jobs 636 -> 1225 of 2098.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates
.gitlab/flaky-jobs.txtfrom a failure-frequency export covering 5006 job failures. The list had been hand-maintained at family granularity since #4028 and was never derived from data.Re-required — these failures were a CI outage, not flakiness: all six
compile *globs,verify windows, andSystem Tests:*.Newly exempted — 13 families with at least 2 recorded failures:
check libxml2 version,helper-rust integration coverage,appsec code coverage,ASAN Internal api randomized tests:*,Loader test on arm64 alpine,Loader test on arm64 libc:*,linux-php-wordpress-parallel, the threephp-wordpress-parallel-*jobs, andtest_integrations_{amqp_latest,memcache,ratchet}:*. Flakiness is not version-bound, so each is exempted at family granularity rather than per matrix cell.Pruned — 88 globs with no evidence behind them: 46 that matched real jobs but recorded zero failures, 8 whose family failed exactly once, and 34 that matched no job in the pipeline at all. That last group is structurally unreachable: the SSI scenario jobs live in a grandchild pipeline in
DataDog/apm-reliability/dd-trace-php, whichmerge-gate.shnever walks (it goes one level deep and hardcodes$CI_PROJECT_ID), andupdate-latest-versionsis a GitHub Actions job.Six zero-failure globs were kept because they never ran rather than never failed:
generate-lib-init-pinned-tag-values(manual, tag-only),package-oci:*andcreate-multiarch-lib-injection-image($SKIP_SHARED_PIPELINE-gated), and threephp-*-realworld-parallel-*benchmark jobs gated on master/nightly.Net: 149 → 74 patterns. Jobs that can block a merge: 636 → 1225 of 2098. Replaying the full observation window, the prune would have caused 8 additional gate blocks — all from the count-1 group; the zero-failure group contributes none.
Verification: every surviving and deleted pattern was matched with
merge-gate.sh's exact semantics ([[ "$job" == $g ]], unquoted glob, whole-string, case-sensitive) against the 2099 real job names from master pipeline 136253425.