perf: union rdeps queries across changed modules into one subprocess#337
Open
rdark wants to merge 1 commit intoTinder:masterfrom
Open
perf: union rdeps queries across changed modules into one subprocess#337rdark wants to merge 1 commit intoTinder:masterfrom
rdark wants to merge 1 commit intoTinder:masterfrom
Conversation
CalculateImpactedTargetsInteractor.queryTargetsDependingOnModules previously spawned one `bazel query rdeps(//..., @@<repo>//...)` subprocess per changed-module-matching canonical repo. A single changed module can substring-match thousands of repos in a bzlmod workspace, and each subprocess pays ~2s of JVM + bazel-client-connect overhead serially. Union all matched repos into a single `rdeps(//..., @@a//... + @@b//... + ...)` query. Bazel computes the reverse-dep graph of //... once regardless of how many patterns are in the union, so runtime collapses from N × (startup + analysis) to 1 × (startup + analysis); the N-1 eliminated subprocesses are the bulk of the saving. - No command-line length concern: BazelQueryService.runQuery writes queries via --query_file, so the query string is arbitrary size. - Failure semantics: a single try/catch wraps the unioned query; on failure, fall back to marking all workspace targets impacted. The previous outer catch-all is removed - audit confirmed every throwable call now has a tight try/catch around it, and the broad catch was silently swallowing errors. - Per-module log line preserves "module X → matched N repos: ..." attribution so as to not break logging contract - Tests: adds testUnionsRdepsAcrossChangedModules with two changed modules to assert the single-query invariant.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Partial fix for #335
CalculateImpactedTargetsInteractor.queryTargetsDependingOnModules previously spawned one
bazel query rdeps(//..., @<repo>//...)subprocess per changed-module-matching canonical repo. A single changed module can substring-match thousands of repos in a bzlmod workspace, and each subprocess pays ~2s of JVM + bazel-client-connect overhead serially.Union all matched repos into a single
rdeps(//..., @@a//... + @@b//... + ...)query. Bazel computes the reverse-dep graph of //... once regardless of how many patterns are in the union, so runtime collapses from N × (startup + analysis) to 1 × (startup + analysis); the N-1 eliminated subprocesses are the bulk of the saving.