Repository navigation
ROCMAI-82: extract driver-install, engines, serve from main.rs - #540
jussielo-amd wants to merge 6 commits into
Conversation
CodeQL check failure — not a new issueThe failing These aren't new: the identical pattern is already open and un-dismissed on No code change made here — this refactor doesn't introduce a new sensitive-data exposure. Flagging for whoever owns the code-scanning policy to decide whether to dismiss these 6 as duplicates of the existing open alerts, or leave them open like the rest. |
|
🔴 Automated review · pr-review-watcher · 2d5a4ee This automation never files a GitHub approval, so no approving review will SummaryThis PR moves the Reviewed: the whole change, which is Verified:
CI: Blocking: 1 · Non-blocking: 9. 🚫 Blocking (must fix before merge)
Non-blocking
|
rominf
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · 2d5a4ee
The branch does not build once it is merged with current main. The full report is in this round's PR comment.
Problem. #407 (5f72ac6, already on main) added calls in main.rs that this PR leaves unresolved. git merge-tree reports a clean merge, but in the merged main.rs:
select_serve_engineis called at 3049 (production) and at 32491, 32508 and 32594 (tests). This PR makes it private inserve_cmd, along withServeEngineSelection.detect_host_gpu_summaryis called unqualified at 2899 and 32593. This PR removes it from therocm_coreimport.
The result is E0425 errors: the merge queue will eject the PR, and a merge that bypasses the queue would break main.
To resolve.
- Rebase onto current
main. - Expose
select_serve_engineandServeEngineSelection(including.engine) aspub(crate)and re-import them inmain.rs, or move the #407 readiness code beside them. - Restore the
detect_host_gpu_summaryimport. - Confirm that clippy and the tests pass on the rebased tree.
This change request will be withdrawn once the rebased head builds.
2d5a4ee to
4956655
Compare
|
Rebased onto current Blocking — compile break fixed. Non-blocking:
Not done: actually relocating the managed-service-spawning tail and |
4956655 to
3b4d8b7
Compare
|
On whether to relocate the managed-service spawning tail and If you defer them, it would help to make sure the module docs say why these stay in The rest of what you list as addressed, including the blocking compile break after the rebase, will be checked by the full review of the new head. Nothing here confirms it yet. |
|
🔴 Automated review · pr-review-watcher · 3b4d8b7 This automation never files a GitHub approval, so no approving review will
Review — needs work (at 3b4d8b7)Full review of the whole change: Blocking
Settled standing blocker (from the round at 2d5a4ee): the branch no longer fails to build once merged with Non-blocking
Decisions for the author
Positive signals
Deployment notesNone What this covered
|
rominf
left a comment
There was a problem hiding this comment.
🔴 Automated review · pr-review-watcher · ffc8177
This automation never files a GitHub approval, so no approving review will
appear here whatever the outcome — the merge decision stays with a human
reviewer.
Round at
ffc81772. The change request from the round at2d5a4ee7("the branch does not build once merged withmain") is being withdrawn after this review. The details are under Settled standing blocker below. The blocking item below is published as a comment and not as a change request. CodeQL is not a required check formain: branch protection lists 17 required contexts, CodeQL is not one of them, and there is no ruleset. The 5 alerts it reports are the samerust/cleartext-loggingalerts that are already open onmain(#790–#795, at oldmain.rs:6805/7481/7762/31576-31578), now at new locations because the code moved. So the only thing merging would leave behind is suppression comments that may not take effect. Where this report calls CodeQL "required", that is incorrect.
Review — needs work
Full review of the whole change.
Implements ROCMAI-82 (Phase 6a: move the driver-install, engines and serve clusters out of main.rs). It departs on purpose from the ticket's "move only the fn bodies, keep shared types at the root": cluster-owned types moved too. The commit messages and docs/architecture.md both say so. The PR description was not read; see coverage.
Blocking
- (standing since 3b4d8b7, revised) The required CodeQL check fails at the head commit, and the suppression comments added for it are probably in the wrong place —
apps/rocm/src/main.rs:4259-4264,apps/rocm/src/main.rs:4544-4547,apps/rocm/src/serve_cmd.rs:936-939,apps/rocm/src/main.rs:28207-28210statusCheckRollupfor ffc8177 showsCodeQL | COMPLETED | FAILURE. Every other check passed.- Commit 95f6d0a added
// codeql[rust/cleartext-logging]comments to silence these alerts. In four of the six places the marker is the first line of a three-line comment, so it is not on the line directly above the flagged statement. Atmain.rs:4259it is four lines aboverender_endpoint_client_config(&report.endpoint_url, key). - Only the single-line markers at
main.rs:28211andmain.rs:28213sit directly above their statement. - The failing check is confirmed. That the placement causes it is inferred: I could not run CodeQL.
- Confidence 85 · mechanical · Fix: put the
codeql[...]marker on the line directly above the flagged statement, with the explanation above it, then confirm CodeQL goes green. If an alert stays open, dismiss it in code scanning with a reason.
Settled standing blocker (from the round at 2d5a4ee): the branch now builds once merged with main. This round's blind review checked the merged tree on its own initiative and found it builds. The objection was then put to the reviewer together with the author's account of the fix. That second check is anchored evidence, not a blind finding.
- On
ffc81772merged with currentmain(4fd7c810, a clean merge),select_serve_engineandServeEngineSelection(both fields included) arepub(crate)and re-imported atmain.rs:38. detect_host_gpu_summaryis back in therocm_coreimport atmain.rs:55.cargo checkandcargo clippy -p rocm --all-targets -- -D warningsexit 0.cargo test -p rocm --bin rocmpasses 1013 tests, with 1 ignored.- The fix landed in 3b4d8b7, and no later commit in the range undoes it.
Non-blocking
- (new; overlaps a 3b4d8b7 finding) New doc text says things the code contradicts —
docs/architecture.md:18,apps/rocm/src/engines_cmd.rs:11-14,apps/rocm/src/serve_cmd.rs:8-9- In each case the code is what governs, because it is what compiles. Correct the text.
architecture.mdsays "driver_install.rs's andengines_cmd.rs's types are used nowhere else". ButDriverInstallResultandDriverInstallError(driver_install.rs:304-311) arepub(crate)withpub(crate)fields, andinstall()inmain.rs:3567-3593readsresult.output,result.executed,error.executedanderror.source. Only the plan/state types are private to the module.- The
engines_cmd.rsheader says the helpers other thanengine_manages_own_runtime"are used only from other root-level commands inmain.rs".ensure_self_managed_engine_readyhas no caller inmain.rs; its only external caller isserve_cmd.rs:737. The header also leaves out thatengine_manages_own_runtimeis called frommain.rs:8332. - The
serve_cmd.rsheader says it is re-imported viause crate::serve_cmd::{serve, ServeArgs};. The actual import atmain.rs:38is{ServeArgs, select_serve_engine, serve}. - Confidence 85 · mechanical · Fix: rewrite the three sentences to match the call graph.
Decisions for the author
- (new)
main.rs'stestsmodule now serves as a shared fixture module — tradeoffmod testsbecamepub(crate) mod tests, andScopedTestEnv,test_paths,write_test_pip_runtimeandtest_runtime_manifest_for_updatewere widened so the three new modules can import them fromcrate::tests::.- No earlier extraction in
apps/rocmreaches into another module's tests like this. It keeps the diff to a pure move. - The alternative is a dedicated
#[cfg(test)]test-support module (asrocm-corehastest_support.rs). That is a cleaner base for phases 6b–6f, but the diff here would no longer be a pure move. - The pattern will spread with each later phase, so choose it on purpose.
- (new) Two tests moved into
serve_cmd.rsonly exercise code that stayed inmain.rs— non-blocking-improvementhybrid_planner_bakes_the_host_engine_into_the_generated_serve_command(serve_cmd.rs:1000) testsbuild_freeform_plan_with_recipes(main.rs:16850). Its siblinghybrid_planner_*tests stayed inmain.rs.launch_lock_makes_gpu_select_and_claim_atomic(serve_cmd.rs:1644) testsselect_gpu_indices_under_launch_lock(main.rs:18614).- Both are about serve, so this is defensible. The cost is that the planner tests are now split across two files.
Positive signals
- The move is faithful.
- I compared the multiset of removed lines with the multiset of added lines across
apps/rocm/src. Everything matches except visibility widenings,useimports, module headers,moddeclarations and the CodeQL comments. - The dispatch call sites are unchanged.
- The set of test names in the merged result is identical to
prw-base's set (536).
- I compared the multiset of removed lines with the multiset of added lines across
- Documenting the "mechanical relocation with owned types" variant in
docs/architecture.mdin the same change keeps the module map current. AGENTS.md §5 and the ticket both ask for that.
Deployment notes
None
What this covered
- Read: the whole change at 70d6aa1 (merge-base)…ffc81772.
- The three new modules were read completely by two fan-out workers. A third worker read
docs/architecture.mdand checked AGENTS.md compliance and history. - I checked the
main.rschanges with a line-multiset equivalence check plus reading the hunks that were not moves.
- The three new modules were read completely by two fan-out workers. A third worker read
- Merge with current base:
- The branch is 8 commits behind
prw-base.git merge-treeagainstprw-base(4fd7c81) merges cleanly. - On the merged tree,
cargo check -p rocm --all-targetsandcargo clippy -p rocm --all-targets -- -D warningsboth exited 0 on Linux. - Tests were not run locally; CI's
build-and-testandwindows-build-and-testpassed at the head.
- The branch is 8 commits behind
- Did not run:
- The prior-changes pass: earlier review comments are off-limits to this run.
- The design questions (Step 7) were worked through by the driver, not by independent workers.
- The Windows
cfgarms were checked by reading only; none of the three new files gate non-test code oncfg. cargo xtask check-architecture-docwas not run locally; its CI check passed.
- Intent:
- The PR description was not read, because this run was limited to the
statusCheckRollupgh call. Intent was judged from the five commit messages and ROCMAI-82, which was retrieved. - The ticket's acceptance criteria are met: unchanged dispatch call sites, tests moved with their clusters, unchanged test count, architecture doc updated in the same PR.
- The PR description was not read, because this run was limited to the
- Not reconciled against the PR's discussion here.
Phase 6a of the ROCMAI-27 modularization plan. Mechanically relocates install_driver/reconcile_driver_install, engines(), and serve() out of main.rs into their own files, mirroring the automations.rs/uninstall.rs convention. Unlike those two, these clusters own private types used nowhere else, so the types moved with their functions rather than staying in main.rs. dispatch() call sites are byte-identical. Tests moved with their clusters into each new file's own test module; total test count is unchanged. Architecture doc updated in the same change. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Review follow-up: ServeEngineSelection had zero remaining callers in main.rs but was left behind instead of moving into serve_cmd.rs with the rest of the cluster. Likewise write_claiming_record, side_by_side_runtimes, test_examine, and dkms_planning_os_releases were widened to pub(crate) as shared test fixtures, but each had exactly one external caller (serve_cmd.rs, engines_cmd.rs, and driver_install.rs respectively) — true single-cluster-local items that belong in that cluster's own test module, not reached back into main.rs via crate::. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Rebasing onto main surfaced two problems review caught: - main gained assess_model_for_host (#407) between branch point and rebase, calling select_serve_engine/ServeEngineSelection::engine and detect_host_gpu_summary — all made unreachable by this split. Restore the import and widen visibility to pub(crate) where needed. - Several helpers (render_engine_inventory_text, path_is_same_or_inside, the Phase 9 test-section header, two inventory/examine tests) had exactly one consumer in a new module but stayed in main.rs, or landed in the wrong test module. Move each to its sole consumer; correct two stale module-doc claims (driver_install's general-purpose helpers, serve_cmd's managed-service-spawning tail) to state the real reason they stayed put instead of a reason that no longer holds. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
The one-time terminal display of a freshly generated API key (already documented as the intended delivery channel on render_endpoint_client_config) is pre-existing, reviewed code that this refactor only moved into serve_cmd.rs or shifted within main.rs. CodeQL's incremental PR analysis re-flagged it as "new" purely because the diff is too large to match the moved/shifted lines back to their baseline location; four of the six alerts are already open, unrelated, pre-existing findings on main (#790, #791, #794, #795). Annotate each site with a codeql[rust/cleartext-logging] suppression comment explaining why it is a false positive, rather than dismissing the alerts out-of-band. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
Correct seven doc-accuracy findings from the pr-review-watcher round at 3b4d8b7, all in prose only, no behavior change: - docs/architecture.md: "dispatch fn" -> "handler fn" (install()/dispatch() stay in main.rs; only the handlers relocate); drop the "private types used nowhere else" claim for serve_cmd.rs (ServeEngineSelection/ServeArgs are pub(crate) and used from main.rs; only ServeGenerationDefaults is private); limit the unqualified-access claim to the five dispatch-adjacent clusters, since endpoint_keys.rs/logging.rs are also mechanically relocated but reached via qualified paths. - driver_install.rs/engines_cmd.rs headers: InstallTarget/Cli aren't referenced from either file; say so instead of claiming they're reached through `crate::`. - serve_cmd.rs header: same Cli correction, and widen the explanation for why serve-only helpers stay in main.rs. collect_serve_notes, validate_bind_host, resolve_endpoint_auth, ensure_public_bind_engine_supported, validate_pinned_gpu_index, and print_managed_launch_plain have no non-test caller left in main.rs and no entanglement with the spawning tail's launch helpers -- they stayed to keep this extraction a minimal diff, not because of coupling. Deferred per reviewer's own suggestion rather than widened into a bigger relocation in this PR. - main.rs re-import comment: select_serve_engine is a fn, not a "selection type", and its re-import serves assess_model_for_host, not dispatch. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
The codeql[rust/cleartext-logging] tags added in 95f6d0a did not (and could not) suppress anything: Rust's CodeQL pack has no AlertSuppression.ql query yet (github/codeql#21637, fix pending in github/codeql#21638), so the tag has zero effect regardless of placement today. But placement was also wrong on its own terms -- GitHub's suppression comments only cover the single line immediately following them, and three of the five tags sat atop multi-line rationale blocks instead of as the last line directly before the flagged code. Move each tag to be the last comment line immediately before its flagged statement, with the rationale prose above it. This is inert now but will start working the moment Rust gains AlertSuppression.ql support, with no further edits needed here. The five alerts this was meant to suppress (#794, #795, #830, #833, #834) are dismissed separately via the code-scanning API as pre-existing, reviewed behavior unchanged by this PR -- the same intentional one-time API key display already open as #790/#791 on main, re-flagged as new only because this refactor moved the lines. Signed-off-by: Jussi Elo <jussi.elo@amd.com>
84f3d3a to
fd17f31
Compare
Summary
main.rsmodularization effort (tracked indocs/architecture.md): mechanically relocatesinstall_driver/reconcile_driver_install,engines(), andserve()out ofmain.rsinto their own files (driver_install.rs,engines_cmd.rs,serve_cmd.rs), mirroring the existingautomations.rs/uninstall.rsconvention.ServeArgs, engine-recipe structs). Those types moved with their functions rather than staying inmain.rs, since leaving ~15 cluster-private types behind would defeat the purpose of shrinking the file.docs/architecture.mdis updated to document this as a variant of the mechanical-relocation pattern.dispatch()'s call sites are byte-identical (diffed directly against the pre-change function — zero differences).#[cfg(test)] mod tests, rather than staying behind inmain.rs's test module.Test plan
cargo build— cleancargo clippy --workspace --all-targets -- -D warnings— cleancargo test --bin rocm— 971 tests before and after this change (exact match); 970 passed, 1 pre-existing ignored, 0 failurescargo xtask manifest --check/cargo xtask check-architecture-doc— cleanprek run --all-files --no-group local-tools— all hooks passtests/e2e-cucumber(cargo xtask e2e) — 151/153 scenarios, 0 unexpected failures (2 pre-existing xfail)If this PR fixes a bug, searched
tests/e2e-cucumber/expectations.tomlfor the fixed ticket ID and removed/narrowed any now-stale xfail rows. — N/A, not a bug fix.If this PR adds a new subcommand or subsystem, its domain implementation lives in its own file per
docs/architecture.md(the clap declaration and dispatch wiring staying inmain.rs/lib.rsis expected, not a violation). — no new subcommand; existinginstall driver/engines/servecommand handlers relocated per this exact pattern.Every new or changed user-facing message was read against the code path that runs after it, and its test asserts the resulting state — not only the wording — per AGENTS.md §3. — no user-facing messages changed; pure code motion.