Integrate UWLab with RSL-RL v5.4.1 (upstream-first)

[JTran-UW]

Summary

Integrate the UW-specific changes on leggedrobotics/rsl_rl v5.4.1 (016c7ede710e358b7d6c205642e2540804d6281f) for the upgraded Isaac Sim 6.1 UWLab stack tracked in UW-Lab/UWLab#36. RSL-RL itself remains simulator-independent.

main points to that upstream base. The UW integration and focused follow-up fixes are linear commits on top, preserving upstream commit identities without adding integration merges or duplicate Greptile replay commits.

Changes

• Broadcast tensor state directly for distributed PPO, retaining normalization buffers and optional RND predictor state.
• Reuse an already-initialized distributed process group instead of initializing it twice.
• Preserve dictionary logger configuration across repeated initialization, and accept built-in string logger aliases case-insensitively without altering custom writer class names.
• Skip vector-observation normalization when an image-only CNN has no vector observation groups.
• Pin all nine workflow-level action references to verified full commit SHAs matching their existing releases. Workflow permissions, deployment gates, and repository protection settings are unchanged.
• Carry forward UW's existing Greptile review configuration and document the supported model API.

Compatibility boundary

This integration targets the RSL-RL 5.x separate actor/critic model APIs used by upgraded UWLab. It does not restore the old combined ActorCritic, its legacy gSDE implementation, or low-level MLP compatibility methods. Older callers should remain on their pinned 3.x stack; the upgraded workflow retains its checkpoint-layout conversion and distribution-aware export path in UWLab.

The former UW main at 37c128981e5e8e2617868075253fa111ee4891bd is preserved as isaaclab2, legacy/3.1.2, and annotated tag uw-v3.1.2.

Verification

• 180 local CPU test cases passed: nine focused cases plus the remaining 171 library cases. Five of the new cases failed before the runtime fixes; lowercase logger aliases and normalization-disabled controls already passed.
• Both UWLab checkpoint-layout tests passed during integration validation.
• After the follow-up runtime fixes, all 18 converted UWLab checkpoints, with 96 recorded/synthetic observations each, strictly loaded and retained bit-exact CPU mean actions, standard deviations, seeded samples, critic values, and native/UW JIT outputs against the saved integration reference (6a0acc5).
• Native and distribution-aware JIT exports passed for all 18 checkpoints; representative checkpoint ONNX export/checking and the library's model-export tests passed.
• Pre-commit passed. See the checks on the current head for the GitHub Python 3.9-3.12 matrix and docs build; results from older heads are not substituted for current CI.
• The broader image-only export failure was not reproduced: diagnostic JIT parity and ONNX export/checking passed with the required empty 1D placeholder. This is not a claim of full ONNX Runtime parity for every CNN configuration.

These are CPU regression/parity/export checks, not a new full GPU success-rate evaluation. A strict local Sphinx warnings-as-errors build previously reproduced an existing upstream ambiguous-reference warning.

Remaining supply-chain limitation

The pinned upstream upload-pages-artifact and pre-commit/action composites still invoke mutable transitive actions/upload-artifact@v4 and actions/cache@v4 references. This PR pins our workflow-level references; it does not claim a fully hermetic dependency graph or silently replace those composite implementations.

Landing and release

The upstream base is an ancestor of this PR. Land through a linear-history-compatible operation; no force-update of main is needed.

After review and merge, verify the final SHA before switching the default branch from isaaclab2 to main, tagging the UW fork release as uw-v5.4.1, and updating UWLab's dependency to the final official revision. Those release actions remain separate.

[greptile-apps]

Confidence Score: 5/5

[High risk] Upgrades distributed training and logging configuration.

The PR appears safe to merge; no new actionable regression or outstanding previous finding was identified.

Summary

The PR integrates the UWLab-targeted RSL-RL 5.x model API on the stated upstream base. Since the previous review, it preserves logger configuration, fixes image-only normalization updates, adds focused tests, and pins workflow-level action references.

_{Reviews (3) · Last reviewed commit: "Pin workflow actions to verified release..."}

[patrickhaoy]

@greptileai

[patrickhaoy]

@greptileai Please re-review current head 84dc4c90a8522c9702efc801bd20c572716f6a1b against the actual main base, upstream 5.4.1 at 016c7ede710e358b7d6c205642e2540804d6281f, and the updated PR description.

Two follow-ups address the confirmed modern issues:

• d172321: copy dictionary writer options before consuming class_name, preserve caller configuration across repeated logger initialization, normalize built-in string aliases case-insensitively while preserving custom class names, and skip vector normalization for image-only CNNs with no vector groups.
• 84dc4c9: pin all nine workflow-level action references to verified SHA commits corresponding to the releases already selected by the previous refs. No workflow permissions, deployment gates, repository rules, or allowlists were changed.

Validation: five new regression cases failed before the runtime fixes. All nine focused cases now pass, as do the remaining 171 library cases (180 CPU cases total). All 18 converted UWLab checkpoints retain bit-exact CPU mean/std/seeded samples/critic values and native/UW JIT outputs against the saved prior reference; JIT and representative ONNX export checks passed. Current-head GitHub CI is separate from these local results.

For the older “required fork behavior dropped” and “legacy checkpoints cannot load” findings: the support contract was explicitly narrowed to the upgraded UWLab RSL-RL 5.x actor/critic API. We are not deferring restoration to PR #3 or promising direct combined 3.x checkpoint compatibility. The 3.x implementation is preserved at isaaclab2, legacy/3.1.2, and uw-v3.1.2 (37c1289); README and the PR document that boundary. The current PR base is already upstream 5.4.1, not the previous legacy main.

The image-only normalization failure was reproduced and fixed. The broader image-only export failure was not reproduced in the diagnostic probes: JIT parity and ONNX export/checking passed using the required empty 1D placeholder, with and without normalization. That is not a claim that every CNN export/ONNX Runtime scenario has been qualified.

Residual security limitation: the pinned upload-pages-artifact and pre-commit/action composites themselves still use mutable transitive upload-artifact@v4 and cache@v4 refs. The workflow-level refs are now pinned, but we are not claiming complete transitive supply-chain immutability. This remains explicitly documented rather than waived or hidden.

Please assess the current diff and report any remaining concrete issues. Existing review threads have not been silently resolved or dismissed.