Repository navigation
Conversation
* Add onnxscript 0.5.4 as a dependency According to the pytorch docs (https://docs.pytorch.org/docs/stable/onnx_export.html), onnx export depends on onnxscript being installed. In PyTorch 2.9, the export behavior now defaults to torch.export instead of torch.onnx, which is a codepath that imports onnxscript eagerly. --------- Co-authored-by: ClemensSchwarke <clemens.schwarke@gmail.com>
Adds a new perceptive actor-critic class, that can define CNN layers for every 2D observation term. --------- Co-authored-by: ClemensSchwarke <clemens.schwarke@gmail.com>
This PR adds a new Logger class to remove the logging functionality from OnPolicyRunner. This also allows to make the DistillationRunnner leaner. A few minor changes improve overall code quality, e.g., neptune_utils now follows the same structure as wandb_utils. Tested for PPO and Distillation for Tensorboard.
…robotics#101) --------- Co-authored-by: ClemensSchwarke <clemens.schwarke@gmail.com>
Co-authored-by: Lorenzo Terenzi <lorenzoterenzi96@gmail.com>
Change weight initialization statement from Xavier to Kaiming
* log code state inside init_logging_writer * remove self and update comments --------- Co-authored-by: ClemensSchwarke <clemens.schwarke@gmail.com>
--------- Co-authored-by: ClemensSchwarke <clemens.schwarke@gmail.com>
…robotics#180) 1. rnn.py: Add missing `raise` before NotImplementedError -- the exception was being constructed but never raised, causing silent failure when resetting hidden state of done environments with a custom hidden state. 2. ppo.py: Fix wrong index in broadcast_parameters -- when RND is enabled, the predictor was loading model_params[1] (critic state) instead of model_params[2] (its own state), corrupting RND weights during multi-GPU training. 3. cnn_model.py: Fix variable name typo `latend_cnn` -> `latent_cnn`.
…ptions (leggedrobotics#181) - Replace mutable list default `[256, 256, 256]` with tuple `(256, 256, 256)` in MLPModel, CNNModel, and RNNModel to prevent potential shared state across instances. - Replace `assert` statements used for input validation with `raise ValueError(...)` in MLPModel, CNNModel, and RND, since assertions are silently stripped when running with `python -O`.
* add docstrings to all functions * add sphinx docs
* add tests * add onnx tests * add on_policy_runner test * add test workflow * add lint workflow * fix onnx test failure
…otics#189) Deep-copy CNN modules in export wrappers to prevent shared state with the original training model. Fixes leggedrobotics#188.
--------- Co-authored-by: adenzler-nvidia <adenzler@nvidia.com>
…eggedrobotics#201) --------- Co-authored-by: epalmaEth <epalma@student.ethz.ch> Co-authored-by: ClemensSchwarke <clemens.schwarke@gmail.com>
…leggedrobotics#205) --------- Co-authored-by: ClemensSchwarke <clemens.schwarke@gmail.com>
(cherry picked from commit e7cd3c7)
…eggedrobotics#4) (cherry picked from commit 37c1289)
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| import numpy as np | ||
| import torch | ||
| import torch.nn as nn | ||
| from torch.distributions import Beta, Normal |
There was a problem hiding this comment.
This synchronization removes the target branch's gSDE implementation and postpones restoring it to a follow-up PR. The resolve-integration-conflicts directive requires upgrade conflicts to preserve UW-Lab behavior and prohibits deferring required conflict resolution. As a result, users relying on gSDE lose that functionality when this upgrade lands. Port gSDE to the 5.x distribution and model API, or otherwise resolve its removal before merging.
Rule Used: Require conflicts with the PR target and the intended upstream rebase base to be resolved before merge. Review conflict resolutions for lost upstream fixes, dropped UW-Lab behavior, and incompatible combinations of APIs, configuration, checkpoints, d... (source)
| # Load the specified models | ||
| if load_cfg.get("actor"): | ||
| self._raw_actor.load_state_dict(loaded_dict["actor_state_dict"], strict=strict) | ||
| if load_cfg.get("critic"): | ||
| self._raw_critic.load_state_dict(loaded_dict["critic_state_dict"], strict=strict) | ||
| if load_cfg.get("optimizer"): | ||
| self.optimizer.load_state_dict(loaded_dict["optimizer_state_dict"]) |
There was a problem hiding this comment.
Existing 3.1.2 checkpoints store the combined policy under model_state_dict, but this loader directly requires separate actor_state_dict and critic_state_dict keys. Loading a checkpoint produced by the target branch therefore raises KeyError before inference or training can resume. The distillation loader likewise recognizes actor_state_dict, but not the old model_state_dict. This violates the checkpoint-and-inference-compatibility directive to preserve supported checkpoint loading or provide explicit migration guidance.
Rule Used: Changes to model state, normalization, or runner save/load must preserve supported checkpoint loading and training resume, or provide explicit migration guidance. Distinguish resuming training from loading a teacher or loading for inference. Respect ... (source)
| def load( | ||
| self, path: str, load_cfg: dict | None = None, strict: bool = True, map_location: str | None = None | ||
| ) -> dict: | ||
| """Load the models and training state from a given path. | ||
|
|
||
| Args: | ||
| path (str): Path to load the model from. | ||
| load_cfg (dict | None): Optional dictionary that defines what models and states to load. If None, all | ||
| models and states are loaded. | ||
| strict (bool): Whether state_dict loading should be strict. | ||
| map_location (str | None): Device mapping for loading the model. | ||
| """ | ||
| loaded_dict = torch.load(path, weights_only=False, map_location=map_location) | ||
| # Load model | ||
| resumed_training = self.alg.policy.load_state_dict(loaded_dict["model_state_dict"]) | ||
| # Load RND model if used | ||
| if hasattr(self.alg, "rnd") and self.alg.rnd: | ||
| self.alg.rnd.load_state_dict(loaded_dict["rnd_state_dict"]) | ||
| # Load optimizer if used | ||
| if load_optimizer and resumed_training: | ||
| # Algorithm optimizer | ||
| self.alg.optimizer.load_state_dict(loaded_dict["optimizer_state_dict"]) | ||
| # RND optimizer if used | ||
| if hasattr(self.alg, "rnd") and self.alg.rnd: | ||
| self.alg.rnd_optimizer.load_state_dict(loaded_dict["rnd_optimizer_state_dict"]) | ||
| # Load current learning iteration | ||
| if resumed_training: | ||
| load_iteration = self.alg.load(loaded_dict, load_cfg, strict) |
There was a problem hiding this comment.
The public load_optimizer parameter was replaced by load_cfg without a compatibility shim or migration guidance. Existing callers using runner.load(path, load_optimizer=False) now receive an unexpected-keyword TypeError. Callers passing False positionally instead reach alg.load(), where .get() is called on the boolean. This violates the checkpoint-and-inference-compatibility directive, which specifically requires preserving the behavior of load_optimizer.
Rule Used: Changes to model state, normalization, or runner save/load must preserve supported checkpoint loading and training resume, or provide explicit migration guidance. Distinguish resuming training from loading a teacher or loading for inference. Respect ... (source)
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Python | ||
| uses: actions/setup-python@v5 |
There was a problem hiding this comment.
The new workflows reference GitHub Actions through mutable tags rather than immutable full commit SHAs. This is most consequential in the docs workflow because action code runs with pages: write and id-token: write; if a referenced tag is repointed or compromised, substituted code can execute with those permissions. The same hardening applies to the lint and test workflows, especially the third-party pre-commit/action@v3.0.1 reference.
How this was verified: All new workflows execute Git-tagged actions, and the docs workflow grants Pages-write and OIDC-token permissions at workflow scope.
12 participants
Bump main to leggedrobotics rsl_rl v5.3.0 (upstream-first history)
Upstream base: leggedrobotics/rsl_rl v5.3.0 (
f58fda2). This branch is that upstream history verbatim, followed by UW-Lab's two commits on top:e6e50aeAdd Greptile review configuration (Add Greptile #2) — cherry-picked frommain'se7cd3c716012abRequire upstream-first history and conflict resolution in Greptile (Require upstream-first history and conflict resolution in Greptile #4) — frommain's37c1289Why a separate PR:
main's current tree is the 3.1.2 code pluscae6402, while its ancestry already claims v5.3.0 through the ancestry-only merge959ccbc. This branch repairs that per.greptile/rules.md: upstream commits first, UW-Lab commits on top, no merge commits. Becausemainis not an ancestor of this branch, landing it means updatingmainto this history (force-push), not a fast-forward.Not carried:
cae6402(gSDE + multi-GPU written against the 3.1.2ActorCritic, which upstream removed in 5.0; 5.x has native multi-GPU). The 5.x gSDE implementation arrives in the follow-up PR that stacks on this branch (port_rsl_rl_5: Mateo'sfeature/locomotioncommits + the UWLab port fixes).Verification: upstream's own test suite at v5.3.0 (
pytest tests) passes; pre-commit clean.🤖 Generated with Claude Code