Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -1,27 +1,65 @@
Training & Deploying COMPASS Navigation Policy with Real2Sim NuRec
====================================================================

COMPASS (Cross-embodiment Mobility Policy via Residual RL and Skill Synthesis) trains and
deploys cross-embodiment navigation policies on NuRec Real2Sim assets — photoreal
Gaussian-splat reconstructions of real spaces — in Isaac Lab.
COMPASS (Cross-embodiment Mobility Policy via Residual RL and Skill Synthesis) `[arXiv:2502.16372] <https://arxiv.org/abs/2502.16372>`_ trains cross-embodiment navigation policies. We support training COMPASS using NuRec assets in Isaac Lab.

This tutorial now lives in the **COMPASS handbook**, next to the code it documents, where it
stays in sync with the NuRec support branch.

NuRec Assets
------------

Real2Sim NuRec assets are available on `HuggingFace <https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-NuRec/tree/v0.2>`_.

Assets are generated by Nvidia's Real2Sim NuRec workflow.
Real2Sim NuRec pipeline generates data using multiple stereo RGB pipelines.
Examples of assets here.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Incomplete placeholder text — "Examples of assets here." reads like an unfinished draft note rather than documentation prose. It will appear verbatim in the rendered HTML and doesn't give the reader any useful information. Consider replacing it with a brief descriptive sentence or removing it entirely.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!


Using 4 stereo RGB cameras

- `Galileo <https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-NuRec/tree/v0.2/nova_carter-galileo>`_

.. figure:: ../../_static/policy_deployment/03_compass_with_NuRec/galileo.gif
:alt: Galileo NuRec Real2Sim asset
:width: 480px

Using 1 stereo RGB camera
Comment on lines +16 to +24

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Camera-count lines lack RST section markup — "Using 4 stereo RGB cameras" and "Using 1 stereo RGB camera" appear to be intended as sub-headings, but they have no RST underline characters beneath them. Without a heading marker (e.g., ~~~~~), they will render as plain paragraph text, losing the visual hierarchy. Either add proper RST heading underlines or reformat them as bold lead-in text so the intent is explicit.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!


- `Living Room <https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-NuRec/tree/v0.2/hand_hold-endeavor-livingroom>`_

.. figure:: ../../_static/policy_deployment/03_compass_with_NuRec/living_room.gif
:alt: Living room NuRec Real2Sim asset
:width: 480px

We also support reconstruction done using `XGRIDS Portal Camera <https://www.xgrids.com/intl/portalcam>`_ using their own proprietary software.

- `Wormhole <https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-NuRec/tree/v0.2/xgrid-wormhole>`_

.. figure:: ../../_static/policy_deployment/03_compass_with_NuRec/wormhole.gif
:alt: Wormhole Room XGRIDS reconstruction
:width: 480px

The complete tutorial for training COMPASS with NuRec assets is available in **COMPASS handbook**.

Where to find it
----------------

- **Repository:** `COMPASS on GitHub <https://github.com/NVlabs/COMPASS>`_
- **Branch:** ``samc/support_nurec_assets_isaaclab_3.0`` (NuRec Real2Sim support)
- **Guide:** `docs/handbook/workflows/nurec_real2sim.md <https://github.com/NVlabs/COMPASS/blob/samc/support_nurec_assets_isaaclab_3.0/docs/handbook/workflows/nurec_real2sim.md>`_ in the COMPASS repository.

.. code-block:: bash

git clone https://github.com/NVlabs/COMPASS.git
cd COMPASS
git checkout samc/support_nurec_assets_isaaclab_3.0
# Open docs/handbook/workflows/nurec_real2sim.md
- **Branch:** ``real2sim/isaaclab_3.0`` (NuRec asset support for training with COMPASS)
- **Guide:** `docs/handbook/workflows/nurec_real2sim.md <https://github.com/NVlabs/COMPASS/blob/real2sim/isaaclab_3.0/docs/handbook/workflows/nurec_real2sim.md>`_ in the COMPASS repository.

The guide covers Isaac Sim / Isaac Lab installation, COMPASS setup, downloading and
registering NuRec Real2Sim scenes, training a residual RL specialist, evaluation,
ONNX / TensorRT export, and ROS2 / sim-to-real deployment.

Sample training with NuRec assets with different embodiments.

1. Galileo training with G1 robot

.. figure:: ../../_static/policy_deployment/03_compass_with_NuRec/galileo_g1_training.gif
:alt: Galileo G1 training view
:width: 480px

2. Xgrid babyroom (along with sim obstacles) with Carter robot

.. figure:: ../../_static/policy_deployment/03_compass_with_NuRec/xgrid_wormhole_carter_training.gif
:alt: XGRIDS Wormhole training view
:width: 480px
Comment on lines +61 to +65

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Caption and alt text name a different room than the GIF filename — the description says "Xgrid babyroom" but the referenced file is xgrid_wormhole_carter_training.gif and the :alt: text reads "XGRIDS Wormhole training view". All three should agree on the room name to avoid confusing readers.

Suggested change
2. Xgrid babyroom (along with sim obstacles) with Carter robot
.. figure:: ../../_static/policy_deployment/03_compass_with_NuRec/xgrid_wormhole_carter_training.gif
:alt: XGRIDS Wormhole training view
:width: 480px
2. XGRIDS Wormhole (along with sim obstacles) with Carter robot
.. figure:: ../../_static/policy_deployment/03_compass_with_NuRec/xgrid_wormhole_carter_training.gif
:alt: XGRIDS Wormhole training view
:width: 480px

Loading