[Proposal] Support Kit-less Deformable USD

[YizeWang]

Proposal

Make scripts/demos/deformables.py --visualizer newton work with the default PhysX deformable scene by fixing the Newton shadow-model import path for PhysX-authored deformable USD.

Motivation

The command:

./isaaclab.sh -p scripts/demos/deformables.py --visualizer newton

uses PhysX physics by default and requests the Newton visualizer. PhysX deformables initialize, but the Newton visualizer then asks NewtonManager to build a shadow Newton model from the USD stage.

That shadow-model build fails in ModelBuilder.add_usd() while parsing a UsdGeom.TetMesh. Isaac Lab authors deformable pose metadata in source/isaaclab/isaaclab/sim/schemas/schemas.py:

deformablePose:default:omniphysics:purposes = ["bindPose"]

Newton's USD importer reads non-schema TetMesh attributes into custom_attributes. deformablePose:default:omniphysics:purposes is metadata, not simulation mesh data, but Newton currently treats it as a generic TetMesh custom attribute. Its length is 1, which is not per-vertex, per-tetrahedron, or per-triangle, so Newton raises:

ValueError: Cannot infer frequency for custom attribute 'deformablePose:default:omniphysics:purposes'

NewtonManager._ensure_visualization_model() logs the failure and leaves the model unset. The Newton viewer then calls set_visible_worlds() without a model and raises:

RuntimeError: Model must be set before calling set_visible_worlds()

The texture download errors and PhysX surface deformable warnings appear earlier but are not the root cause of this crash.

Root Cause

Newton visualizer support for a PhysX-backed scene depends on a shadow Newton model. For deformables, that shadow model is rebuilt from USD:

NewtonVisualizer.initialize()
  -> NewtonManager.get_model()
  -> NewtonManager._ensure_visualization_model()
  -> NewtonManager._build_visualization_model_from_stage()
  -> ModelBuilder.add_usd()
  -> newton usd.get_tetmesh()
  -> TetMesh(custom_attributes=...)

The failure happens before the viewer has a model. The final set_visible_worlds() exception is secondary; it only appears because NewtonManager._ensure_visualization_model() logged the import exception and returned with NewtonManager._model still unset.

Alternatives

• Teach Newton USD import to ignore OmniPhysics deformable metadata such as deformablePose:* when building TetMesh custom attributes.
• Import metadata-like TetMesh attributes with an explicit non-geometry frequency or side-channel if Newton wants to preserve them.
• Add a defensive visualizer error path so shadow-model build failures report the real USD import error instead of later failing in set_visible_worlds().

Build Info

• Isaac Lab Version: On top of Support Multi-Back for Demo Scripts (Part 2) and Unify Code Structure #6140
• Isaac Lab Contrib Version: 0.4.4
• Isaac Lab Newton Version: 0.15.2
• Isaac Sim Version: 6.0.0-alpha.212

Additional context

Failure path:

sim.reset()
  -> SimulationContext.initialize_visualizers()
  -> NewtonVisualizer.initialize()
  -> NewtonManager.get_model()
  -> _build_visualization_model_from_stage()
  -> ModelBuilder.add_usd()
  -> newton usd.get_tetmesh()
  -> TetMesh._infer_frequency()

Observed root exception:

ValueError: Cannot infer frequency for custom attribute 'deformablePose:default:omniphysics:purposes': array length 1 matches none of vertex_count (...), tet_count (...), tri_count (...).

Checklist

• I have checked that there is no similar issue in the repo (required)

Acceptance Criteria

• deformables.py --visualizer newton initializes the Newton visualizer for the default PhysX deformable scene.
• Newton USD import handles deformablePose:default:omniphysics:purposes without treating it as per-vertex, per-tet, or per-triangle TetMesh data.
• Shadow-model build failures, if any remain, report the original import failure and do not cascade into Model must be set before calling set_visible_worlds().