Mesh-based Non-collision Constraints

[zhx06]

Summary

Add mesh-based non-collision constraints via sphere-to-SDF

Detailed description

• Introduces CollisionMode.MESH as an alternative to AABB for no-overlap constraints, using greedy sphere decomposition + differentiable Warp SDF queries against actual collision geometry.
• Solver falls back to AABB for pairs where either object lacks a mesh. Validator mirrors this.
• CLI: --collision_mode mesh enables the new path.

Core files

• relations/warp_sdf_kernels.py — differentiable SDF queries on Warp meshes
• relations/warp_mesh_manager.py — sphere decomposition and mesh caching
• relations/relation_solver.py — vectorized mesh collision loss during optimization
• relations/object_placer.py — mesh collision validation at placement time
• relations/relation_loss_strategies.py — per-pair mesh loss for the strategy API

[isaaclab-review-bot]

🤖 Isaac Lab Review Bot — PR #771

Mesh-based Non-collision Constraints

Summary

This PR adds sphere-to-SDF mesh collision support as an alternative to AABB overlap detection. The architecture is clean — CollisionMode.MESH integrates well into the existing NoCollisionLossStrategy dispatch, and the greedy sphere decomposition + Warp SDF kernel approach is sound. The test suite is thorough (542 lines!) with good coverage of dispatch routing, gradient flow, and integration.

Findings

#	Severity	Finding
1	🟡 Warning	Validator creates fresh WarpMeshManager per call — cache never reused
2	🟡 Warning	Scale applied post-transform in extract_trimesh_from_usd may be incorrect for nested prims
3	🔵 Suggestion	object_base.py abstract method has no explicit return None
4	🔵 Suggestion	Sentinel warning pattern on function object is not thread-safe
5	🔵 Suggestion	Consider documenting the rotated-anchor limitation more prominently

See inline comments for details.

---

Update (5e86ed0a): Reviewed incremental changes since 655ac73.

Addressed Findings

• ✅ Finding #1 resolved — _get_cpu_mesh_manager() now lazily creates and caches the WarpMeshManager on the instance, eliminating redundant allocations per validation call. Good fix.
• ✅ Finding #2 resolved — Removed erroneous .T transpose on ComputeLocalToWorldTransform in usd_helpers.py. USD returns row-major matrices; the transpose was producing incorrect vertex transforms for nested prims.

Other Changes

• Validation logic refactored (_validate_placement): Mesh mode now skips AABB validation entirely (else branch). Previously both checks ran in mesh mode — the AABB check was redundant and could produce false negatives for non-convex shapes. Clean improvement.
• Test suite trimmed: Removed test_sphere_count_respects_budget, test_cache_key_differs_for_different_meshes, test_dispatch_falls_back_when_obj_is_none, and test_mesh_zero_loss_separated_cylinders. These removals look intentional (simplified scope / covered elsewhere), though removing cache-key differentiation test reduces regression coverage on the caching layer.

Remaining Observations

• Findings #3–#5 from original review remain unaddressed (low priority, suggestions only).
• The new _get_cpu_mesh_manager uses hasattr check — works fine but Optional attribute initialized in __init__ would be more explicit.

Overall: Good incremental improvement. The two main warnings from the initial review are resolved. No new concerns.

---

Update (729d892c): Reviewed incremental changes since 5e86ed0a.

Changes in this push (2 files)

1. relation_loss_strategies.py — Added parent_pos_resolved.expand(batch_size, -1) before the per-batch loop. This fixes a shape mismatch when parent_pos_resolved is not already batch-expanded (e.g., single parent broadcast to multiple children). Correct fix.

2. warp_mesh_manager.py — Wrapped getattr(obj, "scale", ...) in tuple() for cache key computation. This prevents unhashable types (e.g., numpy arrays or torch tensors returned by .scale) from breaking the dict lookup. Necessary bugfix.

Assessment

Both changes are small, targeted bugfixes. No new concerns introduced. All previous suggestions (#3–#5) remain low-priority and unaddressed.

[greptile-apps]

Greptile Summary

Introduces CollisionMode.MESH as an opt-in alternative to AABB for no-overlap constraints, backed by greedy sphere decomposition and differentiable Warp SDF queries. The feature adds two new core modules (warp_sdf_kernels.py, warp_mesh_manager.py), extends RelationSolver with a vectorized multi-mesh kernel + AABB broadphase cache, and plumbs mesh validation through ObjectPlacer. BBOX mode is fully unchanged when the new flag is not set.

• warp_sdf_kernels.py implements a custom torch.autograd.Function bridging Warp BVH queries to PyTorch gradients, with both single-mesh and multi-mesh (vectorized) variants and a per-solve sentinel warning mechanism.
• warp_mesh_manager.py provides greedy set-cover sphere decomposition and a content-hash-keyed cache; RelationSolver pre-builds a vectorized pair cache once per solve() call and uses an AABB broadphase to skip distant pairs during the optimization loop.
• The fallback path (AABB for pairs lacking a mesh) is mirrored in both the solver and the validator, and all rotation-frame logic is bidirectional (source → target yaw transform applies correctly).

Confidence Score: 4/5

Safe to merge for the common case; all rotation-frame math and the previous-review regressions are correctly addressed. One edge case worth tracking: non-uniform scale on USD assets with rotated world prims will silently produce a misshapen collision mesh.

The mesh/SDF pipeline, sphere decomposition, autograd bridge, solver caching, and bidirectional yaw math are all well-implemented and thoroughly tested. The two remaining observations are both narrow: the missing explicit return None in the base class is cosmetic, and the scale-ordering issue in extract_trimesh_from_usd only manifests when an asset has a non-identity world-space rotation AND a non-uniform external scale — an uncommon combination in practice. BBOX mode is completely untouched.

isaaclab_arena/utils/usd_helpers.py (scale application order) and isaaclab_arena/assets/object_base.py (implicit None return in base method)

Important Files Changed

Filename	Overview
isaaclab_arena/relations/warp_sdf_kernels.py	New file; correct autograd bridge for single- and multi-mesh SDF queries with per-solve sentinel warning; reset_sdf_sentinel_warning() fixes the lifetime-once warning issue from a previous review.
isaaclab_arena/relations/warp_mesh_manager.py	New file; greedy sphere decomposition + Warp mesh caching; _cache_key correctly converts obj.scale to tuple to avoid TypeError for list-valued scales.
isaaclab_arena/relations/relation_solver.py	Largest change; adds vectorized mesh cache build + multi-mesh SDF loss; AABB broadphase per-iteration is correct; _mesh_orientations reference stored and read consistently; grad_fn guard for zero-loss step is sound.
isaaclab_arena/relations/object_placer.py	MESH-mode validation correctly bypasses AABB gate; _get_cpu_mesh_manager() caches across attempts; _centers_in_target_frame rotation math is bidirectionally correct; deepcopy skips Warp cache fields.
isaaclab_arena/relations/relation_loss_strategies.py	_compute_mesh_loss now correctly applies net-yaw rotation to sphere centers and rotates the world-space offset into parent frame; .expand(batch_size, -1) fixes the batch-size > 1 IndexError from a prior review.
isaaclab_arena/utils/usd_helpers.py	New extract_trimesh_from_usd; correctly uses row-vector verts_h @ world_tf (no transpose, fixing the prior-review bug); scale is applied per-axis after the world transform (see comment).
isaaclab_arena/assets/object_base.py	Adds get_collision_mesh() with implicit None return; valid Python but could confuse static analysis tools — explicit return None suggested.
isaaclab_arena/tests/test_mesh_collision.py	Comprehensive test suite covering sphere decomposition, dispatch routing, bidirectional SDF gradients, batch sizes, broadphase correctness, yaw-rotation cases, and end-to-end placer.

Sequence Diagram

sequenceDiagram
    participant CLI as CLI/ArenaEnvBuilder
    participant OP as ObjectPlacer
    participant RS as RelationSolver
    participant WMM as WarpMeshManager
    participant SDF as warp_sdf_kernels

    CLI->>OP: "place(objects, collision_mode=MESH)"
    OP->>RS: solve(objects, initial_positions, orientations)
    Note over RS: _prepare_mesh_collision_cache builds fwd+rev caches once
    RS->>WMM: get_warp_mesh(mesh, obj)
    WMM-->>RS: wp.Mesh (BVH, content-hash cached)
    RS->>WMM: get_query_spheres(mesh, obj)
    WMM-->>RS: (K,4) sphere tensor (cached)
    loop Adam optimizer steps
        RS->>SDF: multi_mesh_sdf(active_centers, mesh_ids, indices)
        SDF-->>RS: (N,) SDF values + autograd gradients
        Note over RS: AABB fallback for mesh-less pairs
        RS->>RS: loss.backward then optimizer.step
    end
    RS-->>OP: solved positions
    OP->>OP: _validate_no_overlap_mesh(positions, orientations)
    OP->>SDF: mesh_sdf(centers_in_target, warp_mesh)
    SDF-->>OP: SDF values
    OP-->>CLI: PlacementResult(success, positions)

Loading

_{Reviews (8): Last reviewed commit: "improve docstring and test cases" | Re-trigger Greptile}