Skip to content

feat(g1): stack Quest teleop on shared dual-arm IK - #3436

Draft
TomCC7 wants to merge 7 commits into
cc/feat/dual-arm-teleopfrom
pim/feat/g1-quest-teleop-stacked
Draft

feat(g1): stack Quest teleop on shared dual-arm IK#3436
TomCC7 wants to merge 7 commits into
cc/feat/dual-arm-teleopfrom
pim/feat/g1-quest-teleop-stacked

Conversation

@TomCC7

@TomCC7 TomCC7 commented Aug 11, 2026

Copy link
Copy Markdown
Member

Contribution path

Problem

The G1 Quest teleop branch duplicated the dual-arm IK task, relative-pose handling, and web UI that are now provided by the shared dual-arm teleop work. Keeping both implementations would leave two competing control paths and two UI protocols.

Solution

  • Replace the G1-only IK task with the shared teleop_ik task and Pink pose-target solver.
  • Keep only G1-specific model selection, controlled arm joints, task weights, posture weights, velocity limits, and frame names.
  • Reduce full-body Pinocchio models to their configured controlled joints so the G1 solver cannot satisfy hand targets through the waist or legs.
  • Extend the shared video arm teleop module with locomotion output and route it through MovementManager.
  • Preserve G1 camera streaming and episode recording, including controller Cartesian targets.
  • Remove the obsolete G1 Quest module, custom IK task, tests, and UI protocol. The shared UI now owns reference-pose capture and controller input semantics.

How to Test

dimos --simulation mujoco --scene-package office run unitree-g1-teleop

Validation:

  • 4,073 full-suite tests passed; 33 skipped; 282 deselected.
  • Pre-commit hooks passed.
  • Actual G1 URDF solver-reduction test passed under the self-hosted marker.

AI assistance

Codex with GPT-5 implemented the refactor and tests under human direction.

Checklist

  • I have read and approved the CLA.

@codecov

codecov Bot commented Aug 11, 2026

Copy link
Copy Markdown

❌ 3 Tests Failed:

Tests completed Failed Passed Skipped
3850 3 3847 175
View the top 3 failed test(s) by shortest run time
::dimos.control.tasks.cartesian_ik_task.test_pink_control_ik
Stack Traces | 0s run time
ImportError while importing test module '.../tasks/cartesian_ik_task/test_pink_control_ik.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.../tasks/cartesian_ik_task/test_pink_control_ik.py:24: in <module>
    from dimos.control.tasks.cartesian_ik_task.pink_control_ik import (
.../tasks/cartesian_ik_task/pink_control_ik.py:40: in <module>
    from dimos.manipulation.planning.utils.mesh_utils import prepare_urdf_for_drake
E   ImportError: cannot import name 'prepare_urdf_for_drake' from 'dimos.manipulation.planning.utils.mesh_utils' (.../planning/utils/mesh_utils.py)
dimos.imitation.collection.test_blueprint::test_recorder_reads_aggregate_joint_state[blueprint0]
Stack Traces | 0.001s run time
blueprint = <[RuntimeError("Failed to pull LFS file .../dimos/data/.lfs/xarm_description.tar.gz after 3 attemp...lfs/xarm_description.tar.gz']' returned non-zero exit status 1.") raised in repr()] Blueprint object at 0xffb6bdad9040>

    @pytest.mark.parametrize("blueprint", [learning_collect_quest_xarm7, learning_collect_quest_piper])
    def test_recorder_reads_aggregate_joint_state(blueprint: Blueprint) -> None:
        streams = _joint_streams(blueprint)
    
        # Plain name pairing on both ends, no remap in between.
        assert streams[("collectionrecorder", AGGREGATE)] == AGGREGATE
>       assert streams[("controlcoordinator", AGGREGATE)] == AGGREGATE
E       KeyError: ('controlcoordinator', 'coordinator_joint_state')

blueprint  = <[RuntimeError("Failed to pull LFS file .../dimos/data/.lfs/xarm_description.tar.gz after 3 attemp...lfs/xarm_description.tar.gz']' returned non-zero exit status 1.") raised in repr()] Blueprint object at 0xffb6bdad9040>
streams    = {('ControlCoordinator', 'arm_joints'): 'arm_joints', ('ControlCoordinator', 'coordinator_joint_state'): 'coordinator_j..., 'joint_command'): 'joint_command', ('collectionrecorder', 'coordinator_joint_state'): 'coordinator_joint_state', ...}

.../imitation/collection/test_blueprint.py:45: KeyError
dimos.imitation.collection.test_blueprint::test_recorder_reads_aggregate_joint_state[blueprint1]
Stack Traces | 0.002s run time
blueprint = <[RuntimeError("Failed to pull LFS file .../dimos/data/.lfs/piper_description.tar.gz after 3 attem...fs/piper_description.tar.gz']' returned non-zero exit status 1.") raised in repr()] Blueprint object at 0xffb6bdadff20>

    @pytest.mark.parametrize("blueprint", [learning_collect_quest_xarm7, learning_collect_quest_piper])
    def test_recorder_reads_aggregate_joint_state(blueprint: Blueprint) -> None:
        streams = _joint_streams(blueprint)
    
        # Plain name pairing on both ends, no remap in between.
        assert streams[("collectionrecorder", AGGREGATE)] == AGGREGATE
>       assert streams[("controlcoordinator", AGGREGATE)] == AGGREGATE
E       KeyError: ('controlcoordinator', 'coordinator_joint_state')

blueprint  = <[RuntimeError("Failed to pull LFS file .../dimos/data/.lfs/piper_description.tar.gz after 3 attem...fs/piper_description.tar.gz']' returned non-zero exit status 1.") raised in repr()] Blueprint object at 0xffb6bdadff20>
streams    = {('ControlCoordinator', 'arm_joints'): 'arm_joints', ('ControlCoordinator', 'coordinator_joint_state'): 'coordinator_j..., 'joint_command'): 'joint_command', ('collectionrecorder', 'coordinator_joint_state'): 'coordinator_joint_state', ...}

.../imitation/collection/test_blueprint.py:45: KeyError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@TomCC7
TomCC7 force-pushed the pim/feat/g1-quest-teleop-stacked branch from afb9673 to 7c38ec0 Compare August 11, 2026 03:00
Nabla7 and others added 7 commits August 10, 2026 20:05
14-DOF pinocchio IK over both arms as a g1_dual_arm_ik coordinator
task. Wrist targets arrive on coordinator_cartesian_command with
frame_id "<task>/left|right"; the card binds the port ungated (direct)
and the handler drops foreign task names, since by_task_name's exact
match cannot address sub-targets. Engage is both index triggers via
the teleop_buttons broadcast; disengage holds the last pose. set_estop
drops the held pose so a coordinator E-STOP leaves nothing commanding
the arms, and reset_runtime_state clears solver state after a sim
respawn.
Head-relative absolute retargeting: each controller pose relative to
the headset, yaw-normalized and workspace-scaled, becomes a wrist pose
relative to the waist; thumbsticks map to cmd_vel. The web client now
streams the viewer pose (frame_id "head") and pads Joy buttons to the
layout from_joy requires; the base quest module ignores non-controller
poses instead of letting them kill the websocket.
The groot WBC stack plus quest teleop, head RealSense, and a
memory2-backed episode recorder (B toggles an episode, Y discards).
Camera frames ride SHM, not LCM (raw multicast starved the Orin
control loops), and camera/quest/recorder run as dedicated workers so
per-frame numpy/JPEG work never shares a GIL with the coordinator or
the DDS connection. Command/status streams are declared poseless so
recording doesn't warn per message.
Mypy: the ControlTask protocol declares reactivate: bool | None; the
dual-arm task has no arming concept, so it accepts and ignores it.
@TomCC7
TomCC7 force-pushed the pim/feat/g1-quest-teleop-stacked branch from 1b1e9dc to 82c7e51 Compare August 11, 2026 03:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants