diff --git a/docs/.mintignore b/docs/.mintignore index 6cddbcd88b..e4f5be93c2 100644 --- a/docs/.mintignore +++ b/docs/.mintignore @@ -1,2 +1,5 @@ # Ignore dependency trees under the docs workspace (would otherwise be scanned as pages). node_modules/ +**/node_modules/** +package-lock.json +package.json diff --git a/docs/capabilities/agents/index.md b/docs/capabilities/agents/index.md index b168769505..45bbad2fc1 100644 --- a/docs/capabilities/agents/index.md +++ b/docs/capabilities/agents/index.md @@ -1,7 +1,8 @@ --- title: "Agents" +sidebarTitle: "Overview" --- -LLM agents run as native DimOS modules. They subscribe to camera, LiDAR, odometry, and spatial memory streams and they control the robot through skills. +LLM agents run as native dimOS modules. They subscribe to camera, LiDAR, odometry, and spatial memory streams and they control the robot through skills. ## Architecture diff --git a/docs/capabilities/manipulation/a1z.md b/docs/capabilities/manipulation/a1z.md index 0cfeabb0e7..39abea7d35 100644 --- a/docs/capabilities/manipulation/a1z.md +++ b/docs/capabilities/manipulation/a1z.md @@ -88,7 +88,7 @@ dimos hardware a1z doctor The A1Z has no brakes or hardware e-stop button; the PSU switch is the hardware kill switch. Support the arm and clear its workspace before starting or -stopping DimOS. Disabling the motors makes the arm fall. +stopping dimOS. Disabling the motors makes the arm fall. ```bash dimos run keyboard-teleop-a1z diff --git a/docs/capabilities/manipulation/a750.md b/docs/capabilities/manipulation/a750.md index 7fedf869c3..7d7688fd88 100644 --- a/docs/capabilities/manipulation/a750.md +++ b/docs/capabilities/manipulation/a750.md @@ -32,7 +32,7 @@ Pro tip: run this in a separate terminal while plugging in the arm to verify tha dmesg -w ``` -If the device appears but DimOS gets a permission error when opening it, you may need to add yourself to the `dialout` group: +If the device appears but dimOS gets a permission error when opening it, you may need to add yourself to the `dialout` group: ```bash sudo usermod -aG dialout "$USER" @@ -97,10 +97,10 @@ It supports: | Joint position commands | Sends six joint positions plus a velocity ratio | | Enable/disable | Starts and stops the `a750_control` control loop | | Gripper reads/commands | Reads and commands gripper position in meters | -| Cartesian hardware commands | Stubbed; Cartesian motion is handled through DimOS IK | +| Cartesian hardware commands | Stubbed; Cartesian motion is handled through dimOS IK | | Force/torque reads | Not currently implemented | -The `a750_control` package starts a separate thread with real-time priority for its hardware control loop. That loop sends commands and reads back joint state at 1 kHz. The DimOS `read_joint_positions`, `read_joint_velocities`, and `read_joint_efforts` calls return the most recent data cached by that loop rather than synchronously querying the robot, so returned joint data may be up to 1 ms stale. The USB connection also adds roughly 1 ms of latency. +The `a750_control` package starts a separate thread with real-time priority for its hardware control loop. That loop sends commands and reads back joint state at 1 kHz. The dimOS `read_joint_positions`, `read_joint_velocities`, and `read_joint_efforts` calls return the most recent data cached by that loop rather than synchronously querying the robot, so returned joint data may be up to 1 ms stale. The USB connection also adds roughly 1 ms of latency. The adapter requires the optional manipulation dependency: @@ -142,7 +142,7 @@ The A-750 teleop command uses the same keyboard controls as the other manipulato ## Known Limitations -- Joint limits are currently approximate: `[-pi, pi]` for each arm joint, with max velocity `pi rad/s` in the DimOS adapter. -- EEF twist commands are integrated and solved by the DimOS coordinator task, not by a native Cartesian mode in the hardware adapter. +- Joint limits are currently approximate: `[-pi, pi]` for each arm joint, with max velocity `pi rad/s` in the dimOS adapter. +- EEF twist commands are integrated and solved by the dimOS coordinator task, not by a native Cartesian mode in the hardware adapter. - Force/torque data is not exposed yet. - Real hardware mode depends on the external `a750_control` package and a reachable serial device. diff --git a/docs/capabilities/manipulation/adding_a_custom_arm.md b/docs/capabilities/manipulation/adding_a_custom_arm.md index 3754ce4545..d91ce418b1 100644 --- a/docs/capabilities/manipulation/adding_a_custom_arm.md +++ b/docs/capabilities/manipulation/adding_a_custom_arm.md @@ -1,11 +1,11 @@ --- title: "How to Integrate a New Manipulator Arm" --- -This guide walks through integrating a new robot arm with DimOS, from writing the hardware adapter to creating blueprints for planning and control. +This guide walks through integrating a new robot arm with dimOS, from writing the hardware adapter to creating blueprints for planning and control. ## Architecture Overview -DimOS uses a **Protocol-based adapter pattern** — no base class inheritance required. Your adapter wraps the vendor SDK and exposes a standard interface that the rest of the system consumes: +dimOS uses a **Protocol-based adapter pattern** — no base class inheritance required. Your adapter wraps the vendor SDK and exposes a standard interface that the rest of the system consumes: ``` ┌──────────────────────────────────────────────────────────────┐ @@ -72,7 +72,7 @@ Below is a complete annotated adapter. Implement each method by wrapping your ve """YourArm adapter — implements ManipulatorAdapter protocol. SDK Units: -DimOS Units: angles=radians, distance=meters, velocity=rad/s +dimOS Units: angles=radians, distance=meters, velocity=rad/s """ from __future__ import annotations @@ -177,7 +177,7 @@ class YourArmAdapter: def set_control_mode(self, mode: ControlMode) -> bool: """Set control mode. - Map DimOS ControlMode enum values to your SDK's mode codes. + Map dimOS ControlMode enum values to your SDK's mode codes. Return False for modes your arm doesn't support. """ if not self._sdk: @@ -389,7 +389,7 @@ print(adapter_registry.available()) # Should include "yourarm" ## Step 3: Create Your Robot Folder and Blueprints -Each robot in DimOS gets its own folder under `dimos/robot/`. This is where you define all blueprints for your arm — coordinator, planning, perception, etc. This follows the same pattern as Unitree robots (`dimos/robot/unitree/`). +Each robot in dimOS gets its own folder under `dimos/robot/`. This is where you define all blueprints for your arm — coordinator, planning, perception, etc. This follows the same pattern as Unitree robots (`dimos/robot/unitree/`). ### 3a. Create the robot directory @@ -482,9 +482,9 @@ to your robot's own `blueprints.py`. Place your URDF/xacro files under LFS data so they can be resolved via `LfsPath`. `LfsPath` is a `Path` subclass that lazily downloads LFS data on first access — this avoids downloading at import time when the blueprint module is loaded. If the planning blueprint selects the RoboPlan TOPP-RA trajectory -parametrizer, DimOS currently pins RoboPlan to `0.5.1`. Every movable joint in +parametrizer, dimOS currently pins RoboPlan to `0.5.1`. Every movable joint in each selected planning group must provide finite, positive velocity limits. -Authored extended acceleration limits take precedence; when absent, DimOS +Authored extended acceleration limits take precedence; when absent, dimOS temporarily inserts a global `2.0 rad/s²` acceleration fallback during RoboPlan model composition: @@ -503,9 +503,9 @@ model composition: RoboPlan loads both limits from its scene model. If either is absent, zero, negative, or non-finite, plan materialization fails before preview or execution -and identifies the affected joint. DimOS does not substitute +and identifies the affected joint. dimOS does not substitute `RobotModelConfig.max_velocity`, `velocity_limits`, or `max_acceleration` for -this backend. Formal per-joint DimOS overrides will be added separately. +this backend. Formal per-joint dimOS overrides will be added separately. ```python skip from dimos.utils.data import LfsPath diff --git a/docs/capabilities/manipulation/index.md b/docs/capabilities/manipulation/index.md index f552628e4a..cb4af9cbf3 100644 --- a/docs/capabilities/manipulation/index.md +++ b/docs/capabilities/manipulation/index.md @@ -117,7 +117,7 @@ request. For example, `planner.backend=roboplan` requires `world_backend=drake`. Trajectory parametrization is a separate startup choice. Joint-space planners -normally return an untimed geometric path; DimOS accepts the plan only after +normally return an untimed geometric path; dimOS accepts the plan only after the selected backend converts that path to a validated timed trajectory: ```bash @@ -267,7 +267,7 @@ Safety behavior for unsupported RoboPlan features: - Planning-critical unsupported inputs fail loudly before planning. Examples include unsupported obstacle geometry, unavailable robot loading APIs, or unavailable collision query APIs. RoboPlan worlds generate a minimal SRDF from - the DimOS robot config, including configured collision-exclusion pairs. + the dimOS robot config, including configured collision-exclusion pairs. - Unverified non-critical query methods raise explicit `NotImplementedError`. In particular, signed minimum-distance semantics are not implemented for RoboPlan until a safe equivalent is verified. diff --git a/docs/capabilities/manipulation/piper_integration.md b/docs/capabilities/manipulation/piper_integration.md index 4df8656ba3..3d32663391 100644 --- a/docs/capabilities/manipulation/piper_integration.md +++ b/docs/capabilities/manipulation/piper_integration.md @@ -1,6 +1,6 @@ --- title: "Piper Integration" -description: "Connect and run a Piper arm with DimOS manipulation and teleoperation blueprints." +description: "Connect and run a Piper arm with dimOS manipulation and teleoperation blueprints." --- ## Optional SLCAN setup @@ -18,7 +18,7 @@ the Piper adapter already exposes a native SocketCAN interface. ## Bring up a native Piper CAN interface Piper uses SocketCAN at 1,000,000 bit/s. For the default vendor setup, use -the DimOS CLI to configure an existing CAN interface and bring it up: +the dimOS CLI to configure an existing CAN interface and bring it up: ```bash dimos piper can-activate can0 diff --git a/docs/capabilities/manipulation/planning_groups.md b/docs/capabilities/manipulation/planning_groups.md index 67b611474f..4b60c38a72 100644 --- a/docs/capabilities/manipulation/planning_groups.md +++ b/docs/capabilities/manipulation/planning_groups.md @@ -1,4 +1,6 @@ -# Manipulation Planning Groups +--- +title: "Planning Groups" +--- Planning groups are named, selectable kinematic chains used by manipulation planning. They let APIs target a specific part of a robot, such as an arm or @@ -28,7 +30,7 @@ Robot configs can provide planning groups explicitly with does not run discovery or synthesize groups in `model_post_init`; callers must pass explicit `planning_groups` there. -When code uses the discovery helper instead of explicit config, DimOS discovers +When code uses the discovery helper instead of explicit config, dimOS discovers groups in this order: 1. Explicit `srdf_path` provided to the helper. @@ -56,7 +58,7 @@ Supported SRDF group forms: Unsupported SRDF forms are skipped with warnings: link groups, nested group references, mixed group declarations, branching or non-serial groups, and SRDF `` metadata. A chain group's `tip_link` is its pose target frame. -An ordered joint-list group can be pose-targeted only when DimOS can validate a +An ordered joint-list group can be pose-targeted only when dimOS can validate a unique serial target frame. ## Fallback behavior diff --git a/docs/capabilities/memory/index.md b/docs/capabilities/memory/index.md index 2b36c08028..e0cf119471 100644 --- a/docs/capabilities/memory/index.md +++ b/docs/capabilities/memory/index.md @@ -1,3 +1,8 @@ +--- +title: "Spatial Memory" +sidebarTitle: "Overview" +--- +
Python diff --git a/docs/capabilities/navigation/index.md b/docs/capabilities/navigation/index.md index 412bae3c37..b9a41158f2 100644 --- a/docs/capabilities/navigation/index.md +++ b/docs/capabilities/navigation/index.md @@ -1,5 +1,6 @@ --- title: "Go2 Navigation Overview" +sidebarTitle: "Overview" description: "Live column-carving navigation and premap relocalization for the Unitree Go2." --- diff --git a/docs/capabilities/navigation/relocalization.md b/docs/capabilities/navigation/relocalization.md index d3820805f9..e67a9fb850 100644 --- a/docs/capabilities/navigation/relocalization.md +++ b/docs/capabilities/navigation/relocalization.md @@ -7,7 +7,7 @@ Relocalization lets a Go2 navigate on a previously built map instead of only on ![relocalize on the live go2 and nav_to a point in the premap](https://raw.githubusercontent.com/dimensionalOS/dimos-docs-assets/main/capabilities/navigation/assets/reloc_and_nav_to.webp) -> **Note:** Requires DimOS v0.0.13 or newer for PGO loop closure and `dimos map` export. +> **Note:** Requires dimOS v0.0.13 or newer for PGO loop closure and `dimos map` export. This guide takes four steps: @@ -16,7 +16,7 @@ This guide takes four steps: 3. Test relocalization in replay, no robot needed 4. Deploy on the live Go2 -Throughout this guide, `{DB_NAME}` is the stem of your recording, for example `recording_go2` for `recording_go2.db`. For `map_file`, pass the same stem and DimOS appends `.pc2.lcm` automatically. +Throughout this guide, `{DB_NAME}` is the stem of your recording, for example `recording_go2` for `recording_go2.db`. For `map_file`, pass the same stem and dimOS appends `.pc2.lcm` automatically. ## 1. Record a run @@ -175,7 +175,7 @@ for example `--relocalizationmodule.map-file=…`. | Field | Default | Description | |-------|---------|-------------| -| `map_file` | `None` (module disabled) | Premap stem or path. DimOS appends `.pc2.lcm` automatically | +| `map_file` | `None` (module disabled) | Premap stem or path. dimOS appends `.pc2.lcm` automatically | | `fitness_threshold` | `0.45` | Minimum ICP fitness to accept a relocalization (0 to 1) | | `publish_loaded_map` | `false` | Republish raw premap on `loaded_map` every 2 s | | `use_carving` | `true` | Column-carve when merging premap and live scan | diff --git a/docs/capabilities/teleoperation/hosted.md b/docs/capabilities/teleoperation/hosted.md index e68f368971..88f4d9a184 100644 --- a/docs/capabilities/teleoperation/hosted.md +++ b/docs/capabilities/teleoperation/hosted.md @@ -1,9 +1,10 @@ --- title: "Remote Teleop" +sidebarTitle: "dimTELE Remote Teleop" description: "Drive robots from anywhere in the world, with low latency, from a browser, phone, or VR headset." --- -**dimTELE** is hosted teleoperation for DimOS robots: operate them remotely +**dimTELE** is hosted teleoperation for dimOS robots: operate them remotely from any browser or Quest headset over WebRTC. The robot dials out to a hosted broker ([teleop.dimensionalos.com](https://teleop.dimensionalos.com)), so you don't diff --git a/docs/docs.json b/docs/docs.json index 0ac3e94154..c3409dddf0 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -1,7 +1,7 @@ { "theme": "aspen", "name": "Dimensional", - "description": "Official documentation for Dimensional (DimOS) \u2014 the modern operating system for generalist robotics: Python-first, agent-native, and hardware-agnostic.", + "description": "Official documentation for Dimensional (dimOS) \u2014 the modern operating system for generalist robotics: Python-first, agent-native, and hardware-agnostic.", "colors": { "primary": "#1682a3", "light": "#1682a3", @@ -28,46 +28,24 @@ "pages": [ "introduction", "quickstart", + { + "group": "Installation", + "pages": [ + "installation/ubuntu", + "installation/osx", + "installation/nix" + ] + }, "requirements" ] }, { - "group": "Installation", - "pages": [ - "installation/ubuntu", - "installation/osx", - "installation/nix" - ] - }, - { - "group": "Usage", + "group": "Build with dimOS", "pages": [ "usage/index", "usage/modules", - "usage/blueprints", - "usage/configuration", - "usage/lcm", - "usage/transforms", - "usage/visualization", - "usage/cli", - "usage/python-api", - "usage/camera_calibration", - "usage/native_modules", - "usage/tool_streams", - "usage/evals", - { - "group": "Data streams", - "pages": [ - "usage/data_streams/index", - "usage/data_streams/advanced_streams", - "usage/data_streams/quality_filter", - "usage/data_streams/reactivex", - "usage/data_streams/storage_replay", - "usage/data_streams/temporal_alignment" - ] - }, { - "group": "Sensor streams", + "group": "Streams", "pages": [ "usage/sensor_streams/index", "usage/sensor_streams/advanced_streams", @@ -77,37 +55,23 @@ "usage/sensor_streams/temporal_alignment" ] }, - { - "group": "Transports", - "pages": [ - "usage/transports/index", - "usage/transports/dds" - ] - } + "usage/blueprints", + "usage/configuration", + "usage/cli", + "usage/python-api" + ] + }, + { + "group": "Agents", + "pages": [ + "capabilities/agents/index" ] }, { "group": "Capabilities", "pages": [ { - "group": "Agents", - "pages": [ - "capabilities/agents/index" - ] - }, - { - "group": "Manipulation", - "pages": [ - "capabilities/manipulation/index", - "capabilities/manipulation/agentic", - "capabilities/manipulation/adding_a_custom_arm", - "capabilities/manipulation/openarm_integration", - "capabilities/manipulation/piper_integration", - "capabilities/manipulation/a750" - ] - }, - { - "group": "Memory", + "group": "Spatial Memory", "pages": [ "capabilities/memory/index", "capabilities/memory/algo_comparison", @@ -122,6 +86,14 @@ "capabilities/navigation/relocalization" ] }, + { + "group": "Manipulation", + "pages": [ + "capabilities/manipulation/index", + "capabilities/manipulation/agentic", + "capabilities/manipulation/planning_groups" + ] + }, { "group": "Perception", "pages": [ @@ -140,7 +112,7 @@ "group": "Platforms", "pages": [ { - "group": "Quadruped", + "group": "Unitree Go2", "pages": [ { "group": "Unitree Go2", @@ -163,11 +135,39 @@ "pages": [ "platforms/humanoid/g1/index" ] + }, + { + "group": "Arms", + "pages": [ + "capabilities/manipulation/adding_a_custom_arm", + "capabilities/manipulation/openarm_integration", + "capabilities/manipulation/piper_integration", + "capabilities/manipulation/a750", + "capabilities/manipulation/a1z" + ] } ] }, { - "group": "Development", + "group": "Going deeper", + "pages": [ + { + "group": "Transports", + "pages": [ + "usage/transports/index", + "usage/transports/dds" + ] + }, + "usage/transforms", + "usage/visualization", + "usage/native_modules", + "usage/tool_streams", + "usage/camera_calibration", + "usage/lcm" + ] + }, + { + "group": "Contribute", "pages": [ "development/conventions", "development/testing", diff --git a/docs/installation/nix.md b/docs/installation/nix.md index 3ad07be3e1..d60d5f97f2 100644 --- a/docs/installation/nix.md +++ b/docs/installation/nix.md @@ -13,7 +13,7 @@ curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix mkdir -p "$HOME/.config/nix"; echo "experimental-features = nix-command flakes" >> "$HOME/.config/nix/nix.conf" ``` -## Using DimOS as a library +## Using dimOS as a library ```sh skip mkdir myproject && cd myproject @@ -33,7 +33,7 @@ source .venv/bin/activate pip install "dimos[misc,sim,visualization,agents,web,perception,unitree,manipulation,cpu]" ``` -## Developing on DimOS +## Developing on dimOS ```sh skip # this allows getting large files on-demand (and not pulling all immediately) diff --git a/docs/installation/osx.md b/docs/installation/osx.md index 5d7fc79d0f..c658b1613c 100644 --- a/docs/installation/osx.md +++ b/docs/installation/osx.md @@ -1,4 +1,7 @@ -## macOS Install (12.6 or newer) +--- +title: "macOS Install (12.6 or newer)" +sidebarTitle: "macOS" +--- ```sh skip # install homebrew @@ -10,7 +13,7 @@ brew install gnu-sed gcc portaudio git-lfs libjpeg-turbo python pre-commit curl -LsSf https://astral.sh/uv/install.sh | sh && export PATH="$HOME/.local/bin:$PATH" ``` -## Using DimOS as a library +## Using dimOS as a library ```sh skip mkdir myproject && cd myproject @@ -23,7 +26,7 @@ source .venv/bin/activate uv pip install 'dimos[misc,sim,visualization,agents,web,perception,unitree,manipulation,cpu]' ``` -## Developing on DimOS +## Developing on dimOS ```sh skip # this allows getting large files on-demand (and not pulling all immediately) @@ -44,7 +47,7 @@ uv run pytest --numprocesses=auto dimos ## Transport note for macOS -LCM over UDP can be unreliable on macOS for large or high-rate replay workloads. DimOS defaults the global stream transport to **Zenoh** on macOS, so you usually do not need `--transport=zenoh`. Use `--transport=lcm` if you need to force the legacy multicast path. +LCM over UDP can be unreliable on macOS for large or high-rate replay workloads. dimOS defaults the global stream transport to **Zenoh** on macOS, so you usually do not need `--transport=zenoh`. Use `--transport=lcm` if you need to force the legacy multicast path. See the [Zenoh quickstart](/docs/usage/transports/index.md#zenoh-quickstart) for install, Linux versus macOS defaults, and `DIMOS_TRANSPORT`. diff --git a/docs/installation/ubuntu.md b/docs/installation/ubuntu.md index 60c5b986fe..98762330b0 100644 --- a/docs/installation/ubuntu.md +++ b/docs/installation/ubuntu.md @@ -8,7 +8,7 @@ sudo apt-get install -y curl g++ portaudio19-dev git-lfs libturbojpeg python3-de curl -LsSf https://astral.sh/uv/install.sh | sh && export PATH="$HOME/.local/bin:$PATH" ``` -## Using DimOS as a library +## Using dimOS as a library ```sh skip mkdir myproject && cd myproject @@ -21,7 +21,7 @@ source .venv/bin/activate uv pip install 'dimos[misc,sim,visualization,agents,web,perception,unitree,manipulation,cpu]' ``` -## Developing on DimOS +## Developing on dimOS ```sh skip # this allows getting large files on-demand (and not pulling all immediately) diff --git a/docs/introduction.mdx b/docs/introduction.mdx index 9b5ea9ada5..94997e2f99 100644 --- a/docs/introduction.mdx +++ b/docs/introduction.mdx @@ -1,29 +1,60 @@ --- -title: "Welcome to Dimensional" -description: "An open source modern operating system for generalist robotics. Python-first, ROS-optional, and agent native." -sidebarTitle: Introduction +title: "Welcome to dimOS" +description: "An open source operating system for generalist robotics. Python-first, ROS-optional, and agent native." +sidebarTitle: "What is dimOS?" --- -**Dimensional** is the modern operating system for generalist robotics. We are setting the next-generation SDK standard and integrating with the majority of robot manufacturers. +**dimOS** is a Python framework for generalist robotics, built by Dimensional. You install it on your laptop with pip, and it talks to robots over the network. ROS is optional. -With a simple install and no ROS required, you can build physical applications entirely in Python that run on humanoids, quadrupeds, or drones. + + + One codebase for humanoids, quadrupeds, drones, and arms. + + + Develop against recorded sessions and simulation before any hardware. + + + LLM agents are modules: they see streams, call skills, and take natural language. + + -Dimensional is agent native. Describe behavior in natural language and build local and hosted multi-agent systems that work with your hardware. Agents run as native modules, subscribing to embedded streams from perception (LiDAR, camera) and spatial memory down to control loops and motor drivers. + +No robot or GPU needed to try it. The [quickstart](/quickstart) replays a recorded session on your laptop in a few minutes. + + +## See it in action + +Agents on real hardware, driven by natural language. + + + + *"Follow the next white car that comes through the intersection"* — MAVLink drone agent on X. + + + *"Patrol the office… if you see someone in a hoodie, follow them"* — autonomous Go2 agent on X. + + ## Capabilities at a glance - **SLAM**, dynamic obstacle avoidance, route planning, and autonomous exploration via both DimOS native and ROS integrations. + Map a space while driving, avoid obstacles, and plan routes — native modules or ROS. - - Detectors, 3D projections, VLMs, and audio processing. + + **dimTELE** — low-latency WebRTC from a browser, phone, or VR headset. No port forwarding. - Agentive control and MCP. Example: *"Hey robot, go find the kitchen."* + Natural-language control, with every skill also exposed as an MCP tool. - Spatio-temporal RAG, dynamic memory, object localization and permanence. + Remember what was seen and where — then navigate back to it. + + + Motion planning and teleop for arms like xArm7, A-750, and OpenArm. + + + Detect objects in camera, place them in 3D, with VLMs and audio built in. @@ -31,9 +62,9 @@ Dimensional is agent native. Describe behavior in natural language and build loc - Install DimOS, run your first blueprint, and inspect the running system. + Replay a real robot session on your laptop. No robot needed. - Go2 quadruped and G1 humanoid setup, simulation, and blueprints for real hardware. + Unitree Go2 and G1 setup for simulation and real hardware. diff --git a/docs/platforms/humanoid/g1/index.md b/docs/platforms/humanoid/g1/index.md index 5e2c259ff9..3a6789b83a 100644 --- a/docs/platforms/humanoid/g1/index.md +++ b/docs/platforms/humanoid/g1/index.md @@ -41,7 +41,7 @@ Check with: `ip addr show` Recommended to setup [tailscale](https://tailscale.com/tailscale-ssh) to avoid needing to setup rounter specific configuraions for wireless control. -## 2. Install DimOS +## 2. Install dimOS SSH into the robot, then: @@ -52,7 +52,7 @@ bash <(curl -fsSL https://pub-4767fdd15e6a41b6b2ce2558d71ec8d9.r2.dev/install.sh #### Notes -DimOS handles DDS setup automatically. If you're using the Unitree SDK directly, set: +dimOS handles DDS setup automatically. If you're using the Unitree SDK directly, set: ```bash export CYCLONEDDS_HOME="$HOME/cyclonedds/install" ``` diff --git a/docs/platforms/quadruped/go2/setup.md b/docs/platforms/quadruped/go2/setup.md index 26b87ddd45..18690774d0 100644 --- a/docs/platforms/quadruped/go2/setup.md +++ b/docs/platforms/quadruped/go2/setup.md @@ -15,7 +15,7 @@ First, install system dependencies for your platform: - [macOS](/docs/installation/osx.md) - [Nix](/docs/installation/nix.md) -Then install DimOS: +Then install dimOS: ```bash uv venv --python "3.12" @@ -66,16 +66,16 @@ export ROBOT_IP= ping $ROBOT_IP ``` -2. Built-in obstacle avoidance is on. (DimOS handles path planning, but the onboard obstacle avoidance provides an extra safety layer around tight spots) +2. Built-in obstacle avoidance is on. (dimOS handles path planning, but the onboard obstacle avoidance provides an extra safety layer around tight spots) -### Ready to run DimOS +### Ready to run dimOS ```bash export ROBOT_IP= dimos run unitree-go2 ``` -That's it. DimOS connects via WebRTC (no jailbreak required), starts the full navigation stack, and opens the command center in your browser. +That's it. dimOS connects via WebRTC (no jailbreak required), starts the full navigation stack, and opens the command center in your browser. ### What's Running diff --git a/docs/platforms/quadruped/go2/simulation.md b/docs/platforms/quadruped/go2/simulation.md index 3f97423ec2..e4687f54a2 100644 --- a/docs/platforms/quadruped/go2/simulation.md +++ b/docs/platforms/quadruped/go2/simulation.md @@ -2,7 +2,7 @@ Run the full Go2 navigation stack without any hardware — replay recorded sessions or simulate in MuJoCo. Same code, no robot. -See [Setup](/docs/platforms/quadruped/go2/setup.md) for installing DimOS first. +See [Setup](/docs/platforms/quadruped/go2/setup.md) for installing dimOS first. ## Try It — No Hardware Needed diff --git a/docs/quickstart.mdx b/docs/quickstart.mdx index 833ff8cdff..58650d71a9 100644 --- a/docs/quickstart.mdx +++ b/docs/quickstart.mdx @@ -1,129 +1,150 @@ --- title: "Quickstart" -description: "Install DimOS, run a replay or simulation, then explore CLI and MCP. No hardware required." +description: "Install dimOS and watch a recorded Unitree Go2 drive on your laptop. No robot or GPU required." --- -In this quickstart, you will be able to replay a Unitree Go2 office navigation session with no hardware, switch to simulation or a live robot. +This quickstart gets dimOS running on your laptop. You install it, then play back a recorded Unitree Go2 session and watch the robot map and navigate an office in a live visualization. You do not need a robot or a GPU for this. -If you use coding agents (OpenClaw, Claude Code or similar), point them at [`AGENTS.md`](/AGENTS.md). +When you are ready for more, the same install works with physics simulation, a real robot, or an LLM agent you can talk to. -## System requirements +## Before you begin -| Component | Minimum | Recommended | -| --- | --- | --- | -| OS | Ubuntu 22.04, macOS 12.6+ | Ubuntu 24.04 | -| Python | 3.12 | Latest | -| RAM | 16 GB | 32 GB+ | -| Disk | 10 GB SSD | 25 GB+ SSD | -| CPU | 8-core Intel / AMD | 12+ cores | -| GPU (optional) | NVIDIA RTX 3000+ (8 GB VRAM) | RTX 4070+ (12 GB+ VRAM) | +You need a machine running **Ubuntu 22.04 or newer** or **macOS 12.6 or newer**, with **Python 3.12**, about **10 GB of free disk**, and **16 GB of RAM**. A GPU is only required later for perception and AI features, so any reasonably modern laptop can run this quickstart. -> GPU is required only for perception, VLMs, and AI features. Optional for basic robot control. +The full hardware matrix, including tested configurations and Jetson boards, is on the [system requirements](/requirements) page. -## Interactive install +If you use a coding agent such as Claude Code or OpenClaw, point it at the repository's [AGENTS.md](https://github.com/dimensionalOS/dimos/blob/main/AGENTS.md) so it understands the codebase conventions. + +## Install dimOS + +There are two ways to install, and you only need one of them. + +### Option A: guided installer (recommended) + +The installer script walks you through the whole setup interactively. It installs the system packages dimOS needs (such as git-lfs and portaudio), installs the [uv](https://docs.astral.sh/uv/) Python package manager if you don't have it, creates a virtual environment, and installs dimOS into it with the extras you choose. ```bash curl -fsSL https://raw.githubusercontent.com/dimensionalOS/dimos/main/scripts/install.sh | bash ``` -## Manual system install +If you prefer to read the script before running it, it lives at [scripts/install.sh](https://github.com/dimensionalOS/dimos/blob/main/scripts/install.sh) in the repository. -If you prefer to install system dependencies yourself, follow the guide for your OS: +When the installer finishes, activate the environment it created and **skip ahead to [Run your first replay](#run-your-first-replay)**. Everything in Option B has already been done for you. -| OS Guide | Notes | -| --- | --- | -| [Ubuntu](/docs/installation/ubuntu.md) | Primary tested path | -| [Nix](/docs/installation/nix.md) | Flakes and dev shell | -| [macOS](/docs/installation/osx.md) | Homebrew-based; less mature than Linux | +### Option B: manual install + +First install the system dependencies for your OS by following the matching guide: -## Python environment +| OS guide | Notes | +| --- | --- | +| [Ubuntu](/installation/ubuntu) | Primary tested path | +| [macOS](/installation/osx) | Homebrew-based, less mature than Linux | +| [Nix](/installation/nix) | Flakes and dev shell | -DimOS targets Python 3.12. The examples use [`uv`](https://docs.astral.sh/uv/); plain `python -m venv` and `pip` work too. +Then create a Python 3.12 environment. The examples use [uv](https://docs.astral.sh/uv/), though plain `python -m venv` and `pip` work the same way: ```bash uv venv --python "3.12" -source .venv/bin/activate # Windows: .venv\Scripts\activate +source .venv/bin/activate ``` -## Install DimOS +Finally install dimOS with the extras this quickstart uses: ```bash uv pip install 'dimos[base,unitree]' ``` -Extras keep installs lean: `base` is runtime, modules, transports, and CLI; `unitree` adds WebRTC and skills for Go2 / G1 (real or replayed). +Extras keep the install lean. The `base` extra brings the runtime, modules, transports, and CLI, while `unitree` adds WebRTC support and the skills for the Go2 and G1 robots, whether real or replayed. -### Replay a recorded session (no hardware) +## Run your first replay -On first run, the Rerun window may stay black briefly while roughly **75 MB** of data downloads from LFS. +dimOS applications are launched from **blueprints**. A blueprint is a ready to run bundle of modules that you start by name with `dimos run`. The `--replay` flag feeds it recorded sensor data instead of connecting to a robot: ```bash dimos --replay run unitree-go2 ``` +**What you should see:** a [Rerun](https://rerun.io) visualization window opens, and after a short wait it fills with the robot's camera feed, a LiDAR point cloud, and a map of an office being built up as the robot navigates through it, with its planned path drawn on top. + + +On the first run, roughly **75 MB** of recorded session data is downloaded before anything appears, so the Rerun window may stay black for a minute or two. That is normal. If it stays black well beyond the download, check the terminal output for errors. + + +Congratulations, you have a full dimOS navigation stack running on recorded data. Everything below is optional and independent, so pick whichever branch matches your goal. + +## Next steps + ### Simulation (MuJoCo) +Instead of replaying recorded data, you can run the robot in a physics simulation. Install the `sim` extra and pass `--simulation`: + ```bash uv pip install 'dimos[base,unitree,sim]' -dimos --simulation run unitree-go2 -dimos --simulation run unitree-g1-sim # humanoid +dimos --simulation run unitree-go2 # quadruped +dimos --simulation run unitree-g1-sim # humanoid ``` -### Real robot (example: Unitree Go2 over WebRTC) +### Real robot + +With a Unitree Go2 on the same network, point dimOS at its IP address and drop the `--replay` flag: ```bash export ROBOT_IP= dimos run unitree-go2 ``` -Do not skip the platform guide - latency, time sync, and safety habits matter: [Unitree Go2](/docs/platforms/quadruped/go2/index.md). + +Before driving real hardware, read the [Unitree Go2 platform guide](/platforms/quadruped/go2/index). It covers network setup, latency, time sync, and the safety habits that keep you and the robot out of trouble. Do not skip it. + +### LLM agent -## Featured runfiles +The agentic blueprints add an LLM agent that controls the robot through natural language. The default agent uses OpenAI's `gpt-4o`, so you need an `OPENAI_API_KEY` in your environment before starting it. Other providers and local models are covered in the [agents guide](/capabilities/agents/index). -| Command | What it does | -| --- | --- | -| `dimos --replay run unitree-go2` | Quadruped navigation replay - SLAM, costmap, A-star planning | -| `dimos --replay --replay-db go2_bigoffice run unitree-go2-memory` | Quadruped temporal memory replay | -| `dimos --simulation run unitree-go2-agentic` | Quadruped agentic + MCP server in simulation | -| `dimos --simulation run unitree-g1-sim` | Humanoid in MuJoCo simulation | -| `dimos --replay run drone-basic` | Drone video + telemetry replay | -| `dimos --replay run drone-agentic` | Drone + LLM agent with flight skills (replay) | -| `dimos run demo-camera` | Webcam demo - no hardware needed | -| `dimos run keyboard-teleop-xarm7` | Keyboard teleop with mock xArm7 (`dimos[manipulation]` extra) | -| `dimos --simulation run unitree-go2-agentic-ollama` | Quadruped agentic with local LLM (Ollama running, `ollama serve`) | - -Blueprint reference: [Blueprints](/docs/usage/blueprints.md). - -## Agent CLI and MCP +```bash +export OPENAI_API_KEY= +dimos --simulation run unitree-go2-agentic --daemon # run in the background +dimos agent-send "explore the room" # talk to the agent +``` -The `dimos` CLI runs blueprints, inspects state, talks to agents, and invokes skills via MCP. +Every robot skill is also exposed over MCP, which means external tools and coding agents can call them directly: ```bash -dimos run unitree-go2-agentic --daemon # background -dimos status -dimos log -f -dimos agent-send "explore the room" dimos mcp list-tools dimos mcp call relative_move --arg forward=0.5 -dimos stop ``` -Full reference: [CLI](/docs/usage/cli.md). +Manage the background run with `dimos status`, `dimos log -f`, and `dimos stop`. The full command reference is in the [CLI guide](/usage/cli). + +## More blueprints to try + +| Command | What it does | +| --- | --- | +| `dimos --replay run unitree-go2` | Quadruped navigation replay with SLAM, costmap, and A-star planning | +| `dimos --replay --replay-db go2_bigoffice run unitree-go2-memory` | Quadruped spatial memory replay | +| `dimos --simulation run unitree-go2-agentic` | Quadruped LLM agent plus MCP server in simulation (needs `OPENAI_API_KEY`) | +| `dimos --simulation run unitree-g1-sim` | Humanoid in MuJoCo simulation | +| `dimos --replay run drone-basic` | Drone video and telemetry replay | +| `dimos --replay run drone-agentic` | Drone with an LLM agent and flight skills, replayed | +| `dimos run demo-camera` | Webcam demo, no robot needed | +| `dimos run keyboard-teleop-xarm7` | Keyboard teleop with a mock xArm7 (needs the `manipulation` extra) | +| `dimos --simulation run unitree-go2-agentic-ollama` | Quadruped agent using a local LLM (needs Ollama running via `ollama serve`) | + +To learn how blueprints are composed, or to write your own, see the [blueprints guide](/usage/blueprints). ## What next? - - Natural language control and MCP-exposed skills. + + Natural language control, agent configuration, and MCP-exposed skills. - - Hardware support matrix and bring-up guides. + + Hardware support and bring-up guides for the Go2 quadruped and G1 humanoid. - - Modules, streams, and blueprints behind every workflow. + + Modules, streams, and blueprints, the building blocks behind every workflow. - - Navigation, perception, spatial memory, and manipulation. + + Navigation, perception, spatial memory, and manipulation in depth. diff --git a/docs/usage/blueprints.md b/docs/usage/blueprints.md index a5e7366b47..241a0dbe12 100644 --- a/docs/usage/blueprints.md +++ b/docs/usage/blueprints.md @@ -1,4 +1,6 @@ -## Blueprints +--- +title: "Blueprints" +--- Blueprints (`BlueprintAtom`) are instructions for how to initialize a `Module`. @@ -86,7 +88,7 @@ Blueprints are frozen data classes, and `autoconnect()` always constructs an exp ## Publishing external blueprints -DimOS can discover runnable blueprints from installed Python packages. External +dimOS can discover runnable blueprints from installed Python packages. External packages declare entry points in the `dimos.blueprints` group: ```toml @@ -98,7 +100,7 @@ go2 = "my_robot_stack.go2:go2_blueprint" keyboard-teleop = "my_robot_stack.teleop:KeyboardTeleop" ``` -After the package is installed in the same Python environment as DimOS, users can run +After the package is installed in the same Python environment as dimOS, users can run those blueprints by fully qualified name: ```bash @@ -108,7 +110,7 @@ dimos run unitree-go2 my-robot-stack.keyboard-teleop External names are always `.`: -- The namespace comes from the installed distribution name. DimOS lowercases it and +- The namespace comes from the installed distribution name. dimOS lowercases it and collapses runs of `-`, `_`, and `.` into `-`, so `My_Robot.Stack` becomes `my-robot-stack`. - The local blueprint name is the entry point name. It must be lowercase kebab-case @@ -117,7 +119,7 @@ External names are always `.