Rviz Tool Cursor

An Rviz tool for ROS 2 (tested on Jazzy) that visualizes a given geometry as the cursor and orients the object normal to objects in the scene. On a click of the left mouse button, the tool will publish the 3D point and 6D pose of the object on which the cursor rests. When multiple points are clicked, the tool will display all selection points and lines connecting them and will publish a message containing all the current selection points.

The package also provides a paint brush tool for annotating scenes with graded (weighted) regions, Photoshop-style: it paints voxel maps and mesh surfaces with adjustable size, hardness, and strength, and publishes the painted region as a point cloud with per-voxel weights.

Controls

• Left mouse button: Add a selection point (triggers the publication of the point, pose, and pose array messages)
• Left mouse button hold: Adds selection points when lasso mode is enabled
• Right mouse button: Deactivates the tool
• Center mouse button: Clears the current selection points and boundary

Implementations

Tool Cursor

The base class of the tool cursor has the following properties:

• Pose Topic: The topic on which to publish the 6D pose in the Rviz environment when the left mouse button is clicked
• Point Topic: The topic on which to publish the 3D point (no orientation) in the Rviz environment when the left mouse button is clicked.
  • Note: If you want this to match the output of the Publish Point tool, you can remap /tool_cursor_point to /clicked_point. Alternatively, you can edit this in the Panels > Tool Properties menu.
• Pose Array Topic: The topic on which to publish the vector of 6D poses that have been selected in the Rviz environment when the left mouse button is clicked
• Patch Size: The number of pixels on a side with which to create a patch used for estimated the surface normal
• Lasso Mode: Toggle lasso mode where selection points are constantly acquired when the left mouse button is held down
• Close Loop: Toggle the visualization of the line connecting the last point to the first point
• Show Points: Toggle the display of the selection points
• Show Lines: Toggle the display of the lines connecting the selection points
• Point Color: Controls the color of the selection points
• Line Color: Controls the color of the selection boundary lines
• Point Size: Controls the size (in pixels) of the selection points

Mesh Tool Cursor

The mesh tool cursor visualizes a specified mesh file (.stl or .ply) as the cursor. The tool has the following additional properties:

• Mesh Filename: The filename of the mesh file to be displayed (supports package:// and file:// URIs)
• Color: The color of the cursor visualization

Circle Tool Cursor

The circle tool cursor visualizes a 2D circle of specified radius as the cursor. The tool has the following additional properties:

• Radius: The radius of the circle visualization
• Color: The color of the cursor visualization

Paint Brush Tool Cursor

A Photoshop-style paint brush for annotating 3D scenes with graded regions (see the demo animation at the top of this page).

The brush accumulates weights in [0, 1] on a voxel canvas — full strength inside the hardness radius, smooth falloff to the brush edge — and publishes the painted region as a sensor_msgs/PointCloud2 (fields x, y, z, intensity) on the Painted Cloud Topic. It paints voxel maps and mesh surfaces in the same session, on the same canvas:

• Voxel maps (Map Topic set to a PointCloud2 voxel map): weights are assigned to the map's voxels. Picking raycasts through the voxel grid (independent of the display's render style), the voxel resolution is inferred from the map's point spacing, and weights are keyed by voxel position so paint survives live map updates (e.g. a growing SLAM map).
• Meshes and markers: whatever geometry is closer under the cursor than the map receives the paint — the surface brush samples the visible surface from the depth buffer, and the sphere brush paints entire mesh surfaces through occlusion (e.g. both sides of a car) by voxelizing the scene's mesh geometry.

Visualize the output by adding a stock PointCloud2 display on the painted topic with the intensity color transformer, using a point size slightly larger than the underlying map so the overlay reads on top — but no larger than 1.5× the voxel resolution, or brush sampling can misattribute hits on the overlay itself.

Controls (in addition to the base controls):

• Left drag: paint (stamps are spaced along the stroke)
• Ctrl + left drag: erase
• Ctrl + scroll: change brush radius (plain scroll still zooms the camera)
• Shift + scroll: change hardness
• Middle click: clear all paint

Properties:

• Tool Radius / Color: as in the circle tool; the cursor is drawn as a flat circle in Surface mode and a wireframe sphere in Sphere mode, with an inner outline marking the hardness radius
• Hardness: fraction of the radius painted at full strength before the smooth falloff (Photoshop hardness)
• Strength: weight added by a single stamp at the brush center
• Stamp Spacing: minimum distance between stamps, as a fraction of the brush diameter
• Brush Shape: Surface paints only the visible/local surface; Sphere paints every solid surface intersecting the 3D radius — occupied map voxels and mesh surfaces (through occlusion, e.g. both sides of the car) — without filling empty space, and shows world-aligned X/Y/Z guide axes through the brush center for spatial reference. Sphere-erase removes any paint within the radius.
• Strict Surfaces (default on): only paint on mesh entities, the subscribed map, and existing paint, so visual aids like grid lines cannot receive floating paint. Disable it to paint on arbitrary rendered geometry (e.g. TRIANGLE_LIST markers or point clouds not subscribed as the map).
• Voxel Resolution, Infer Resolution From Map: paint canvas voxel size (disable inference for unordered clouds and set the resolution manually)
• Map Topic: voxel map to paint on (empty = paint rendered geometry only)
• Painted Cloud Topic: output topic for the painted region
• Show Guide Axes, Guide Axis Length: sphere-brush reference axes

Build

cd <workspace>
colcon build
source install/setup.bash

Demo

Everything below can be launched with one command:

colcon build --packages-select rviz_tool_cursor
source install/setup.bash
ros2 launch rviz_tool_cursor demo.launch.py    # add grow:=true (and optionally grow_speed:=0.25) to reveal the voxel map over time

This starts RViz with demo.rviz, the dummy voxel map publisher on /dummy_voxel_map, and the Roadster mesh marker on /marker. The sections below describe the pieces individually for manual setups.

Testing with a dummy mesh

The MeshToolCursor ships with a default cursor mesh (resources/default.stl, loaded via package://rviz_tool_cursor/resources/default.stl), so no extra assets are needed. The tool picks 3D points off rendered geometry, so the only other thing required is a surface in the scene to click on — the bundled car model (resources/roadster.stl) makes a good target with plenty of curved surfaces.

1. Build and launch RViz:

   colcon build --packages-select rviz_tool_cursor
   source install/setup.bash
   rviz2 -d demo.rviz

   The bundled demo.rviz config preloads all three cursor tools in the toolbar, the Marker display on /marker, the voxel map display on /dummy_voxel_map, and the painted-region overlay on /painted_region (the paint brush's Map Topic is preset to /dummy_voxel_map, so it paints the voxel map when the dummy publisher is running and free-paints otherwise). With plain rviz2, add the tools via the + toolbar button and the displays manually instead.

2. If not using the demo config, add the tool: click the + button on the RViz toolbar and select MeshToolCursor under rviz_tool_cursor.

3. Publish a dummy surface from a second terminal (source the workspace here too, so the package:// URI resolves):

   ros2 topic pub -r 1 /marker visualization_msgs/msg/Marker \
     '{header: {frame_id: map}, type: 10, mesh_resource: "package://rviz_tool_cursor/resources/roadster.stl", pose: {position: {z: 1.5}, orientation: {w: 1.0}}, scale: {x: 0.2, y: 0.2, z: 0.2}, color: {r: 0.8, g: 0.1, b: 0.1, a: 1.0}}'

   Then add a Marker display in RViz subscribed to /marker. (The car is shown at 1/5 scale and lifted 1.5 m so it floats clear of the dummy voxel map's terrain; for a plain flat surface instead, use type: 1 with scale: {x: 3.0, y: 3.0, z: 0.2}.)

4. Watch the output topics from a third terminal:

   ros2 topic echo /selection_point      # PoseStamped per click
   ros2 topic echo /tool_cursor_point    # PointStamped per click
   ros2 topic echo /selection_points     # PoseArray of all clicks

5. Exercise the tool:

   • Hover over the car: the cursor mesh snaps to the surface and orients along the surface normal.
   • Hover over empty space: the cursor falls back to the ground plane.
   • Left click (or hold, with lasso mode enabled): messages are published on all three topics and lines are drawn between selection points.
   • Middle click clears the selection; right click exits the tool.

To test with a different mesh, set the Mesh Filename property of the tool to any .stl or .ply file using a package:// or file:// URI.

Testing the paint brush on a dummy voxel map

A synthetic voxel map publisher is included for exercising the PaintBrushToolCursor map mode (hilly terrain plus a thin wall and a column, 0.1 m grid, published at 1 Hz on /dummy_voxel_map in the map frame). It is started by demo.launch.py, or standalone:

ros2 run rviz_tool_cursor publish_dummy_voxel_cloud.py
# options: --resolution 0.05 | --grow [--grow-speed 0.25]  (reveal the map over time to test paint persistence)

Run only one dummy publisher at a time — a second instance on the same topic (e.g. a manual --grow run next to the demo launch's publisher) makes the displayed map flash between the two clouds. With the demo launch, use grow:=true instead of starting another publisher.

1. Launch rviz2 -d demo.rviz — the map display, painted-region overlay, and the tool (with Map Topic preset to /dummy_voxel_map) are already configured. Without the demo config: add PointCloud2 displays on /dummy_voxel_map and on /painted_region (the latter with the Intensity color transformer and a larger point size), add the PaintBrushToolCursor tool, and set its Map Topic to /dummy_voxel_map.
2. Paint on the map, then switch Brush Shape to Sphere to paint through the wall volumetrically and see the world-aligned guide axes; relaunch with grow:=true to confirm paint survives live map updates.

Customization

In order to create a custom Rviz tool cursor, the ToolCursor base class requires the implementation of the virtual function createToolVisualization() which defines the cursor visualization:

Ogre::MovableObject* MyToolCursor::createToolVisualization()
{
  // Create and return an Ogre::MovableObject representing the cursor
}