Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion docs/RENDERING.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ The Bridge selection behind Figure 2's act panel and Figure 1's lift illustratio
The selected Domino, Fan, Balloons and Boil runs are declared in `scripts/paper_figures/export_static_scenes.py`; `--domains` re-exports a subset.
Domino, Fan and Balloons declare their environment settings there, and Boil restores its run's settings from the launch command in `info.log`.
Boil shows the two-jug test episode of the run behind its trajectory stripe, so its final frame matches the stripe's last frame.
Balloons draws its ceiling, the height at which balloons burst, as a red cap over the chute instead of the environment's translucent plate over the table, and draws the strings of tied balloons, which the environment otherwise draws only when it renders an image.
Balloons writes its ceiling, the height at which balloons burst, as the red cap over the chute that the environment draws, and draws the strings of tied balloons, which the environment otherwise draws only when it renders an image.
Boil draws water no higher than the jug rim and redraws the spill puddle that restoring a state omits.
Figure 1 also draws the test task's cyan jug in the red that the same jug has in training, since the ambient response washes cyan out; the Boil stripe keeps the recorded cyan.
Figure 1 also colors Boil water by the heat that its recorded bubbling level fixes, as the environment does, since restoring a state resets the hidden heat and would draw boiled water blue.
Expand Down Expand Up @@ -221,6 +221,20 @@ Balloons shares Figure 1's red chute cap and balloon strings.
Boil liquid is drawn no higher than the jug rim, because the environment lets water rise above the rim before it overflows, which reads as an upturned jug.
The Bridge model frame draws the glue the model remembers as the environment's glue patches.

## Run videos in the current layouts

The harness's `run.mp4` replays a run's actions in the environment of the run's own code.
The Balloons and Fan scenes changed after their paper runs were recorded, so replaying those actions in the current environment would not reproduce the runs.
`render_run_videos.py` instead restores every recorded state, moves it into the current layout as the figures do, and renders it with the environment's PyBullet camera beside the harness video's panel.
It renders the runs behind the Balloons and Fan stripes, at the stride and frame rate of each run's own video:

```bash
PYTHONPATH=.:scripts/paper_figures python scripts/paper_figures/render_run_videos.py \
--domains Balloons Fan --out-dir logs/paper_run_videos
```

A frame takes about a second, so render on a compute node; the project page uses these videos.

## Real-world trajectory

The last row of Figure 3 shows the real-robot Fan-Domino cascade run of 2026-09-22 (`exp_20260922_134142`).
Expand Down
40 changes: 31 additions & 9 deletions predicators/envs/pybullet_balloons_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,9 @@
says what opening one does.
- The ``band``: a translucent slab crossing the box's chute,
spanning the heights the goal wants the box to float at.
- The ``ceiling``: a plate drawn over the table, at ``ceiling_z``.
Nothing in this file says what reaching it does to a balloon.
- The ``ceiling``: a red cap drawn over the box's chute at
``ceiling_z`` (a plate over the table in the hatch scene). Nothing in
this file says what reaching it does to a balloon.
"""
from typing import Any, ClassVar, Dict, FrozenSet, List, Optional, Set, Tuple

Expand Down Expand Up @@ -155,12 +156,17 @@ def get_base_sim_source_files(cls) -> List[str]:
clip_slider_color: ClassVar[Tuple[float, float, float,
float]] = (0.92, 0.92, 0.94, 1.0)

# The ceiling: a plate drawn over the table.
# The ceiling: a picture of the burst height. The chute scene draws it
# as an opaque red cap across the chute, the only column the balloons
# rise in; the hatch scene draws a translucent plate over the table.
# Both are ceiling_half_extents[2] thick, centred on ceiling_z.
ceiling_z: ClassVar[float] = table_height + 0.78
ceiling_half_extents: ClassVar[Tuple[float, float,
float]] = (0.45, 0.35, 0.004)
ceiling_color: ClassVar[Tuple[float, float, float,
float]] = (0.85, 0.85, 0.90, 0.35)
ceiling_cap_color: ClassVar[Tuple[float, float, float,
float]] = (0.80, 0.16, 0.14, 1.0)

# The chute: two fixed vertical walls flanking the box's column, a
# slot the box must rise through to reach the band. They collide ONLY
Expand Down Expand Up @@ -302,6 +308,22 @@ def obstacle_geometry(cls) -> List[Tuple[Pose3D, Pose3D]]:
return [(half, (cls.box_xy[0] + sign * (cls.chute_half_gap + half[0]),
cls.box_xy[1], z)) for sign in (-1.0, 1.0)]

@classmethod
def ceiling_geometry(
cls) -> Tuple[Pose3D, Pose3D, Tuple[float, float, float, float]]:
"""Half extents, centre and colour of the ceiling picture.

In the chute scene it is a cap spanning the chute walls; in the
hatch scene, a plate over the table.
"""
if CFG.balloons_scene == "hatch":
return (cls.ceiling_half_extents, (cls.x_mid, 1.35, cls.ceiling_z),
cls.ceiling_color)
half = (cls.chute_half_gap + 2 * cls.chute_wall_half_thickness,
cls.chute_wall_half_depth, cls.ceiling_half_extents[2])
return (half, (cls.box_xy[0], cls.box_xy[1], cls.ceiling_z),
cls.ceiling_cap_color)

@classmethod
def box_top_point(cls, state: State,
box: Object) -> Tuple[float, float, float]:
Expand Down Expand Up @@ -487,16 +509,16 @@ def initialize_pybullet(
# The ceiling is a picture, not a body: the arm swings through
# its height, and what a balloon does there is a rule of the
# concrete env, read off the balloon's height.
ceiling_visual = p.createVisualShape(
p.GEOM_BOX,
halfExtents=cls.ceiling_half_extents,
rgbaColor=cls.ceiling_color,
physicsClientId=physics_client_id)
ceiling_half, ceiling_position, ceiling_color = cls.ceiling_geometry()
ceiling_visual = p.createVisualShape(p.GEOM_BOX,
halfExtents=ceiling_half,
rgbaColor=ceiling_color,
physicsClientId=physics_client_id)
bodies["ceiling_id"] = p.createMultiBody(
baseMass=0.0,
baseCollisionShapeIndex=-1,
baseVisualShapeIndex=ceiling_visual,
basePosition=(cls.x_mid, 1.35, cls.ceiling_z),
basePosition=ceiling_position,
physicsClientId=physics_client_id)
# The chute walls: real collision bodies, but filtered below to
# collide only with the box. Centred on the box's column, one on
Expand Down
15 changes: 15 additions & 0 deletions predicators/envs/pybullet_fan.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,21 @@
from predicators.structs import Action, EnvironmentTask, GroundAtom, Object, \
Predicate, State, StepOption, TaskEvaluator, Type

# The flags that turn the default Fan domain, the paper's exposed deck and
# ramp, back into the historical uniform/maze arena. Its tasks and physics
# are unchanged; these were the defaults before September 28, 2026.
HISTORICAL_ARENA_FLAGS: Dict[str, Any] = {
"fan_exposed_transfer": False,
"fan_inertial_transfer": False,
"fan_ramp_transfer": False,
"fan_ramp_rise": 0.004,
"fan_ramp_landing_extension": 0.0,
"fan_test_num_pos_x": 6,
"fan_test_num_pos_y": 6,
"fan_train_num_walls_per_task": [1],
"fan_test_num_walls_per_task": [2, 3],
}


class FanTransferEvaluator(TaskEvaluator):
"""Require settled arrival; a ball falling off the deck ends the level."""
Expand Down
33 changes: 19 additions & 14 deletions predicators/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -1152,29 +1152,34 @@ class GlobalSettings:
fan_known_controls_relation = True
fan_combine_switch_on_off = False
fan_use_kinematic = False
# Separate pilot: a protected calibration tray and exposed L-shaped deck.
# Historical uniform/maze tasks and their physics remain unchanged.
fan_exposed_transfer = False
# Separate inertial pilot; requires exposed transfer. Preserve r1 physics.
fan_inertial_transfer = False
# Optional visible downhill ramp, with a protected training counterpart.
fan_ramp_transfer = False
# The defaults below are the paper's Fan domain (scripts/configs/empiric
# and the paper figures): a protected calibration tray for training, an
# exposed L-shaped deck with a downhill ramp for the test, no walls, and
# a 3 x 3 grid. HISTORICAL_ARENA_FLAGS in pybullet_fan.py turns the
# transfers off and restores the walls and the 6 x 6 test grid of the
# historical uniform/maze arena, whose tasks and physics are unchanged.
fan_exposed_transfer = True
# Inertial ball dynamics; requires exposed transfer.
fan_inertial_transfer = True
# A visible downhill ramp, with a protected training counterpart.
fan_ramp_transfer = True
# Visible elevation drop shared by training and test ramps, in metres.
fan_ramp_rise = 0.004
fan_ramp_rise = 0.003
# Extra exposed landing length, without moving the target or adding walls.
fan_ramp_landing_extension = 0.0
fan_ramp_landing_extension = 0.10
fan_train_num_pos_x = 3
fan_train_num_pos_y = 3
# The historical 6 x 6 uniform test split. The loc bounds in
# The historical arena's test split was 6 x 6. The loc bounds in
# pybullet_fan.py admit at most 10 x 9 cells at the 8 cm pitch, which
# fills the arena up to the fan rows; the historical maze split used
# that full grid.
fan_test_num_pos_x = 6
fan_test_num_pos_y = 6
fan_train_num_walls_per_task = [1]
fan_test_num_pos_x = 3
fan_test_num_pos_y = 3
# The historical arena used [1] for training and [2, 3] for the test.
fan_train_num_walls_per_task = [0]
# Under "maze" generation this is the number of wall cells; the walls
# are laid as straight segments (see fan_maze_max_segment_len).
fan_test_num_walls_per_task = [2, 3]
fan_test_num_walls_per_task = [0]
# How ball, target and walls are laid out, per split:
# "uniform": ball, target and walls at uniformly random cells, kept
# only if a cardinal path exists (the historical generator);
Expand Down
34 changes: 24 additions & 10 deletions scripts/paper_figures/export_static_scenes.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@
- Fan and Balloons states move into the current scene layouts; motion
relative to the platforms or chute is unchanged.
- Balloons draws its burst height as a red cap over the chute. The
environment pictures that height as a translucent plate over the whole
table, but balloons only rise with the box, inside the chute.
environment now draws the same cap in place of its old translucent
plate over the whole table; the export still writes the cap's values
and its note, so the committed scenes are unchanged.
- Boil liquid is drawn no higher than the jug rim. The environment lets
water rise above the rim before it overflows, which reads as an
upturned jug. Its spill puddle, which restoring a state omits, is
Expand All @@ -27,7 +28,7 @@
import shlex
import sys
from pathlib import Path
from typing import Any, Dict, Tuple
from typing import Any, Dict, List, Tuple
from unittest.mock import patch

import pybullet as p
Expand Down Expand Up @@ -156,21 +157,34 @@ def _migrate_balloons_layout(env: Any, state: State) -> State:


def _cap_chute(env: Any, scene: Dict[str, Any]) -> None:
"""Redraw an exported Balloons ceiling as a red cap over the chute.
"""Write the Balloons ceiling cap into an exported scene.

The cap keeps the plate's height and thickness, so its underside is
still the burst height, and spans the chute walls.
The environment draws this cap across the chute walls, with its
underside at the burst height. Writing its exact values keeps the
committed scenes, exported when the environment still drew a plate,
unchanged.
"""
assert CFG.balloons_scene == "chute"
ceiling, = (shape for shape in scene["shapes"]
if shape["body"] == env._ceiling_id) # pylint: disable=protected-access
half = (env.chute_half_gap + 2 * env.chute_wall_half_thickness,
env.chute_wall_half_depth, env.ceiling_half_extents[2])
half, position, _ = env.ceiling_geometry()
ceiling.update(dimensions=[2 * h for h in half],
position=[*env.box_xy, env.ceiling_z],
position=list(position),
rgba=list(CHUTE_CAP_RGBA))


def _current_flags(argv: List[str]) -> List[str]:
"""Drop the ``--name value`` overrides that the current code no longer
defines; nothing reads them."""
_, overrides = utils.create_arg_parser().parse_known_args(argv[1:])
kept = list(argv)
for flag, _ in zip(overrides[:-1:2], overrides[1::2]):
if flag[2:] not in CFG.__dict__:
at = kept.index(flag)
del kept[at:at + 2]
return kept


def _load_run_config(run: Path) -> None:
"""Restore the flags of a trusted local run from its launch command."""
info = re.sub(r"\x1b\[[0-9;]*m", "", (run / "info.log").read_text())
Expand All @@ -179,7 +193,7 @@ def _load_run_config(run: Path) -> None:
if "Running command:" in line)
argv = sys.argv
try:
sys.argv = shlex.split(command)[1:]
sys.argv = _current_flags(shlex.split(command)[1:])
utils.reset_config(utils.parse_args())
finally:
sys.argv = argv
Expand Down
Loading
Loading