Skip to content

Rename the predicatorv3 configs to empiric with rounds and subsets - #189

Merged
yichao-liang merged 1 commit into
masterfrom
empiric-configs
Sep 28, 2026
Merged

yichao-liang merged 1 commit into
masterfrom
empiric-configs

Conversation

@yichao-liang

Copy link
Copy Markdown
Collaborator

Summary

scripts/configs/predicatorv3/ (51 files) becomes scripts/configs/empiric/, which holds only the paper's experiment:

  • common.yaml: the shared flags, including the principled joint belief (belief_joint_draws: 16, no carried posterior).
  • envs.yaml: the five benchmark settings (Balloons composition, four-span Bridge, two-jug Boil, ramp Fan, Domino high-friction turn).
  • approaches.yaml: the seven arms (EMPIRIC, model-free, model-free + scene package, standalone, oracle dynamics, no fitting, no uncertainty).
  • benchmark.yaml: the default launcher, which includes the three, over seeds 0-4.

With --round joint_r1, benchmark.yaml resolves to the same 35 runs as continual_principled_belief_r1.yaml did: same experiment ids, approaches, envs, args and flags. The only differences are seeds 0-4 instead of 0-1, and two top-level flags that equal their defaults.

Launching:

  • A launch names its round with --round or a ROUND key, which suffixes every experiment id (<env>-<arm>_<round>). launch.py refuses a continual launch without one, since those runs auto-resume from their run folders.
  • --envs, --approaches and --seeds N-M pick a subset.
  • A launch worth keeping on record can still get its own YAML that includes benchmark.yaml and sets ROUND.
  • EXTENDS, which gave arms round-specific ids, is removed.

Deleted:

  • the phased exp_*.yaml configs, their all.yaml menus, common.yaml and oracle.yaml;
  • the dated continual_* launchers and the menu entries outside the benchmark (the scene-only, zero-shot, from-assets, real-to-sim, EMPIRIC + scene package and Sonnet entries; their classes stay);
  • scripts/engaging/relaunch_on_timeout.py, whose job the self-requeue trap now does;
  • scripts/domino_debug/.

random_actions_pybullet.yaml moves next to the ExoPredicator configs it belongs with, and generate_random_action_gifs.py now finds it.

Tests that loaded dated launchers now load the benchmark. The arms outside it keep their tests, with their flags defined in the tests. Doc links to deleted launchers point at the files under the iclr-empiric-submission tag, and code comments that pointed at the band probe name it at that tag too.

Test plan

  • yapf, isort 5.10.1, docformatter, mypy and pylint on this commit (sbatch replay of CI's static jobs).
  • The 8 CI shards in the CI container. The first run caught a recursion in the comparison test's new config helper; after the fix, test_continual_comparison_approach.py passes (41 tests) and the other shards' failures were only that helper.
  • benchmark.yaml --round joint_r1 against continual_principled_belief_r1.yaml: identical runs apart from the seeds.

🤖 Generated with Claude Code

scripts/configs/empiric/ holds only the paper's experiment: the seven
arms (approaches.yaml) on the five benchmark settings (envs.yaml) with
the shared flags and the principled joint belief (common.yaml), which
benchmark.yaml includes. With --round joint_r1 it resolves to the same
35 runs as continual_principled_belief_r1.yaml, over seeds 0-4.

A launch names its round with --round or a ROUND key, which suffixes
every experiment id, and launch.py refuses a continual launch without
one, since those runs auto-resume from their run folders. --envs,
--approaches and --seeds pick a subset. EXTENDS, which gave arms
round-specific ids, is gone.

Deleted: the phased exp_*.yaml configs and their all.yaml menus,
oracle.yaml, the dated continual launchers, the menu entries outside
the benchmark, relaunch_on_timeout.py (the self-requeue trap covers
timeouts) and scripts/domino_debug/. random_actions_pybullet.yaml moves
next to the ExoPredicator configs it belongs with.

Tests that loaded dated launchers now load the benchmark; the arms
outside it (scene package, real-to-sim, from assets, scene only, zero
shot) keep their tests with their flags defined there. Doc links to
deleted launchers point at the iclr-empiric-submission tag.
@yichao-liang
yichao-liang merged commit 325657d into master Sep 28, 2026
14 checks passed
@yichao-liang
yichao-liang deleted the empiric-configs branch September 28, 2026 09:32
yichao-liang added a commit that referenced this pull request Sep 30, 2026
Docs and comments reached the launchers and the Domino band probe that
#189 deleted through the iclr-empiric-submission tag. The tag is moving
to the refactored master, where those files no longer exist, so the 24
mentions now name commit 72d7258, the tag's current target, which
stays in master's history.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant