[Feature] Add the DreamerV3 block RSSM core

[vmoens]

Stack from ghstack (oldest at bottom):

• [Algorithm] Add a reproducible DreamerV3 DMC Walker benchmark #4075
• [Feature] Complete DreamerV3 continuation training semantics #4074
• [BugFix] Match DreamerV3 categorical and KL semantics #4073
• -> [Feature] Add the DreamerV3 block RSSM core #4072
• [Feature] Add DreamerV3 slow-critic regularization #4068
• [Feature] Normalize DreamerV3 REINFORCE return scale #4067
• [BugFix] Decode DreamerV3 rewards and values end to end #4066
• [BugFix] Correct DreamerV3 symexp two-hot semantics #4065

Summary:

• add the grouped block-linear recurrent core used by DreamerV3
• add RMS-normalized prior and posterior predictor options
• preserve the existing GRU implementation as the default for checkpoint compatibility

Rationale:
The maintained DreamerV3 implementation used a conventional GRUCell, while the
reference architecture projects deterministic state, stochastic state, and action
separately before applying grouped recurrent gates. Providing the block core as an
explicit mode enables architecture-parity experiments without invalidating existing
TorchRL models.

Test plan:

• pytest test/modules/test_dreamer_components.py -k DreamerV3Components -q
• pytest test/objectives/test_dreamer_v3.py -q
• verify the frozen numerical fixture, normalized-action invariance, gradients,
  RMS-normalized posterior shapes, and torch.compile full-graph execution

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/4072

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 3 Unrelated Failures

As of commit a1dc57e with merge base 2a77e48 ():

NEW FAILURES - The following jobs have failed:

• Lint / lint-done (gh)
• Lint / python-source-and-configs / linux-job (gh)
  RuntimeError: Command docker exec -t 1a978f99101ca2b3889495bc4a90e2f0fdf97d170ee7f3637bf6ea70a8f5f8fc /exec failed with exit code 1

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• Unit-tests on Linux / tests-cpu (3.14, bulk) / linux-job (gh) (similar failure)
  test/test_configs.py::TestConfigClassParity::test_wrapped_class_kwargs_have_config_fields[LBFGSConfig]

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Unit-tests on Linux / tests-cpu (3.10, bulk) / linux-job (gh) (trunk failure)
  test/test_configs.py::TestConfigClassParity::test_wrapped_class_kwargs_have_config_fields[LBFGSConfig]
• Unit-tests on Linux / tests-cpu (3.13, bulk) / linux-job (gh) (trunk failure)
  test/test_configs.py::TestConfigClassParity::test_wrapped_class_kwargs_have_config_fields[LBFGSConfig]

This comment was automatically generated by Dr. CI and updates every 15 minutes.