Skip to content

Add coupled (FV3JEDI + SOCA) JCB templates - #2187

Open
shlyaeva wants to merge 11 commits into
developfrom
feature/jcb-coupled-templates
Open

Add coupled (FV3JEDI + SOCA) JCB templates #2187
shlyaeva wants to merge 11 commits into
developfrom
feature/jcb-coupled-templates

Conversation

@shlyaeva

@shlyaeva shlyaeva commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Description

Assemble coupled atmosphere + ocean JEDI YAMLs from the existing single-component JCB fragments, rather than maintaining hand-written coupled YAMLs. Coupled hofx3d and 3dvar are supported.

New coupled algorithm wrappers (parm/jcb-gdas/algorithm/coupled/) loop over a coupled_components list and, for each component, emit its YAML key and include that component's existing model fragment unchanged:

  • coupled_geometry_background.yaml.j2 - per-component geometry, plus the FV3JEDI exclude variables list.
  • coupled_background.yaml.j2 - per-component background/state (used for the variational background and the hofx state).
  • coupled_background_error.yaml.j2 - a Coupled Block Diagonal covariance with one block per component.
  • coupled_3dvar_outer_loop_1.yaml.j2 - a single shared-geometry outer loop.

These wrappers live in algorithm/coupled/ (not model/) so they are exempt from the model-directory naming/variable constraints, and they reuse the stock jcb-algorithms hofx3d/3dvar templates via section-file overrides and model_component: coupled_.

New client integration config gdas-coupled-templates.yaml exercises both algorithms with one atmosphere observer (conventional_ps) and one marine observer (adt_rads_all). It uses the renderer's app_paths_model_extra / app_paths_observations_extra hooks to span both components.

Requires the matching jcb renderer support (NOAA-EMC/jcb#45).

Companion PRs

NOAA-EMC/jcb#46

Automated CI tests to run in Global Workflow

  • atm_jjob
  • C96C48_ufs_hybatmDA
  • C96C48_hybatmsnowDA
  • C96_gcafs_cycled
  • C48mx500_3DVarAOWCDA
  • C48mx500_hybAOWCDA
  • C96C48_ufsgsi_hybatmDA
  • C48_ufsenkf_atmDA
  • C96C48_hybatmDA

shlyaeva and others added 2 commits June 30, 2026 06:38
Assemble coupled atmosphere + ocean JEDI YAMLs from the existing
single-component JCB fragments, rather than maintaining hand-written coupled
YAMLs. Coupled hofx3d and 3dvar are supported.

New coupled algorithm wrappers (parm/jcb-gdas/algorithm/coupled/) loop over a
coupled_components list and, for each component, emit its YAML key and include
that component's existing model fragment unchanged:

- coupled_geometry_background.yaml.j2 - per-component geometry, plus the
  FV3JEDI exclude variables list.
- coupled_background.yaml.j2 - per-component background/state (used for the
  variational background and the hofx state).
- coupled_background_error.yaml.j2 - a Coupled Block Diagonal covariance with
  one block per component.
- coupled_3dvar_outer_loop_1.yaml.j2 - a single shared-geometry outer loop.

These wrappers live in algorithm/coupled/ (not model/) so they are exempt from
the model-directory naming/variable constraints, and they reuse the stock
jcb-algorithms hofx3d/3dvar templates via section-file overrides and
model_component: coupled_.

New client integration config gdas-coupled-templates.yaml exercises both
algorithms with one atmosphere observer (conventional_ps) and one marine
observer (adt_rads_all). It uses the renderer's app_paths_model_extra /
app_paths_observations_extra hooks to span both components.

Requires the matching jcb renderer support (NOAA-EMC/jcb); the sorc/jcb
submodule pointer will be bumped once that change merges upstream.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@DavidNew-NOAA DavidNew-NOAA left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these templates should be in the model directory, not algorithm

@DavidNew-NOAA

Copy link
Copy Markdown
Collaborator

Sorry, disregard my previous comment, I don't read PR descriptions. Let me study more carefully.

@DavidNew-NOAA

DavidNew-NOAA commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

I'm still hung up on the model templates being in the algorithm directory. If there are model-directory naming/variable constraints, then we should find a way to adapt or relax them in JCB for something like the coupled case.

@shlyaeva

Copy link
Copy Markdown
Collaborator Author

@DavidNew-NOAA do you have a suggestion? I don't know enough about jcb to make an intelligent choice, but happy to change (have claude change) this. Let me know if it's easier to discuss on a call.

@DavidNew-NOAA

Copy link
Copy Markdown
Collaborator

@shlyaeva I don't have an immediate suggestion, because I don't know exactly how JCB enforces these constraints, but I can look into it.

@DavidNew-NOAA

Copy link
Copy Markdown
Collaborator

@shlyaeva I think you could take this line:

https://github.com/NOAA-EMC/jcb/blob/1a777fdfd99fe69dc64627f7e6a83c851eb61e53/test/client_integration/test_client_requirements.py#L95

And change it to something like

assert key.split('_')[0] in app_model_components

to enforce that the keys are prefixed with valid model component names but not necessarily one component's name

Relocate the coupled wrapper fragments from algorithm/coupled/ to their own
model component directory, model/coupled/, alongside the other model
components, addressing review feedback that model templates should live under
model/ rather than algorithm/.

No change to JCB's client requirement tests is needed: the wrappers only use
coupled_ template keys and coupled_ filenames, so they satisfy the existing
per-component model-directory rules with 'coupled' treated as the component.

The client config now uses app_path_model: gdas/model/coupled (from which the
renderer derives model_component: coupled_) plus app_paths_model_extra for the
atmosphere and marine single-component directories whose fragments are
included. app_path_algorithm is no longer needed. Rendered hofx3d and 3dvar
output is byte-identical to before the move, and the full jcb test suite
passes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shlyaeva

Copy link
Copy Markdown
Collaborator Author

@DavidNew-NOAA I've moved them to model/coupled; what do you think of the current version?

@DavidNew-NOAA

Copy link
Copy Markdown
Collaborator

@shlyaeva So I see you didn't need to make any changes to JCB like I suggested and the client integration testing still passed?

DavidNew-NOAA
DavidNew-NOAA previously approved these changes Jun 30, 2026

@DavidNew-NOAA DavidNew-NOAA left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No further objections

@shlyaeva

shlyaeva commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator Author

@DavidNew-NOAA yes! great, thank you!

@shlyaeva

shlyaeva commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator Author

I've merged jcb PR, and updated the jcb hash here. @DavidNew-NOAA do you mind re-reviewing and @AndrewEichmann-NOAA or @guillaumevernieres do you mind providing a second review?

DavidNew-NOAA
DavidNew-NOAA previously approved these changes Jul 6, 2026

@AndrewEichmann-NOAA AndrewEichmann-NOAA left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No complaints, but I'm not familiar with what to expect

@shlyaeva shlyaeva self-assigned this Jul 6, 2026
@shlyaeva

Copy link
Copy Markdown
Collaborator Author

@DavidNew-NOAA I did need an additional change in jcb for the tests to pass. I tagged you on a PR there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants