Skip to content

Guidelines for workflow definition#125

Open
frazane wants to merge 2 commits intomainfrom
chore/enforce-guidelines
Open

Guidelines for workflow definition#125
frazane wants to merge 2 commits intomainfrom
chore/enforce-guidelines

Conversation

@frazane
Copy link
Copy Markdown
Contributor

@frazane frazane commented Apr 1, 2026

This PR introduces guidelines for extending the evalml workflow and enforces them. It's almost entirely renaming some rules and files so it should not break anything, but I would test anyway.

  • Rename verif.smk → verification.smk and all verif_* rules to verification_* to match the {module}_ prefix convention
  • Rename rules to follow {module}_{operation} pattern (e.g. prepare_checkpoint → inference_get_checkpoint, create_inference_venv → inference_create_venv)
  • Rename scripts to mirror their calling rules (e.g. extract_baseline.py → data_extract_baseline.py, verif_single_init.py → verification_metrics.py)
  • Remove vzarr_references.py (dead code)
  • Add a "Workflow development guidelines" section to the README documenting rule/script naming, log paths, output layout, and wildcard conventions

@frazane frazane marked this pull request as ready for review April 1, 2026 13:06
@frazane frazane requested review from Louis-Frey and dnerini April 1, 2026 13:06
@Louis-Frey
Copy link
Copy Markdown
Contributor

Thanks Francesco, good Idea! I will test the configs and let you know.

Copy link
Copy Markdown
Contributor

@Louis-Frey Louis-Frey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I tested all the configs, with the exception of forecasters-co1e.yaml (this was expected to fail), and before main was merged into this branch.

forecasters-ich1-oper-fixed.yaml produces some errors when initially run on the 2024 cases, but when I run it first on the 2025 case, and then on the 2024 cases again, it runs fine. I will take it low-prio on my list to look into that as well.

The other configs run without error. Good to go from my side.

@frazane
Copy link
Copy Markdown
Contributor Author

frazane commented Apr 7, 2026

forecasters-ich1-oper-fixed.yaml produces some errors when initially run on the 2024 cases, but when I run it first on the 2025 case, and then on the 2024 cases again, it runs fine. I will take it low-prio on my list to look into that as well.

Since this is the config that we use for the latest experiments, I would not merge this until this is fixed. What was the error?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants