Repository navigation
Compile root cfg from --override values without a config.yaml - #80
Merged
Volv-G merged 1 commit intoOct 13, 2026
Conversation
A root @pipeline that loads config (a `cfg` parameter and/or propagate_config=True), declares no explicit `config=`, and receives at least one --override now compiles from an empty base config when the implicit default config.yaml is absent, exactly as if an empty config.yaml existed (raw_cfg={}, no config identity, identical compile key and output). This enables compile-time fan-out such as `for shard in cfg.shards` with `--override 'shards=[us, eu]'`. A warning names only the override keys. Still strict: missing explicit `config=`, missing default with no overrides (error now hints at --override), a nested child's missing config, and overrides on a pipeline with neither `cfg` nor propagate_config.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A root
@pipelinethat loads config can now compile from--overridevalues when the implicit defaultconfig.yamlis absent. This enables compile-time fan-out without a config file:tangle sdk pipelines compile sharded.py -o sharded.yaml --override 'shards=[us, eu]'Behavior
config=, defaultconfig.yamlmissing, ≥1--override(cfgparam and/orpropagate_config=True){}+ typed overrides; warning names override keys only--override KEY=VALUEconfig=missing (with or without overrides)cfgand nopropagate_configA missing default is modelled exactly as an empty
config.yaml(raw_cfg={}, no config identity), so the compile key and output bytes match an empty file. For apropagate_configroot the overrides become the broadcast payload; children still take only keys they declare.Tests
New tests in
tests/test_pipeline_compiler.py: list override → distinct named tasks (API + CLI), typed int →range(), missing-vs-empty key/bytes equivalence, strict cases above, propagate-only root broadcast. Fulluv run pytestpasses locally.