Allow explicit resolution order for @experiment - #270
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds two small, backward-compatible ways to control experiment selection when a script defines more than one
@experiment: anorderfield (@experiment(name=..., order=-1), default0) that lets you hint which experiment sorts to the top of the picker instead of relying on physical position in the file — sorting is ascending, so lower values appear earlier (e.g.order=-1puts an experiment above the defaultorder=0ones, at the top of the list) — with ties falling back to declaration order via a stable sort incollect_clabe_experiments; and a new--experiment-nameCLI flag that resolves an experiment by name and skips the interactive prompt entirely, unblocking non-interactive/scripted runs (raisingSystemExitwith the list of available names if the given name doesn't match). Both changes are additive with safe defaults (order=0,experiment_name=None) that preserve today's behavior exactly, touch only_experiments.pyandcli.py, and are covered by new tests for name-based selection, the not-found error path, and order-with-ties stability.