feat(workflow): auto-detect committed baseline in agentops-pr templates#306
Merged
Merged
Conversation
The agentops-pr GitHub Actions and Azure DevOps templates emitted by 'agentops workflow generate' now wrap 'agentops eval run' with a small bash guard that auto-detects a committed baseline file at .agentops/baseline/results.json. When the file is present (e.g. dropped into the repo per docs/tutorial-baseline-comparison.md Section 4), the eval step compares against it via --baseline. When absent, BASELINE_ARG stays empty and the bash invocation is byte-equivalent to the prior behaviour, so existing PR pipelines do not change. Deploy templates (dev/qa/prod) are intentionally untouched. Implementation: _eval_substitutions, _github_eval_substitutions, and _ado_eval_substitutions now accept a kind kwarg and only inject the baseline-detection bash when kind == 'pr'. Two helper functions (_github_baseline_autodetect_block, _ado_baseline_autodetect_block) keep the indentation correct for both the GitHub Actions run block and the ADO AzureCLI@2 inlineScript. Tests added in tests/unit/test_cicd.py guard the PR / deploy split for both GitHub Actions and Azure DevOps. Validation: 946 unit tests pass. Closes #155 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This was referenced Jun 14, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #155.
The PR-gate workflow templates emitted by �gentops workflow generate did not pass --baseline to �gentops eval run, even though docs/tutorial-baseline-comparison.md Section 4 promises:
Both the GitHub Actions and Azure DevOps PR templates now wrap the eval invocation with a small bash guard that auto-detects
.agentops/baseline/results.json:When the file is present, the eval step compares against it. When absent,
BASELINE_ARGstays empty and the bash invocation is byte-equivalent to the prior behaviour. Deploy templates (dev/qa/prod) are intentionally untouched.Implementation
_eval_substitutions,_github_eval_substitutions,_ado_eval_substitutionsnow take akindkwarg.kind == 'pr'injects the baseline-detection bash.run: |block and the ADOAzureCLI@2inlineScript.Tests
Added to
tests/unit/test_cicd.py:test_pr_template_autodetects_committed_baselinetest_deploy_templates_do_not_inject_baseline_autodetecttest_azure_devops_pr_template_autodetects_committed_baselinetest_azure_devops_deploy_templates_do_not_inject_baseline_autodetectValidation
python -m pytest tests/unit -x -q— 946 passed, 1 skipped.Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com