feat(prompt-deploy): tag PR candidates as agentops:candidate=true#309
Merged
Conversation
When `agentops.pipeline.prompt_deploy stage` runs in a PR-stage context (GitHub Actions `pull_request` event or Azure DevOps `BUILD_REASON=PullRequest`), the version it creates in the dev Foundry project now carries metadata `agentops:candidate=true`, `agentops:pr=<number>`, and `agentops:created_at=<ISO timestamp>`. This solves the auditability and naive-consumer concerns raised in issue #214 at the source: portal viewers can filter the Versions tab on `agentops:candidate` to separate abandoned PR candidates from deployed-of-record versions, and downstream consumers that resolve "latest" can refuse to pick up candidates. Deployed-of-record runs (push to main/develop/release/** or workflow_dispatch) are not tagged, so absence of `agentops:candidate` is the deployed-of-record signal. The PR/deploy workflow templates (GitHub Actions and Azure DevOps) and the prompt-agent quickstart tutorial are updated to describe the new contract. A scheduled cleanup workflow can land later as a follow-up filtering on the same tag if numeric accumulation in Foundry becomes painful. Closes #214 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements Option 2 (tag-at-source) from #214. When
agentops.pipeline.prompt_deploy stageruns in a PR-stage context (GitHub Actionspull_requestevent or Azure DevOpsBUILD_REASON=PullRequest), the version it creates in the dev Foundry project now carries metadata:agentops:candidate=trueagentops:pr=<number>agentops:created_at=<ISO timestamp>Deployed-of-record runs (push to
main/develop/release/**orworkflow_dispatch) are not tagged; absence ofagentops:candidateis the deployed-of-record signal.Why this approach
Tag-at-source addresses #214's root cause (auditability + naive consumers) instead of the symptom (accumulation):
agentops:candidateto separate abandoned PR candidates from deployed versions of record.foundry-agent.json) is unaffected.N days.A scheduled cleanup workflow can land later as a follow-up filtering on the same tag (
delete where agentops:candidate=true and pr_state in (closed,merged)) if numeric accumulation in Foundry becomes painful.API capability validation
The Foundry SDK already accepts
metadata: dict[str, str]onclient.agents.create_version(the existing code at_create_agent_versionpassesagentops.env,agentops.prompt_sha256,agentops.git_shathrough this same field). No SDK changes needed.Acceptance criteria from #214
"candidates can accumulate"caveat — the new contract explains the tag instead.<agent>to a candidate version by mistake gets a clear signal: the tagagentops:candidate=true.Files changed
src/agentops/pipeline/prompt_deploy.py— new_detect_pr_stage()helper;_deployment_metadata()augments metadata with candidate tags when in PR mode.src/agentops/templates/workflows/agentops-pr-prompt-agent.yml— comment updated.src/agentops/templates/pipelines/azuredevops/agentops-pr-prompt-agent.yml— comment updated.docs/tutorial-prompt-agent-quickstart.md— step 13 caveat replaced with the new contract.CHANGELOG.md— Unreleased entry.tests/unit/test_prompt_deploy.py— 7 new tests covering the detector and the tag-at-source behavior in both PR and deployed-of-record paths.Validation
python -m pytest tests/ -x -q— 976 passed, 1 skipped locally.azd upnot required.agentops-pr-prompt-agent.ymlon a real Foundry sandbox to confirm the tag appears in the portal) is not gated on this PR; the metadata path is exercised by the existing typed-create-version regression test plus the new candidate-tagging tests.Closes #214