Skip to content

feat(prompt-deploy): tag PR candidates as agentops:candidate=true#309

Merged
placerda merged 1 commit into
developfrom
feat/214-tag-pr-candidates
Jun 15, 2026
Merged

feat(prompt-deploy): tag PR candidates as agentops:candidate=true#309
placerda merged 1 commit into
developfrom
feat/214-tag-pr-candidates

Conversation

@placerda

Copy link
Copy Markdown
Contributor

Summary

Implements Option 2 (tag-at-source) from #214. When agentops.pipeline.prompt_deploy stage runs in a PR-stage context (GitHub Actions pull_request event or Azure DevOps BUILD_REASON=PullRequest), the version it creates in the dev Foundry project now carries metadata:

  • agentops:candidate=true
  • agentops:pr=<number>
  • agentops:created_at=<ISO timestamp>

Deployed-of-record runs (push to main/develop/release/** or workflow_dispatch) are not tagged; absence of agentops:candidate is the deployed-of-record signal.

Why this approach

Tag-at-source addresses #214's root cause (auditability + naive consumers) instead of the symptom (accumulation):

  • Portal viewers can filter the Versions tab on agentops:candidate to separate abandoned PR candidates from deployed versions of record.
  • Downstream consumers that resolve "latest" can refuse to pick up candidates by skipping versions with the tag. The supported pinning mechanism (foundry-agent.json) is unaffected.
  • No workflow agendado, no race against cleanup, no magic N days.

A scheduled cleanup workflow can land later as a follow-up filtering on the same tag (delete where agentops:candidate=true and pr_state in (closed,merged)) if numeric accumulation in Foundry becomes painful.

API capability validation

The Foundry SDK already accepts metadata: dict[str, str] on client.agents.create_version (the existing code at _create_agent_version passes agentops.env, agentops.prompt_sha256, agentops.git_sha through this same field). No SDK changes needed.

Acceptance criteria from #214

  • Tutorial no longer needs the "candidates can accumulate" caveat — the new contract explains the tag instead.
  • Any consumer that resolves <agent> to a candidate version by mistake gets a clear signal: the tag agentops:candidate=true.
  • "User who has run 10 PRs sees at most 1 candidate version in dev" — intentionally not addressed in this slice. Numeric accumulation is a separate concern; tag-at-source enables a follow-up cleanup workflow without coupling the two changes.

Files changed

  • src/agentops/pipeline/prompt_deploy.py — new _detect_pr_stage() helper; _deployment_metadata() augments metadata with candidate tags when in PR mode.
  • src/agentops/templates/workflows/agentops-pr-prompt-agent.yml — comment updated.
  • src/agentops/templates/pipelines/azuredevops/agentops-pr-prompt-agent.yml — comment updated.
  • docs/tutorial-prompt-agent-quickstart.md — step 13 caveat replaced with the new contract.
  • CHANGELOG.md — Unreleased entry.
  • tests/unit/test_prompt_deploy.py — 7 new tests covering the detector and the tag-at-source behavior in both PR and deployed-of-record paths.

Validation

  • python -m pytest tests/ -x -q976 passed, 1 skipped locally.
  • No infra/azd changes; azd up not required.
  • Live workflow validation (running agentops-pr-prompt-agent.yml on a real Foundry sandbox to confirm the tag appears in the portal) is not gated on this PR; the metadata path is exercised by the existing typed-create-version regression test plus the new candidate-tagging tests.

Closes #214

When `agentops.pipeline.prompt_deploy stage` runs in a PR-stage
context (GitHub Actions `pull_request` event or Azure DevOps
`BUILD_REASON=PullRequest`), the version it creates in the dev
Foundry project now carries metadata `agentops:candidate=true`,
`agentops:pr=<number>`, and `agentops:created_at=<ISO timestamp>`.

This solves the auditability and naive-consumer concerns raised in
issue #214 at the source: portal viewers can filter the Versions tab
on `agentops:candidate` to separate abandoned PR candidates from
deployed-of-record versions, and downstream consumers that resolve
"latest" can refuse to pick up candidates. Deployed-of-record runs
(push to main/develop/release/** or workflow_dispatch) are not
tagged, so absence of `agentops:candidate` is the deployed-of-record
signal.

The PR/deploy workflow templates (GitHub Actions and Azure DevOps)
and the prompt-agent quickstart tutorial are updated to describe the
new contract. A scheduled cleanup workflow can land later as a
follow-up filtering on the same tag if numeric accumulation in
Foundry becomes painful.

Closes #214

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@placerda placerda merged commit 353964e into develop Jun 15, 2026
6 of 12 checks passed
@placerda placerda deleted the feat/214-tag-pr-candidates branch June 15, 2026 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant