Skip to content

opencode_env: don't retry the install when version resolution fails - #1074

Open
jayzuccarelli wants to merge 14 commits into
huggingface:mainfrom
jayzuccarelli:opencode-install-fatal-version-failure
Open

jayzuccarelli wants to merge 14 commits into
huggingface:mainfrom
jayzuccarelli:opencode-install-fatal-version-failure

Conversation

@jayzuccarelli

@jayzuccarelli jayzuccarelli commented Aug 10, 2026

Copy link
Copy Markdown

Summary

_bootstrap_sandbox installs opencode per sandbox, and with the default opencode_version="latest" the upstream installer resolves the version through api.github.com (60 req/hr unauthenticated). When that lookup fails it prints Failed to fetch version information on stdout and exits 1 with an empty stderr — exactly the signature _exec_with_retry classifies as transient. So every rate-limited bootstrap burns 3 API calls instead of 1, and the final RuntimeError buries the actual cause in stdout_tail.

This PR takes the retry-amplification half of #1072:

  • _exec_with_retry gains a fatal_markers escape hatch: a failure whose stdout matches a marker is deterministic and not retried, even with empty stderr.
  • The install call site marks the installer's version-resolution failure as fatal and re-raises with an actionable message: pin opencode_version to a release tag (the pinned branch of the installer resolves via a plain curl -sI to github.com, no API quota involved).

The larger direction — prebaking the binary into the template or moving the default off "latest" — is a design call better made on the issue.

Test plan

Red-green: the three new tests in tests/envs/test_opencode_factory_lifecycle.py fail on main (install retries all 3 attempts, no actionable message) and pass with the fix. Full suite: PYTHONPATH=src:envs pytest tests/ → 1532 passed, 133 skipped; the one failure (test_push_count_deploys_multiple_spaces) reproduces identically on clean main.


Note

Low Risk
Bootstrap and install-command behavior only; clearer errors and fewer wasted retries, with no auth or data-path changes.

Overview
Stops retry amplification when the OpenCode installer fails to resolve latest via the GitHub API (rate limit): that failure lands on stdout with empty stderr, which used to look “transient” and trigger three install attempts (and up to three sandbox creates).

_exec_with_retry now accepts fatal_markers on stdout so deterministic failures are not retried. The install step marks the installer’s Failed to fetch version information message as fatal and surfaces _NonRetryableBootstrapError with guidance to pin opencode_version; create() does not retry that class of bootstrap error.

build_install_cmd passes a pinned version as curl | bash -s -- --version <tag> instead of an OPENCODE_VERSION prefix on curl, so the pin actually reaches the installer and avoids the API lookup when pinned.

Tests cover fatal vs transient retry behavior, bootstrap/create retry counts, and install command shape.

Reviewed by Cursor Bugbot for commit 5e0c014. Bugbot is set up for automated code reviews on this repo. Configure here.

The upstream installer resolves opencode_version='latest' through
api.github.com (60 req/hr unauthenticated) and reports a failed lookup on
stdout with an empty stderr — the exact signature _exec_with_retry treats
as transient. Each sandbox bootstrap then burns 3 API calls instead of 1,
and the raised error hides the cause in stdout_tail.

Teach _exec_with_retry a fatal_markers escape hatch, mark the installer's
version-resolution failure as fatal at the install call site, and raise an
actionable error pointing at pinning opencode_version (the pinned branch
of the installer never touches the API).

Addresses the retry-amplification half of huggingface#1072; avoiding the API lookup
entirely (prebaked template / pinned default) is left as a design call on
the issue.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread envs/opencode_env/harness.py
OPENCODE_VERSION= prefixed onto curl never reaches the bash side of the
curl | bash pipeline, and the upstream installer reads VERSION / --version
anyway — so pinning silently tracked latest. Pass --version via
bash -s -- instead, per the installer's own usage text.
@jayzuccarelli

Copy link
Copy Markdown
Author

@sergiopaniego you've been shipping the recent opencode_env changes, so pinging you here rather than the whole team. This one makes _bootstrap_sandbox stop retrying the install when the upstream installer's version lookup fails (the retry never helps, the GitHub API rate limit is per hour), and fixes the OPENCODE_VERSION pin so it actually reaches the installer's bash side. The cursor bot's point about the pin is addressed in the second commit. Small diff, CI green. Anything you'd like changed?

@cursor
cursor Bot requested a review from burtenshaw September 9, 2026 13:04

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Reviewed exact head 3b122d74dbc232e5ac456857e7b16164e497bbcd against current main f8433a56f2fde5a6f20765b491b8365fc123f7fa. The fatal version-lookup classification and bash -s -- --version propagation are correct; the prior Bugbot concern is resolved, all 9 focused tests pass locally, the changed scope is lint-clean, and the merge-tree is conflict-free. No Tier-1 or alignment defect remains.

This is not merge-ready yet because the standard workflows never executed (action_required at this exact head), and the branch is behind current main. A maintainer must approve the external-branch workflows, refresh the branch, and require all exact-head checks to pass. burtenshaw is already requested; no code change is otherwise required.

View PR

Open in Web View Automation 

Sent by Cursor Automation: Release

@jayzuccarelli

Copy link
Copy Markdown
Author

@burtenshaw you're the requested reviewer here, so pinging you directly. Small change to _bootstrap_sandbox: stop retrying the install when the upstream installer's version lookup fails, and pass the pin through --version so it actually applies. Bugbot has reviewed the current head clean; it just needs the workflows approved.

@burtenshaw

Copy link
Copy Markdown
Collaborator

cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread envs/opencode_env/harness.py

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at 9dafe91b

Two solid env-only fixes with good coverage:

  1. Version pin now uses bash -s -- --version x.y.z instead of an env-var prefix on curl | bash (which never reached the installer).
  2. fatal_markers stops retries on deterministic stdout failures (e.g. GitHub rate-limit); actionable path works because stdout_tail is embedded in the raised RuntimeError.

Focused lifecycle tests pass locally. Not 0.6.0 wheel cargo (envs/opencode_env only). LGTM.

View PR

Open in Web View Automation 

Sent by Cursor Automation: Release

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES at 9dafe91b — withdrawing earlier APPROVE

The _exec_with_retry(..., fatal_markers=…) stop is real, but OpenCodeSessionFactory.create() still wraps _create_once in a broad except Exception and retries with a new sandbox (harness.py ~248–261). The actionable rate-limit RuntimeError from _bootstrap_sandbox is therefore still retried up to _create_attempts times — amplifying GitHub API hits and paid sandbox churn.

Existing tests call _bootstrap_sandbox directly and miss this.

Fix: classify non-retryable bootstrap failures (typed exception, or re-raise a sentinel that create() does not catch) and add a create()-level regression that asserts one sandbox + no outer retry on the version-fetch fatal marker.

Install-cmd --version pin and fatal_markers themselves still look correct. Env-only; not 0.6.0 wheel cargo.

View PR

Open in Web View Automation 

Sent by Cursor Automation: Release

@burtenshaw

Copy link
Copy Markdown
Collaborator

cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at eba7bd78

The prior outer-loop blocker is fixed. _NonRetryableBootstrapError now bypasses create() retries, and test_create_retries_only_transient_install_failures proves fatal version-lookup provisions one sandbox while transient install failures still get three.

Focused lifecycle suite: 11/11 pass. Residual non-blocking nits (do not gate merge):

  • _exec_with_retry still says “failed after N attempts” when a fatal marker stops after one
  • pinned opencode_version is still shell-interpolated unquoted
  • pre-existing ruff format drift in harness.py (unchanged by this commit)

Environment-only; not 0.6.0 wheel cargo. Still needs fork Approve-and-run / exact-head repository CI before merge.

View PR

Open in Web View Automation 

Sent by Cursor Automation: Release

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit eba7bd7. Configure here.

@jayzuccarelli

Copy link
Copy Markdown
Author

@burtenshaw thanks for pushing that straight to the branch. It's the same shape I had staged locally, and your parametrized test covers the transient side better than mine would have.

Independently verified eba7bd78 on a clean clone: test_create_retries_only_transient_install_failures is red on the parent (assert 3 == 1, three sandboxes provisioned) and green at this head, 11/11 in the lifecycle file, ruff clean on both changed files. Bugbot re-approved at this head as well.

No further changes from my side. The two leftover Bugbot nits are cosmetic (the "failed after N attempts" message when a fatal marker stops at one, and the unquoted pinned version) and can be a follow-up if they're worth having. This just needs approve-and-run on the fork workflows.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at 20594e6a (merge-main only)

Rebased onto main after #1178. Feature tip unchanged from prior APPROVE at eba7bd78 (outer create() re-raises _NonRetryableBootstrapError for fatal version-lookup; 1 vs 3 sandbox attempts). Environment-only; not Thursday wheel cargo.

Still needs exact-head repository CI / Approve-and-run on the fork.

View PR

Open in Web View Automation 

Sent by Cursor Automation: Release

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at 2d1f2969 (merge-main only)

Rebased onto main after #1219 docs merge. Feature fix unchanged: _NonRetryableBootstrapError still stops fatal install retries across new sandboxes.

Env-only; not 0.6.0 wheel cargo. Fork may still need Approve-and-run for exact-head CI.

View PR

Open in Web View Automation 

Sent by Cursor Automation: Release

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge-main re-APPROVE at 4d6cf7af

Ben merged main into this fork tip. Feature diff unchanged: _NonRetryableBootstrapError still stops outer create() retries on fatal version-lookup, with test_create_retries_only_transient_install_failures coverage.

Fork PR: repository CI still needs Approve-and-run. Not OpenEnv wheel cargo (env-only).

View PR

Open in Web View Automation 

Sent by Cursor Automation: Release

@jayzuccarelli

Copy link
Copy Markdown
Author

@burtenshaw the branch is current with main again at 4d6cf7af, but the four repo workflows (Tests, Package CI, Discovery catalog, Build PR Documentation) are sitting at action_required on it, as they have on every head since eba7bd78: each main sync spawns a fresh unapproved batch and voids the previous one.

One approve-and-run on the current head should be all this needs. Bugbot is green and the branch is mergeable.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-APPROVE at f4cca066 (merge-main only)

Ben merged main (26c9465e / #1179) into this fork PR. Stable feature patch-id is unchanged vs prior approved tip 4d6cf7af (acce4fcd…); harness / runtime / lifecycle-test blobs are identical.

Prior technical approval stands. Still needs maintainer Approve and run for repository CI. Env-only — not 0.6.0 wheel cargo.

View PR

Open in Web View Automation 

Sent by Cursor Automation: Release

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants