Skip to content

feat(setup): probe prek and uv in the sandbox doctor - #1362

Merged
potiuk merged 1 commit into
apache:mainfrom
potiuk:docs/sandbox-hint-followups
Sep 23, 2026
Merged

potiuk merged 1 commit into
apache:mainfrom
potiuk:docs/sandbox-hint-followups

Conversation

@potiuk

@potiuk potiuk commented Sep 23, 2026

Copy link
Copy Markdown
Member

Summary

  • The doctor skill's probes are meant to move in lock-step with docs/setup/sandbox-troubleshooting.md, but fix(agent-isolation): let prek and uv run inside the sandbox #1359 added the prek or uv not found inside the sandbox entry without a matching probe. Probe 7 (probe-7-dev-tools.sh) now covers it:
    • ✓ when prek / uv run inside the sandbox and ~/.cache is writable;
    • ✗ when they start but cannot write their cache.
    • Inside the sandbox a read-denied ~/.local/bin looks exactly like an absent one. So when neither tool is found, the probe reads the worktree's settings.local.json (readable in the sandbox) to tell a missing grant (⚠) from a tool that is not installed (⊘).
  • Follow-ups from fix(agent-isolation): let prek and uv run inside the sandbox #1359:
    • The error-hint hook and secure-agent-setup.md now count seven probes.
    • The setup doc's hint-signature table gains the command not found: prek|uv|uvx row.
    • The troubleshooting entry no longer says the new paths apply only "from the next session". Measured: the harness applies settings.local.json changes from the next sandboxed command.

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Tool / bridge contract (tools/<system>/*.md)
  • Python package (tools/*/ with pyproject.toml)
  • Groovy reference impl
  • Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
  • Documentation (docs/, README.md, CONTRIBUTING.md)
  • Project template (projects/_template/)
  • CI / dev loop (prek, workflows, validators)
  • Other:

Test plan

  • prek run --all-files, run inside the sandbox: every hook passes except the workspace pytest hook. Two suites fail there for sandbox reasons, and both pass outside it:
    • agent-isolation's test_agent_socket_rows_follow_the_connections_held_open opens a Unix socket, which the Linux sandbox refuses.
    • pr-management-stats calls the real gh.
  • Probe 7 run by hand in all four states on Linux (bubblewrap, Claude Code 2.1.280):
    • ✓: tools found, cache writable.
    • ✗: HOME pointing at an unwritable directory.
    • ⚠: tools off PATH, grant absent.
    • ⊘: tools off PATH, grant present.
  • For skill changes: setup-isolated-setup-doctor eval suite, 23/23 with magpie-run-evals.sh (claude -p).
  • For skill behaviour changes: interpret-probes grows a dev_tools_status field (skip when no dev-tools line is present, so existing fixtures expect skip) and the new fixture case-16-dev-tools-not-granted.
  • Other:

RFC-AI-0004 compliance

  • Sandbox — the probe is read-only apart from touching and removing one file in ~/.cache, which the reference config already makes writable. It adds no host access.

Linked issues

Refs #1359

Notes for reviewers (optional)

  • Probe scope: the probes still don't cover every catalogue entry. The signing-overlay entries need a live touch or a terminal, and setup-isolated-setup-verify check 10 covers them. The doctor's wording now says so instead of claiming the probes cover "the six failure modes the catalog documents".
  • Token and fingerprint regeneration: docs/mode-economics.md was regenerated with skill-token-count --write. The doctor's surface_hash was refreshed by the skill-surface-hash hook because its description now names prek and uv.

Generative AI disclosure: this change was prepared with Claude Code (Claude Opus 5) and reviewed by the author before submission.

🤖 Generated with Claude Code

The doctor skill's probes are meant to move in lock-step with the
sandbox troubleshooting catalogue, but apache#1359 added the "prek or uv not
found inside the sandbox" entry without a matching probe.

Probe 7 (probe-7-dev-tools.sh) checks that prek and uv run inside the
sandbox and can write ~/.cache. Inside the sandbox a read-denied
~/.local/bin looks exactly like an absent one, so when neither tool is
found it reads the worktree's settings.local.json to tell a missing
grant (warn) from a tool that is not installed (skip).

- The error-hint hook and the setup doc now count seven probes, and
  the doc's hint table gains the prek / uv signature from apache#1359.
- The troubleshooting entry no longer says the new paths apply only
  from the next session: the harness picks up settings.local.json
  changes on the next sandboxed command.
- The interpret-probes eval grows a dev_tools_status field and a
  case-16 fixture for the missing grant.

Generated-by: Claude Code (Opus 5)
@potiuk potiuk added family:setup setup-* skills family:tools tools/* capability:platform Framework / agent substrate skills (install, verify, doctor, override, status, setup bootstrap) labels Sep 23, 2026
@potiuk
potiuk merged commit 7c0f6cd into apache:main Sep 23, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

capability:platform Framework / agent substrate skills (install, verify, doctor, override, status, setup bootstrap) family:setup setup-* skills family:tools tools/*

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant