Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/mode-economics.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ special-token spellings counted as ordinary text.
Coverage: **75 of 75 local `skills/*/SKILL.md` files**.
External `source.md` redirects and harness symlinks are excluded.

Measurement manifest SHA-256: `2886a19a0423bd16e50da8cafb0459ec5c4dabd3c4116bfad918d9d61994fe3c`.
Measurement manifest SHA-256: `9bd49f575b4d14334c0fb96b4f974da65917253eb3bd289dc158e7a05f3bc493`.

| Skill file | Measured tokens | Source SHA-256 (first 16 characters) |
|---|---:|---|
Expand Down Expand Up @@ -189,7 +189,7 @@ Measurement manifest SHA-256: `2886a19a0423bd16e50da8cafb0459ec5c4dabd3c4116bfad
| [security-model-verify](../skills/security-model-verify/SKILL.md) | 5,541 | `17170f3e573564b8` |
| [security-tracker-stats-dashboard](../skills/security-tracker-stats-dashboard/SKILL.md) | 3,815 | `8bf5804a8f214d6a` |
| [setup](../skills/setup/SKILL.md) | 4,185 | `751ad44b2bd50e11` |
| [setup-isolated-setup-doctor](../skills/setup-isolated-setup-doctor/SKILL.md) | 5,143 | `3d35982d6d18e109` |
| [setup-isolated-setup-doctor](../skills/setup-isolated-setup-doctor/SKILL.md) | 5,539 | `ba5d0773615f463d` |
| [setup-isolated-setup-install](../skills/setup-isolated-setup-install/SKILL.md) | 3,875 | `71c657ea2aaa48c4` |
| [setup-isolated-setup-update](../skills/setup-isolated-setup-update/SKILL.md) | 4,013 | `8f851263948c42a3` |
| [setup-isolated-setup-verify](../skills/setup-isolated-setup-verify/SKILL.md) | 4,691 | `fa9ec75b26714c97` |
Expand Down
73 changes: 73 additions & 0 deletions docs/setup/sandbox-troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,11 @@
- [Root cause](#root-cause-8)
- [Fix](#fix-8)
- [Notes](#notes-8)
- [Git hooks silently skipped for commits made inside the sandbox](#git-hooks-silently-skipped-for-commits-made-inside-the-sandbox)
- [Symptom](#symptom-9)
- [Root cause](#root-cause-9)
- [Fix](#fix-9)
- [Notes](#notes-9)
- [Adding a new entry](#adding-a-new-entry)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->
Expand Down Expand Up @@ -1068,6 +1073,74 @@ Confirm with `prek --version`, or with the doctor skill's

---

## Git hooks silently skipped for commits made inside the sandbox

### Symptom

Nothing, which is the problem: a `git commit` run by the agent
succeeds without `pre-commit` (and so without `prek`), `commit-msg`
or any other hook having run, and CI is the first place the skipped
checks fail. Asked directly, git inside the sandbox cannot see the
hook:

```console
$ git hook run pre-commit
error: cannot find a hook named pre-commit

$ ls ~/.claude/git-hooks
ls: cannot access '/home/<you>/.claude/git-hooks': No such file or directory
```

The same commands in a terminal find the hook.

### Root cause

Claude Code filesystem allowlist. Whole-user scope sets
`git config --global core.hooksPath ~/.claude/git-hooks`, and the
sandbox read-denies the home directory apart from the paths granted
back. Git inside the sandbox finds no hook directory, and git treats
a missing hook as "nothing to run", so it neither fails nor warns.
Per-project scope is not affected: its hooks live in the repository's
own `.git/hooks/`, which is inside the project root.

### Fix

Grant the shared hook directory, read-only, in user-scope settings,
where `core.hooksPath` itself lives:

```jsonc
// ~/.claude/settings.json
{
"sandbox": {
"filesystem": {
"allowRead": [
"~/.claude/git-hooks/" // core.hooksPath: sandboxed git runs pre-commit, commit-msg, ...
]
}
}
}
```

When the hooks are symlinks into the sync repository, add
`~/.claude-config/git-hooks/` as well: the sandbox checks the
resolved path. Confirm with `git hook run pre-commit` from the agent.

### Notes

- In the session that adds the grant, a new directory under
`~/.claude/` may stay hidden until Claude Code is restarted, even
where other grants take effect on the next command.
- The dispatcher flavour also runs
`~/.claude/scripts/sandbox-add-project-root.sh` from
`post-checkout`; `~/.claude/scripts/` is granted already for the
other hooks.
- There is no error text to match, so the error-hint hook cannot
point here. The doctor skill's *git-hooks* probe and
`setup-isolated-setup-verify` check 8 look at the directory from
inside the sandbox instead.

---

## Adding a new entry

When you hit a sandbox-shaped failure not in this list:
Expand Down
29 changes: 28 additions & 1 deletion docs/setup/secure-agent-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -1105,6 +1105,33 @@ upgrade path. The walking pass under whole-user scope is also a
one-time bulk operation — once existing checkouts are populated,
the global `post-checkout` keeps everything aligned going forward.

#### The sandbox has to read the shared hook dir

Global `core.hooksPath` points git at `~/.claude/git-hooks/`, under
the home directory the sandbox read-denies. Git run inside the
sandbox then sees no hook directory at all and skips every hook
without an error — `pre-commit` (and so `prek`), `commit-msg`,
`pre-push` alike — so an agent's sandboxed commit goes out
unchecked and CI is the first to notice. Grant the directory
read-only in user-scope settings, where the scope lives too:

```jsonc
// ~/.claude/settings.json
"sandbox": {
"filesystem": {
"allowRead": [
"~/.claude/git-hooks/"
// and "~/.claude-config/git-hooks/" when the hooks are
// symlinks into the sync repo: the sandbox checks the
// resolved path
]
}
}
```

The install skill proposes it at Step P.3-whole-user, and
`setup-isolated-setup-verify` check 8 flags it when missing.

#### Whole-user with the per-repo dispatcher

The `core.hooksPath`-shadowing trade-off above has a clean
Expand Down Expand Up @@ -1514,7 +1541,7 @@ automatically. Five classes of failure are recognised today:

The hint also tells the user to run
`/magpie-setup:isolated-setup-doctor` for a structured probe of all
seven failure modes, so a single mid-flow failure can lead to a
eight failure modes, so a single mid-flow failure can lead to a
broader sandbox health-check.

### Why install it user-scope, not project-scope
Expand Down
35 changes: 30 additions & 5 deletions plugins/magpie-setup/skills/isolated-setup-doctor/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ description: >-
Probe the secure-agent setup for restrictions that block legitimate
work — SSH agent reachability, port binding, containers, the scratch
directory, the signing key, `gh` outside the sandbox, `prek` and
`uv` inside it. Names the
`uv` inside it, the global git hook dir. Names the
troubleshooting entry and settings fix for each. Read-only.
when_to_use: >-
When the user says "doctor my sandbox", "why is the sandbox blocking
Expand All @@ -19,7 +19,7 @@ when_to_use: >-
capability:
- capability:platform
- capability:reassess
surface_hash: sha256:c68cb4cb391524f3
surface_hash: sha256:3b8655e844ac99ab
license: Apache-2.0
---

Expand Down Expand Up @@ -65,15 +65,15 @@ If a fail shows a failure mode not catalogued there, propose appending a new ent
The skill **never** edits a settings file, **never** runs a command with `dangerouslyDisableSandbox`, and **never** installs anything.
If a check fails, surface it and point at the catalog entry; do not auto-fix.
- **Run every probe, even on early failure.** Do not stop at the first ✗.
A user may have one of seven independent restrictions or all seven, and finding them one re-run at a time is annoying.
A user may have one of eight independent restrictions or all eight, and finding them one re-run at a time is annoying.
- **Distinguish ✗ (failing) from ⊘ (not applicable).** ✗ means the probe ran and the sandbox blocked it.
⊘ means the probe was skipped because a prerequisite is absent (e.g. no `docker` / `podman` on `PATH` → docker probe ⊘, not ✗).
- **Surface evidence.** Each report line names the probe command, the exit code, and the relevant stderr snippet.
"Looks blocked" is not useful; "ssh-add -l → rc=2 → `Could not open a connection to your authentication agent`" is.
- **Map each ✗ to a catalog entry.** The fail report links directly to the matching section of [`docs/setup/sandbox-troubleshooting.md`](../../../../docs/setup/sandbox-troubleshooting.md).
Do not paraphrase the remediation; the catalog is the single source of truth.

## The 7 probes
## The 8 probes

The probes cover the catalog's failure modes that a sandboxed command can detect on its own; the signing entries that need a live touch or a terminal are verified by `setup-isolated-setup-verify` check 10 instead.
New probes are added when new entries land in the catalog, so the two stay in lock-step.
Expand Down Expand Up @@ -289,11 +289,36 @@ bash <skill-dir>/scripts/probe-7-dev-tools.sh
**On ✗ or ⚠ → remediation:**
[`docs/setup/sandbox-troubleshooting.md` — `prek` or `uv` not found, or cannot write its cache, inside the sandbox](../../../../docs/setup/sandbox-troubleshooting.md#prek-or-uv-not-found-or-cannot-write-its-cache-inside-the-sandbox).

### Probe 8 — Global git hook dir readable (whole-user scope)

Tests whether git run inside the sandbox can see the hooks a global `core.hooksPath` points at.
Git treats a hook it cannot see as one that does not exist, so an unreadable hook dir costs every hook — `pre-commit` and `prek` included — without an error.
Only a look from inside the sandbox shows it, which is where this probe runs.

**Command:**

```bash
bash <skill-dir>/scripts/probe-8-git-hooks.sh
```

**Interpretation:**

| Result | Status | Meaning |
|---|---|---|
| `✓ (… readable; hooks visible: …)` | Pass | Sandboxed git runs the listed hooks. |
| `✗ (core.hooksPath … not readable inside sandbox …)` | Fail | The hook dir is hidden; every sandboxed commit skips every hook. |
| `✗ (… readable, but not the target of: …)` | Fail | The dir is visible but a hook's symlink target is not — typically hooks symlinked from a sync repo the sandbox does not grant. |
| `⚠ (… holds none of pre-commit, commit-msg, pre-push, post-checkout)` | Warn | `core.hooksPath` is set to a dir with none of the common hooks; check it is the intended one. |
| `⊘ (no global core.hooksPath …)` | Skip | Per-project scope: hooks live in each repo's `.git/hooks`, inside the project root. |

**On ✗ → remediation:**
[`docs/setup/sandbox-troubleshooting.md` — Git hooks silently skipped for commits made inside the sandbox](../../../../docs/setup/sandbox-troubleshooting.md#git-hooks-silently-skipped-for-commits-made-inside-the-sandbox).

## After the report

If every probe is ✓ or ⊘:

> All seven probes pass (or are not applicable). The sandbox is
> All eight probes pass (or are not applicable). The sandbox is
> not currently blocking the known failure modes catalogued in
> `docs/setup/sandbox-troubleshooting.md`. If you hit a different
> sandbox-shaped failure, follow the catalog's *Adding a new
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
#!/usr/bin/env bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

# Git treats a hook it cannot see as a hook that does not exist, so an
# unreadable core.hooksPath costs every hook without an error. The only
# way to notice is to look from inside the sandbox, which is where this
# probe runs.

hooks_path=$(git config --global --get core.hooksPath 2>/dev/null)
if [ -z "$hooks_path" ]; then
echo "PROBE: git-hooks → ⊘ (no global core.hooksPath; hooks live in each repo's .git/hooks)"
exit 0
fi
case "$hooks_path" in
"~/"*) hooks_path="$HOME/${hooks_path#\~/}" ;;
esac

if ! ls "$hooks_path" >/dev/null 2>&1; then
echo "PROBE: git-hooks → ✗ (core.hooksPath $hooks_path not readable inside sandbox; sandboxed git skips every hook)"
exit 0
fi

visible=""
unreadable=""
for hook in pre-commit commit-msg pre-push post-checkout; do
[ -e "$hooks_path/$hook" ] || [ -L "$hooks_path/$hook" ] || continue
if [ -r "$hooks_path/$hook" ] && [ -r "$(readlink -f "$hooks_path/$hook" 2>/dev/null)" ]; then
visible="$visible $hook"
else
unreadable="$unreadable $hook"
fi
done
visible="${visible# }"
unreadable="${unreadable# }"

if [ -n "$unreadable" ]; then
echo "PROBE: git-hooks → ✗ ($hooks_path readable, but not the target of: $unreadable)"
elif [ -z "$visible" ]; then
echo "PROBE: git-hooks → ⚠ ($hooks_path readable but holds none of pre-commit, commit-msg, pre-push, post-checkout)"
else
echo "PROBE: git-hooks → ✓ ($hooks_path readable; hooks visible: $visible)"
fi
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,20 @@ standalone `git-global-post-checkout.sh`. Still run step 2 (set
The operator must migrate any per-repo hooks they want to keep.
`git config --global --unset core.hooksPath` is the reversal.

4. **Let the sandbox read the shared hook dir.** Propose adding
`~/.claude/git-hooks/` to `sandbox.filesystem.allowRead` in the
operator's **user-scope** `~/.claude/settings.json` (merge diff
and ask, as for the other user-scope grants), plus
`~/.claude-config/git-hooks/` when the hooks are symlinked from
the sync repo — the sandbox checks the resolved path.
Without it the sandbox hides the directory, and git run by the
agent finds **no hook at all**: every sandboxed `git commit`
skips `pre-commit` (prek included), `commit-msg` and the rest
without a word, and CI is the first place the skipped checks
fail. The grant is read-only; the hooks already run outside the
sandbox. Rationale:
[`docs/setup/sandbox-troubleshooting.md` → Git hooks silently skipped for commits made inside the sandbox](../../../../docs/setup/sandbox-troubleshooting.md#git-hooks-silently-skipped-for-commits-made-inside-the-sandbox).

After this step, future `git clone`, `git worktree add`, and
`git checkout` operations anywhere on the host invoke the
framework's universal post-checkout, which keeps each
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,15 @@ reading this file; read the section for a check whose condition holds.
or `setup-isolated-setup-update` to refresh the script copy.
- ⚠ if the script drifted from its framework source-of-truth —
surface the diff, propose `setup-isolated-setup-update`.
- ✗ if the hook dir is not readable **from a sandboxed Bash**:
`git hook run --ignore-missing post-checkout` succeeds either
way, so probe with `test -r ~/.claude/git-hooks/post-checkout`
and `test -r "$(readlink -f ~/.claude/git-hooks/post-checkout)"`
run inside the sandbox. Unreadable means every git command the
agent runs skips every hook silently — `pre-commit` and `prek`
included. Remediation: the user-scope `allowRead` grant in
[`docs/setup/sandbox-troubleshooting.md` → Git hooks silently skipped for commits made inside the sandbox](../../../../docs/setup/sandbox-troubleshooting.md#git-hooks-silently-skipped-for-commits-made-inside-the-sandbox),
applied by the user.
- **Loud reminder** (every run, not a ✗), by flavour:
- *Simple:* surface a one-line note that per-repo
`.git/hooks/*` are inert across the host (per [`docs/setup/secure-agent-setup.md` → *Per-project vs whole-user scope*](../../../../docs/setup/secure-agent-setup.md#per-project-vs-whole-user-scope)).
Expand Down
2 changes: 1 addition & 1 deletion tools/agent-isolation/sandbox-error-hint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,6 @@ yellow="${esc}[1;33m"
reset="${esc}[0m"

printf '%s[sandbox-hint]%s %s\n' "$yellow" "$reset" "$hint" >&2
printf '%s %s Run %s/setup-isolated-setup-doctor%s for a structured probe of all seven failure modes.\n' "$yellow" "$reset" "${esc}[1m" "${esc}[0m" >&2
printf '%s %s Run %s/setup-isolated-setup-doctor%s for a structured probe of all eight failure modes.\n' "$yellow" "$reset" "${esc}[1m" "${esc}[0m" >&2

exit 1
4 changes: 2 additions & 2 deletions tools/skill-evals/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ Suites are currently implemented for:
- **pr-management-stats** — 13 cases across 2 steps (classify, pressure-weight)
- **pr-management-triage** — 51 cases across 5 steps (pre-filter, decision-table, terminal-links, pagination-dedup, interaction-progress)
- **list-skills** — 8 cases across 2 steps (step-1-command, step-2-present)
- **setup-isolated-setup-verify** — 15 cases across 3 steps (runtime-routing, step-1-classify, step-2-recommend)
- **setup-isolated-setup-verify** — 16 cases across 3 steps (runtime-routing, step-1-classify, step-2-recommend)
- **setup-isolated-setup-update** — 15 cases across 4 steps (runtime-routing, step-snapshot-drift, step-tool-freshness, step-after-report)
- **setup-isolated-setup-doctor** — 23 cases across 3 steps (runtime-routing, interpret-probes, after-report)
- **setup-isolated-setup-doctor** — 24 cases across 3 steps (runtime-routing, interpret-probes, after-report)
- **contributor-activity-sweep** — 12 cases across 3 steps (step-0-resolve-inputs, step-1-classify-reviews, step-2-render)
- **optimize-skill** — 5 cases across 1 step (step-diagnose)
- **committer-onboarding** — 27 cases across 4 steps (step-0-validate-vote, step-1-icla-comms, step-2-checklist, step-3-completion-summary)
Expand Down
8 changes: 6 additions & 2 deletions tools/skill-evals/evals/setup-isolated-setup-doctor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@

Behavioral evals for the `setup-isolated-setup-doctor` skill.

## Suites (23 cases total)
## Suites (24 cases total)

| Suite | Step | Cases | What it covers |
|---|---|---|---|
| `runtime-routing` | Runtime routing | 2 | Codex and Gemini route to their native adapters and never require Claude files |
| `interpret-probes` | Probe interpretation (`## The 7 probes`) | 16 | all-pass, ssh-fail, localhost-fail, docker-skipped, multiple-fail, ssh-skipped-no-env, injection-in-probe-output, signing-key-fail, gh-sandbox-fail, container-gateway pass/not-running/socket-denied/no-backend/relative-CONTAINER_HOST, scratch-on-shared-session-root, dev-tools-not-granted |
| `interpret-probes` | Probe interpretation (`## The 8 probes`) | 17 | all-pass, ssh-fail, localhost-fail, docker-skipped, multiple-fail, ssh-skipped-no-env, injection-in-probe-output, signing-key-fail, gh-sandbox-fail, container-gateway pass/not-running/socket-denied/no-backend/relative-CONTAINER_HOST, scratch-on-shared-session-root, dev-tools-not-granted, git-hooks-unreadable |
| `after-report` | Report synthesis (`## After the report`) | 5 | all-clear-all-pass, all-clear-with-skips, ssh-fail-with-catalog-link, multiple-fail-two-catalog-links, injection-asks-autofix-rejected |

## Run
Expand Down Expand Up @@ -89,6 +89,10 @@ The fifteen cases span:
`~/.local/bin` is missing from the worktree's `settings.local.json`,
so `prek` and `uv` are hidden; every other probe passes or is
skipped. Expected `dev_tools_status: "warn"`, `has_failures: false`.
- **case-17-git-hooks-unreadable**: `git-hooks` ✗ because the global
`core.hooksPath` dir is hidden inside the sandbox, so sandboxed git
skips every hook. Expected `git_hooks_status: "fail"`,
`has_failures: true`.

### after-report

Expand Down
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"ssh_status": "pass", "localhost_status": "pass", "docker_status": "pass", "scratch_status": "skip", "signing_key_status": "skip", "gh_sandbox_status": "skip", "dev_tools_status": "skip", "has_failures": false}
{"ssh_status": "pass", "localhost_status": "pass", "docker_status": "pass", "scratch_status": "skip", "signing_key_status": "skip", "gh_sandbox_status": "skip", "dev_tools_status": "skip", "git_hooks_status": "skip", "has_failures": false}
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"ssh_status": "pass", "localhost_status": "pass", "docker_status": "pass", "scratch_status": "skip", "signing_key_status": "skip", "gh_sandbox_status": "skip", "dev_tools_status": "skip", "has_failures": false}
{"ssh_status": "pass", "localhost_status": "pass", "docker_status": "pass", "scratch_status": "skip", "signing_key_status": "skip", "gh_sandbox_status": "skip", "dev_tools_status": "skip", "git_hooks_status": "skip", "has_failures": false}
Loading