refactor(toolbox): hooks over copies, one Actions pin, reproducible r…

.github/actions/toolbox/action.yml

@@ -1,21 +1,28 @@
 name: toolbox
-description: Install go-task and the 1Password CLI at the versions in katoptra/lib's toolchain.lock.toml
+description: Install go-task, and the 1Password CLI unless op is false, at the versions in katoptra/lib's toolchain.lock.toml
+inputs:
+  op:
+    description: Install the 1Password CLI; a mirror without an op.env has no use for it
+    default: 'true'
 runs:
   using: composite
   steps:
     - shell: bash
       env:
-        # GitHub checks the action's repository out beside the workflow, so the lock at
-        # the repository root is three levels up from this file.
-        LOCK: ${{ github.action_path }}/../../../toolchain.lock.toml
+        # GitHub checks the action's repository out beside the workflow, so the repository
+        # root, with the lock and the reader in docker/, is three levels up from this file.
+        LIB: ${{ github.action_path }}/../../..
+        OP: ${{ inputs.op }}
       run: |
         set -eu
-        lock() { python3 -c 'import functools,sys,tomllib; x=tomllib.load(open(sys.argv[2],"rb")); print(functools.reduce(lambda d,k: d[k], sys.argv[1].split("."), x))' "$1" "$LOCK"; }
+        lock() { python3 "$LIB/docker/lock.py" "$1" "$LIB/toolchain.lock.toml"; }
         arch="$(dpkg --print-architecture)"
         curl -fsSL "$(lock task.base_url)/$(lock task.linux_${arch}.archive)" -o /tmp/task.tgz
         echo "$(lock task.linux_${arch}.sha256)  /tmp/task.tgz" | sha256sum -c -
         sudo tar -xzf /tmp/task.tgz -C /usr/local/bin task
+        task --version
+        test "$OP" = true || exit 0
         curl -fsSL "$(lock op.base_url)/$(lock op.linux_${arch}.archive)" -o /tmp/op.zip
         echo "$(lock op.linux_${arch}.sha256)  /tmp/op.zip" | sha256sum -c -
         sudo unzip -q -o /tmp/op.zip op -d /usr/local/bin
-        task --version && op --version
+        op --version

.github/workflows/check.yml

@@ -10,7 +10,10 @@ jobs:
     timeout-minutes: 15
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-      # The action is this repository's, pinned to a release commit: a mirror whose Actions
-      # policy requires a full SHA on every uses: rejects a tag here too. Dependabot bumps it.
-      - uses: katoptra/lib/.github/actions/toolbox@a80d19d76766d058e9e10190277525f9d452b21d # v1.0.1
+      # This repository at the commit the caller pinned this workflow to, so the toolbox
+      # action and the lock it reads are the release the workflow is: one pin, not two.
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with: {repository: katoptra/lib, ref: '${{ github.job_workflow_sha }}', path: .lib}
+      - uses: ./.lib/.github/actions/toolbox
+        with: {op: 'false'}   # a dry run needs no secrets
       - run: task check

.github/workflows/sync.yml

@@ -1,7 +1,7 @@
-# Reusable: one mirror, one run. The caller is a ten-line workflow_dispatch that
+# Reusable: one mirror, one run. The caller is a workflow_dispatch of a few lines that
 # jshvn/dispatch triggers; it passes `vars` through and inherits its repository secrets.
 # 1Password is read once per run: one `op run` resolves op.env around the whole pipeline,
-# fail ping included. A mirror without an op.env runs on its repository secrets instead,
+# failure path included. A mirror without an op.env runs on its repository secrets instead,
 # exported into the sync step's environment and crossing into the container by the names
 # its PASS var lists.
 name: sync
@@ -30,26 +30,41 @@ jobs:
       OP_SERVICE_ACCOUNT_TOKEN: ${{ secrets.OP_SERVICE_ACCOUNT_TOKEN }}
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-      # The action is this repository's, pinned to a release commit: a mirror whose Actions
-      # policy requires a full SHA on every uses: rejects a tag here too. Dependabot bumps it.
-      - uses: katoptra/lib/.github/actions/toolbox@a80d19d76766d058e9e10190277525f9d452b21d # v1.0.1
+      # This repository at the commit the caller pinned this workflow to, so the toolbox
+      # action and the lock it reads are the release the workflow is: one pin, not two.
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with: {repository: katoptra/lib, ref: '${{ github.job_workflow_sha }}', path: .lib}
+      - uses: ./.lib/.github/actions/toolbox
+        with: {op: "${{ hashFiles('op.env') != '' }}"}   # a mirror without an op.env never calls op
       # df is the host's: the image and the batches share the runner's disk.
       - run: df -h . && task image
       - name: task sync
+        id: sync
         env:
           SECRETS: ${{ toJSON(secrets) }}
+          VARS: ${{ inputs.vars }}
         # Every inherited secret becomes an environment variable of this step, by name, in
         # memory: no file, no command line, and GitHub masks the values in the log. With an
-        # op.env the pipeline reads 1Password and these are unused.
+        # op.env the pipeline reads 1Password and these are unused. VARS is data the shell
+        # splits into words, never an expression spliced into the script.
         run: |
           eval "$(printf '%s' "$SECRETS" | jq -r 'to_entries[] | "export \(.key)=\(.value | @sh)"')"
-          task sync -- ${{ inputs.vars }}
+          task sync -- $VARS
+      # The pipeline's own failure path runs inside its container, so a run the runner cut
+      # off (the timeout, a cancellation, an image that never pulled) reported and pinged
+      # nothing. This is that path from outside. An ordinary failure has already run it.
+      - name: The run was cut off
+        if: always() && steps.sync.outcome != 'success' && steps.sync.outcome != 'failure'
+        env:
+          HEALTHCHECK_URL: ${{ secrets.HEALTHCHECK_URL }}
+        run: task op -- task failed
       - name: Chain the next run
         if: success()
         env:
           GH_TOKEN: ${{ github.token }}
+        # The caller's own file name, whatever it is; workflow_ref is owner/repo/.github/workflows/<file>@<ref>.
         run: |
           if [ -f .run/chain ]; then
-            gh workflow run sync.yml --ref "$GITHUB_REF_NAME"
+            f="${GITHUB_WORKFLOW_REF##*/}"; gh workflow run "${f%%@*}" --ref "$GITHUB_REF_NAME"
             echo "chained: next run queued" >> "$GITHUB_STEP_SUMMARY"
           fi

CLAUDE.md

@@ -10,14 +10,32 @@ The toolbox every katoptra mirror includes by URL. Read `README.md` for the cont
   mirror's root value.
 - Verb names are reserved across host and container. `plan` is the host-side read-only
   run; an engine's batch planner is `split`. The hooks, `prepare`, `verify`, `index`,
-  `smoke`, `report-engine` in the engine and `report-mirror` in the toolbox, are the
-  only verbs a mirror redefines, each excluded on the include that defines it.
-  `report-engine` exists in both files, so an engine consumer excludes it on the
-  toolbox include.
+  `smoke`, `smoke-mirror`, `report-engine` in the engine and `report-mirror` in the
+  toolbox, are the only verbs a mirror redefines, each excluded on the include that
+  defines it; `pipeline` and `plan-pipeline` come from the engine and a mirror excludes
+  them only for another order. `report-engine` exists in both files, so an engine
+  consumer excludes it on the toolbox include. Extension is a hook, never a copy: a
+  verb that needs more than the engine does gets a `-mirror` hook the engine calls last.
 - A mirror's root var shadows a command-line `KEY=value` inside an included verb, so an
-  engine tunable is an inline default and never a root var of the example.
-- Every tool in an image comes from `toolchain.lock.toml` with a checksum. The AWS CLI
-  zip is the marked exception.
+  engine tunable is an inline default and never a root var of the example. A var of
+  another name that reads the tunable, `BATCHES_MAX` from `MAX_BATCHES`, still sees the
+  command line (verified), so each default is spelled once.
+- A called task sees none of its caller's call vars. A hook a verb calls with an
+  overridable `RUN` or `STAGING` gets them passed on explicitly, as `smoke` does.
+- A global var that reads another var is rendered once, from the root's and the command
+  line's values, never from a call var: `GPGCHECK` takes the fingerprint as an awk `-v`
+  at each use, where a call var `TL_KEY` is visible.
+- No `dir:` on an engine verb. In a flattened include read by path, task joins even an
+  absolute `dir:` onto the include's directory (verified, 3.53.1), so `verify` starts
+  each command with `cd {{.STAGING}}` instead.
+- `render` mounts an empty directory over `.run`, so a `sh:` var read from it at parse
+  time renders the same from any working tree. A verb that reads `.run` at parse time
+  must tolerate an empty one.
+- `.taskrc.yml` keys are `trusted-hosts` and `cache-expiry`; task ignores a key it does
+  not know, silently, and refetches on every invocation.
+- Every tool in an image comes from `toolchain.lock.toml` with a checksum, read through
+  `docker/lock.py`, the one reader the Dockerfiles and the toolbox action share. The AWS
+  CLI zip is the marked exception.
 - Images set `TASK_REMOTE_OFFLINE=1`. Inside a run the include resolves from the
   mirror's `.task/remote` cache, bind-mounted with the repo, never from the network.
 - `run` and `render` mount the repository's git top level at `/work` and set the working
@@ -27,7 +45,11 @@ The toolbox every katoptra mirror includes by URL. Read `README.md` for the cont
   so the container engine's own progress lines never reach `render.txt`.
 - Inside a `sh:` var, `printf -- '-e %s'` prints dashes: task's built-in shell takes
   the `--` as the format. Use `printf '%s %s ' -e "$v"`.
-- Actions pinned to a full SHA with the version in a trailing comment.
+- Actions pinned to a full SHA with the version in a trailing comment. A mirror pins the
+  two reusable workflows that way; each checks this repository out at its own commit
+  (`github.job_workflow_sha`) for the toolbox action and the lock, so a workflow pin is
+  the one pin. The include and the image float at `v1` by design: moving that tag is
+  the rollout.
 
 ## Verifying a change
 
@@ -37,4 +59,7 @@ cd examples/proton && task image-build && task run -- task tools && task check
 ```
 
 A verb change updates the `render.txt` files via `task render-update`; `offline` is
-the engine's own check over `examples/rsync/fixtures/`.
+the engine's own check over `examples/rsync/fixtures/`: the list diff over `run-root`
+and `run-empty`, `retry`'s exit codes, and `prepare` and `verify` over `tree/`, a signed
+subtree whose tlpdb is signed by a throwaway key pinned in the example. Regenerate the
+tree with a new key only to change its shape; the private half was never kept.