release-train: staging -> main

.publish-forbidden

@@ -0,0 +1,79 @@
+# .publish-forbidden — what must never reach the public mirror, even if the
+# allowlist (.publish-include) let it through by mistake.
+#
+# Read by scripts/publish-guard.sh. Four sections; `#` starts a comment. A
+# header the guard does not know, a needle listed in both string tiers, or an
+# empty [strings-refuse] is refused as "could not tell" (exit 2).
+#
+#   [paths]           gitignore-style names. A pattern containing `/` is
+#                     anchored to the staged root; one without matches ANY path
+#                     component; a trailing `/` means "as a directory".
+#   [strings-refuse]  extended regexes, matched case-insensitively against
+#                     every staged TEXT file. A hit REFUSES the publish and
+#                     names the file and line (never the matched text).
+#   [strings-report]  same syntax. Hits are COUNTED and printed (per-needle
+#                     totals, ten most-hit files) but do not refuse — unless
+#                     the guard runs with --strict, which promotes this tier
+#                     to refusal. A needle moves up to [strings-refuse] the
+#                     day it is decided the mirror must never carry it.
+#   [allow]           exact tokens removed from a line before a needle is
+#                     re-tested, so a line is spared only when the allowed
+#                     token was the whole reason it hit.
+#
+# This file is ONE list read by both the guard and its tests; the tests write
+# their own inputs and never iterate this file to check itself.
+
+[paths]
+tests/
+scripts/tests/
+ci/
+.github/
+docs/rfcs/
+docs/migration-tools/
+CLAUDE.md
+STYLE.md
+Makefile
+.cursor/
+*.go
+go.mod
+go.sum
+__pycache__
+.DS_Store
+.env*
+*.pem
+*.key
+kubeconfig*
+
+[strings-refuse]
+# Mailboxes (the public support address is spared under [allow]).
+[A-Za-z0-9._%+-]+@tracebloc\.io
+# AWS account identifiers and ARNs.
+arn:aws:
+[0-9]{12}\.dkr\.ecr\.
+#
+# CUSTOMER AND TENANT IDENTIFIERS ARE DELIBERATELY NOT LISTED HERE. This file
+# is public, and a list of customer names would itself be the disclosure the
+# scan exists to prevent. Those needles are supplied privately at publish time:
+# the workflow writes the PUBLISH_FORBIDDEN_TENANTS secret (one needle per line,
+# same regex syntax) to a file and passes it as --extra-forbidden; they join
+# this tier. The guard refuses to run the scan when that list is missing or
+# empty.
+
+[strings-report]
+# Internal tracker and RFC identifiers — a reader of the mirror cannot open
+# them. Counted until the decision to strip them from the deliverable (or to
+# accept them) is taken; --strict refuses them.
+backend#
+rfcs#
+RFC-0
+RFC-BACKEND
+e2e-test-agent#
+tracebloc/backend
+# Non-production tracebloc hosts; same decision pending.
+dev-api\.tracebloc\.io
+stg-api\.tracebloc\.io
+dev\.tracebloc\.io
+stg\.tracebloc\.io
+
+[allow]
+support@tracebloc\.io

.publish-include

@@ -0,0 +1,29 @@
+# .publish-include — what the public mirror of this repo MAY carry.
+#
+# Read by scripts/publish-guard.sh. One glob per line; `#` starts a comment.
+# `*` and `?` do not cross `/`, `**` does; a leading `!` takes matching files
+# back out. Only tracked files are considered. Anything not matched here is
+# excluded by construction — .publish-forbidden is the second lock, and it
+# refuses the excluded directories below even if the `!` lines were deleted.
+
+# The two Helm charts that `helm repo add tracebloc https://tracebloc.github.io/client`
+# serves. Their unit-test suites and CI values never ship (see .helmignore).
+client/**
+!client/tests/**
+!client/ci/**
+ingestor/**
+
+# The installer: the two bootstraps plus every file they fetch and verify
+# against scripts/manifest.sha256 (the sub-scripts live under scripts/lib/).
+scripts/install.sh
+scripts/install.ps1
+scripts/install-k8s.sh
+scripts/install-k8s.ps1
+scripts/lib/**
+scripts/manifest.sha256
+
+# Front matter, and the operator docs README links to. One level only:
+# docs/rfcs/ and docs/migration-tools/ stay home.
+README.md
+LICENSE
+docs/*.md

.publish-include-pages

@@ -0,0 +1,7 @@
+# .publish-include-pages — what the mirror's GitHub Pages branch MAY carry.
+#
+# Read by scripts/publish-guard.sh with `--include` when the gh-pages branch is
+# mirrored (it is what `helm repo add tracebloc https://tracebloc.github.io/client`
+# reads). The branch holds the chart index and the packaged charts, nothing else.
+index.yaml
+*.tgz

client/Chart.yaml

@@ -2,8 +2,8 @@ apiVersion: v2
 name: client
 description: A unified Helm chart for tracebloc on AKS, EKS, bare-metal, and OpenShift
 type: application
-version: 1.9.113
-appVersion: "1.9.113"
+version: 1.9.117
+appVersion: "1.9.117"
 keywords:
   - tracebloc
   - kubernetes

client/MIGRATION.md

@@ -2,6 +2,95 @@
 
 This guide explains how to migrate from the legacy per-platform charts (`aks/`, `bm/`, `eks/`, `oc/`) to the unified `client/` chart.
 
+## Upgrading to 1.9.115 — `env.TRACEBLOC_DDP` defaults ON (RFC-0067 D7)
+
+`env.TRACEBLOC_DDP` now renders as **`"1"`** at the chart default. This is the
+ARM step of backend#3147: RFC-0067 D7's precondition bundle holds — the engine
+defaults its effective-batch mechanism to `per_rank_split` (tracebloc-engine#1010;
+each rank trains on B/N so the effective batch stays B and the numerics are the
+one-GPU experiment's, e2e-test-agent#444), a run on which that mechanism cannot
+apply falls back to **one GPU** rather than to an uncompensated N×B
+(client-runtime#553), and both were verified on the published `:dev` engine and
+jobs-manager digests before this default flipped.
+
+**What changes on upgrade: nothing expands yet.** Multi-GPU needs **both**
+switches — `TRACEBLOC_DDP` truthy **and** `env.MULTI_GPU_MIN_PARAMETERS` set
+(1.9.103) — and the floor still has no default, so an edge that never set a
+floor keeps one GPU per run, with `GPU_COUNT_SIZE_FLOOR_UNSET` in the
+jobs-manager log. To arm an edge, set the floor:
+
+```bash
+helm upgrade <release> tracebloc/client --reuse-values --set-string env.MULTI_GPU_MIN_PARAMETERS=1000000
+```
+
+(1,000,000 admits ResNet-18-class models and refuses LeNet-class ones; it is the
+floor RFC-0067's G6 measured with, not a general recommendation — see
+backend#3147 for where the speedup crossover sits on your hardware.)
+
+**Rollback lever, per edge:**
+
+```bash
+helm upgrade <release> tracebloc/client --reuse-values --set-string env.TRACEBLOC_DDP=0
+```
+
+An explicit `"0"` travels to the jobs-manager and wins over the default; the
+runtime reads it as OFF (`GPU_COUNT_SWITCH_OFF`) and spawns one GPU per run
+whatever the floor says. `TRACEBLOC_AMP` is unchanged (still OFF by default).
+
+## Upgrading to 1.9.114 — training pods pull from the tracebloc registry (`ghcr.io`) by default
+
+The training-image host now follows `images.traceblocRegistry`: `JOB_IMAGE_HOST`,
+the registry prefix the jobs-manager stamps onto every training image it spawns
+(`tracebloc/client-<task>-<cpu|gpu>:<CLIENT_ENV>`), renders as **`ghcr.io/`** at
+the chart default instead of `docker.io/`, on both jobs-manager containers. It is
+resolved by the same `tracebloc.tbRegistry` helper as the control-plane images,
+as ONE precedence chain: a `global.imageRegistry` mirror wins, then
+`images.traceblocRegistry`, then the chart default. From this version the
+control plane (moved in 1.9.113) and the training pods pull from the same
+registry and cannot be pointed at different ones.
+
+**Why:** every training image is published to GHCR at the same digests as its
+Docker Hub copy (the GHCR migration), so this changes where the training pods
+pull from, not which bytes run.
+
+**What you need to do: nothing for most edges.**
+
+- **Egress.** No new host: `ghcr.io` (and `pkg-containers.githubusercontent.com`,
+  where GHCR redirects layer downloads) is already required for the
+  control-plane images since 1.9.113 and for the ingestor image before that.
+  If your allowlist was built by hand from an older egress table, add both
+  before upgrading. Docker Hub is still needed for k3s, `tracebloc/mysql-client`
+  and busybox.
+- **One rollout, then one pull per task.** The jobs-manager pod template changes
+  (`JOB_IMAGE_HOST`), so the upgrade rolls the jobs-manager once. The next
+  experiment of each task pulls its training image from `ghcr.io` — a full pull
+  the first time, as after any tag move; the digest-pinned spawn path
+  (`TRAINING_IMAGE_DIGESTS`) is unaffected, the same digest exists on both
+  registries.
+- **Mirrors.** Edges with `global.imageRegistry` set are unaffected: the mirror
+  re-homes every image, `JOB_IMAGE_HOST` included, and always wins — exactly as
+  before.
+- **Runtime default.** The chart always sets `JOB_IMAGE_HOST`, so the
+  client-runtime's own fallback for an *unset* variable (changed separately, in
+  that project) only ever applies to installs from before the chart carried the
+  key.
+
+**Rollback (per edge):** the same knob as 1.9.113 — it moves the control plane
+AND the training-image host back to Docker Hub together, and, being
+user-supplied, it persists across the fleet auto-upgrade until you clear it:
+
+```bash
+helm upgrade <release> tracebloc/client -n <namespace> \
+  --reset-then-reuse-values --set images.traceblocRegistry=docker.io
+```
+
+Confirm which host the training pods will pull from:
+
+```bash
+kubectl get deploy -n <namespace> <release>-jobs-manager \
+  -o jsonpath='{.spec.template.spec.containers[0].env[?(@.name=="JOB_IMAGE_HOST")].value}{"\n"}'
+```
+
 ## Upgrading to 1.9.113 — the control-plane images pull from `ghcr.io` by default
 
 `images.traceblocRegistry` now defaults to **`ghcr.io`**: the four

client/templates/NOTES.txt

@@ -5,15 +5,15 @@
 {{ "\033[1;35m" }}Components:{{ "\033[0m" }}
   - {{ "\033[1;34m" }}Jobs Manager:{{ "\033[0m" }}            {{ "\033[0;33m" }}{{ include "tracebloc.fullname" . }}-jobs-manager{{ "\033[0m" }}
   - {{ "\033[1;34m" }}MySQL Host:{{ "\033[0m" }}                   {{ "\033[0;33m" }}mysql-client{{ "\033[0m" }}
-{{- if ne .Values.resourceMonitor false }}
+{{- if (include "tracebloc.resourceMonitorEnabled" .) }}
   - {{ "\033[1;34m" }}Resource Monitor (DS):{{ "\033[0m" }}   {{ "\033[0;33m" }}{{ include "tracebloc.resourceMonitorName" . }}{{ "\033[0m" }}
 {{- end }}
 
 {{ "\033[1;35m" }}Configuration:{{ "\033[0m" }}
   {{ "\033[1;34m" }}Service Account:{{ "\033[0m" }}  {{ "\033[0;33m" }}{{ include "tracebloc.serviceAccountName" . }}{{ "\033[0m" }}
   {{ "\033[1;34m" }}Secret:{{ "\033[0m" }}           {{ "\033[0;33m" }}{{ include "tracebloc.secretName" . }}{{ "\033[0m" }}
   {{ "\033[1;34m" }}RBAC scope:{{ "\033[0m" }}       {{ "\033[0;33m" }}{{ if ne .Values.clusterScope false }}Cluster{{ else }}Namespace{{ end }}{{ "\033[0m" }}
-  {{ "\033[1;34m" }}Image registry:{{ "\033[0m" }}   {{ "\033[1;32m" }}{{ include "tracebloc.tbRegistry" . }}{{ "\033[0m" }} (tracebloc control-plane images)
+  {{ "\033[1;34m" }}Image registry:{{ "\033[0m" }}   {{ "\033[1;32m" }}{{ include "tracebloc.tbRegistry" . }}{{ "\033[0m" }} (tracebloc control-plane + training images)
 {{- if (default dict .Values.hostPath).enabled }}
   {{ "\033[1;34m" }}Storage:{{ "\033[0m" }}          {{ "\033[0;33m" }}hostPath (bare-metal){{ "\033[0m" }}
   {{ "\033[1;34m" }}Host dirs:{{ "\033[0m" }}        {{ "\033[0;33m" }}/tracebloc/data, /tracebloc/logs, /tracebloc/mysql (on the node){{ "\033[0m" }}

client/templates/_helpers.tpl

@@ -126,6 +126,42 @@ tracebloc.io/seal-check-name: {{ .name | quote }}
 {{ include "tracebloc.fullname" . }}-resource-monitor
 {{- end }}
 
+{{/*
+  tracebloc.resourceMonitorEnabled — the SINGLE reader of "is the resource-monitor
+  on", coalescing the two value shapes during the RFC-0076 alias window
+  (remove_by: 2026-12-31, client#1009):
+
+    legacy scalar   resourceMonitor: <bool>
+    new object      resourceMonitor.enabled: <bool>   (D2: <component>.enabled)
+
+  This is a bool→object rename, so a stored values.yaml or a bare
+  `--set resourceMonitor=true` still arrives as a SCALAR. Reading
+  `.Values.resourceMonitor.enabled` blindly would `fail` with "can't evaluate
+  field enabled in interface {}" on the scalar and, on a `--reuse-values`
+  upgrade that carries the scalar forward, silently drop the setting. So decide
+  the shape with kindIs and prefer the new `.enabled` form:
+
+    map     -> .enabled, defaulting to true when the key is absent
+    bool    -> the scalar itself
+    absent  -> enabled (the historical default: `ne <nil> false` was true)
+
+  Effective behaviour is unchanged: resourceMonitor.enabled=true does exactly
+  what resourceMonitor=true did. Emits "true" or nothing, so callers use
+  `(include "tracebloc.resourceMonitorEnabled" .)` in an `and`/`or` and
+  `not (include ...)` for the disabled case — the same idiom as
+  tracebloc.nodeAgentsInUse.
+*/}}
+{{- define "tracebloc.resourceMonitorEnabled" -}}
+{{- $rm := .Values.resourceMonitor -}}
+{{- if kindIs "map" $rm -}}
+{{- if ne (dig "enabled" true $rm) false -}}true{{- end -}}
+{{- else if kindIs "invalid" $rm -}}
+{{- "true" -}}
+{{- else -}}
+{{- if ne $rm false -}}true{{- end -}}
+{{- end -}}
+{{- end }}
+
 {{- define "tracebloc.rbacName" -}}
 {{ include "tracebloc.fullname" . }}-jobs-manager-rbac
 {{- end }}
@@ -384,11 +420,12 @@ nvidia-device-plugin-daemonset
     * `resourceMonitor: false` — there is no DaemonSet at all, so there is
       nothing to reconcile and a cross-namespace `set image` would just fail.
 
-  Nil-safe: `.Values.resourceMonitor` absent reads as enabled, matching the
-  `ne .Values.resourceMonitor false` gate on the DaemonSet itself.
+  Nil-safe via tracebloc.resourceMonitorEnabled, which absent reads as enabled,
+  matching the gate on the DaemonSet itself and honouring both the legacy scalar
+  and the new resourceMonitor.enabled object form.
 */}}
 {{- define "tracebloc.resourceMonitorRefreshPinned" -}}
-{{- if eq .Values.resourceMonitor false -}}
+{{- if not (include "tracebloc.resourceMonitorEnabled" .) -}}
 true
 {{- else if (default dict (default dict .Values.images).resourceMonitor).digest -}}
 true
@@ -603,30 +640,36 @@ docker.io ghcr.io
 {{- end -}}
 
 {{/*
-tracebloc.tbRegistry — the registry the tracebloc-PUBLISHED control-plane images
-(tracebloc/jobs-manager, tracebloc/pods-monitor, tracebloc/resource-monitor, and
-the requests-proxy, which runs the jobs-manager image) are pulled from.
+tracebloc.tbRegistry — the registry the tracebloc-PUBLISHED images are pulled
+from: the control-plane images (tracebloc/jobs-manager, tracebloc/pods-monitor,
+tracebloc/resource-monitor, and the requests-proxy, which runs the jobs-manager
+image) AND the host jobs-manager stamps onto every training image it spawns
+(JOB_IMAGE_HOST, rendered as "<registry>/" on both jobs-manager containers).
 
-ONE precedence chain, so the four call sites, the image-refresh CronJob and
-NOTES.txt cannot disagree about where those images live:
+ONE precedence chain, so the four control-plane call sites, the two
+JOB_IMAGE_HOST sites, the image-refresh CronJob and NOTES.txt cannot disagree
+about where those images live:
 
   1. `global.imageRegistry`      — a private mirror re-homes EVERY image the
                                     chart pulls (#585), tracebloc/* included.
                                     It always wins.
-  2. `images.traceblocRegistry`  — the tracebloc-only knob: moves just the
-                                    tracebloc-published images, leaving busybox,
-                                    squid, alpine/*, the device plugins and the
-                                    ingestor where they are. Also the per-edge
-                                    rollback: set it to the previous registry.
+  2. `images.traceblocRegistry`  — the tracebloc-only knob: moves the
+                                    tracebloc-published images -- control plane
+                                    and training-image host TOGETHER -- leaving
+                                    busybox, squid, alpine/*, the device plugins
+                                    and the ingestor where they are. Also the
+                                    per-edge rollback: set it to the previous
+                                    registry.
   3. "ghcr.io"                   — the chart default since the GHCR migration.
                                     The images are still dual-published to
                                     Docker Hub at the same digests, so
                                     "docker.io" is the documented rollback.
 
 NOT routed through here, on purpose: `tracebloc/mysql-client` (frozen,
 digest-pinned, published only to Docker Hub — see images.mysqlClient), the
-third-party images (each has its own `registry` key), and — for now — the
-training-image host JOB_IMAGE_HOST, which moves in its own step.
+third-party images (each has its own `registry` key), and the ingestor, which is
+named by full repository (images.ingestor.repository, already on ghcr.io) and
+follows only the global mirror.
 
 Every read is nil-guarded and `| default`-chained: values.yaml ships
 `global.imageRegistry: ""` (the key EXISTS, so `dig`'s own fallback never
@@ -1550,7 +1593,7 @@ https://api.tracebloc.io/
   became a second tenant, two of them were widened and the rest were not." The
   tri-state made `enabled` a second copy of the answer for a third time.
 */ -}}
-{{- if or (ne .Values.resourceMonitor false) (eq (include "tracebloc.telemetryCollectorState" .) "enabled") }}true{{ end -}}
+{{- if or (include "tracebloc.resourceMonitorEnabled" .) (eq (include "tracebloc.telemetryCollectorState" .) "enabled") }}true{{ end -}}
 {{- end -}}
 
 {{/*