Add Kubernetes deployment units

README.md

@@ -27,6 +27,11 @@ producer and consumer; it carries no Cardano assumption.
 The boundary proof lives in the test suite: `stelae/tests/toy_profile.rs`
 implements a second, trivial profile against the protocol surface alone.
 
+## Kubernetes
+
+[`k8s/`](k8s/) contains the instance-agnostic Helm deployment units for the
+Dolos publisher and the Stelae registry.
+
 ## Specification
 
 The normative specification is [`SPEC.md`](SPEC.md). The Dolos profile's own

k8s/README.md

@@ -0,0 +1,10 @@
+# Kubernetes
+
+Helm-packaged deployment units related to the Stelae protocol:
+
+- [`dolos-publisher/`](dolos-publisher/) runs Dolos backfill Jobs that publish
+  network steles.
+- [`registry/`](registry/) runs the zot registry that stores and serves them.
+
+Both charts are instance-agnostic. Deployment-specific values and secrets stay
+in the operator's infrastructure repository.

k8s/dolos-publisher/README.md

@@ -0,0 +1,83 @@
+# Dolos publisher — deployment unit
+
+The publisher is one Kubernetes Job per network, rendered from the Helm
+chart in [`chart/`](chart/) with a values file per network maintained by the
+operator. The operator's runbook owns deployment, re-run, first-run, and
+monitoring procedures.
+Each Job runs the official `ghcr.io/txpipe/dolos` image on the Demeter m2
+EKS cluster: it restores its network's latest stele (or starts from
+genesis), replays to the next epoch boundary, publishes, prunes, repeats,
+and exits 0 at the aggregator tip. Each publish is its own checkpoint, so a
+pod restart costs one epoch and nothing more.
+
+## What the chart renders
+
+Release `stelae-publisher-<network>` in namespace `stelae-publisher` — the
+registry's namespace, which the chart does not create; the deploy passes
+`--namespace`. Two objects:
+
+- **ConfigMap** `<release>-config`, mounted at `/etc/publisher`:
+  `entrypoint.sh`, the restore-or-genesis script templated from the chart
+  (`concurrency` and `--insecure` from values), and `dolos.toml`, the
+  network's config carried **verbatim** from the values string `dolosToml`.
+  Its `[snapshot] state_epochs` list is signed input, frozen at the
+  network's first publish (decisions 0028, 0030, 0038) and extended only
+  above the then-current tip. Carrying the file verbatim means no template
+  can change it silently, and its comments stay in the values file.
+- **Job** `<network>-backfill-<run>`: the official image pinned by
+  `image.tag`, `/data` an `emptyDir`, the ConfigMap at `/etc/publisher`, the
+  registry pair from the Secret named `publisherSecretName`
+  (`stelae-registry-publisher` — referenced, never templated).
+
+## The `run` mechanism
+
+A Job's pod template is immutable: it cannot be upgraded in place. The
+chart makes the re-run a deliberate act by putting a counter in the Job's
+name. Bumping `run` in the values file and upgrading makes Helm create the
+new Job and remove the old one; the new pod restores from `latest` and the
+epoch cost is paid on purpose. Changing anything else in the pod template
+without a bump fails the upgrade against the immutable field — the guard
+the raw manifests lacked. Deleting a Job by hand instead of bumping leaves
+the release history and the cluster disagreeing: the next upgrade recreates
+it under the old name. Bump, never delete.
+
+## Values a network must supply
+
+| value | holds |
+|---|---|
+| `network` | prefixes the Job name, labels the pod |
+| `run` | the re-run counter |
+| `repo` | the `oci://` URL the publisher writes |
+| `concurrency` | uploads in flight per publish — measured per network, never a default |
+| `dolosToml` | the network's `dolos.toml`, verbatim |
+| `image.tag` | the `sha-<short>` pin; there is no chart-wide dolos version |
+| `resources` | the working set; no default |
+
+Everything else defaults in [`chart/values.yaml`](chart/values.yaml)
+with its reason beside it: `insecure` on (the write path is in-cluster plain
+HTTP), `publisherSecretName`, `backoffLimit`,
+`terminationGracePeriodSeconds`, and empty placement.
+
+## Constraints
+
+- **Concurrency is a network fact.** Mainnet 16 because zot parallelises
+  where the old Worker did not; the testnets 4 because in-cluster zot
+  serialises blob commits, and a deeper queue only adds latency to any
+  mainnet window beside it. Never lowered to make bursts smaller.
+- **Single-publisher discipline.** Exactly one writer per
+  `cardano/<network>`, ever.
+- **Placement is a value.** The dedicated `stele-backfill` nodegroup was
+  deleted on 2026-08-31; every network runs on the shared best-effort pool
+  today, and a dedicated node again is a `nodeSelector` and `tolerations`
+  change in that network's values file.
+
+Local check, no cluster needed:
+
+```bash
+helm lint k8s/dolos-publisher/chart --values "$PUBLISHER_VALUES"
+helm template stelae-publisher-preprod k8s/dolos-publisher/chart \
+    --namespace stelae-publisher --values "$PUBLISHER_VALUES"
+```
+
+Its predecessors are in git history: raw manifests applied by hand, and before
+them a Cloudflare Worker with a container-backed Durable Object.

k8s/dolos-publisher/chart/.helmignore

@@ -0,0 +1,5 @@
+.DS_Store
+.git/
+.gitignore
+*.tmp
+*.orig

k8s/dolos-publisher/chart/Chart.yaml

@@ -0,0 +1,13 @@
+apiVersion: v2
+name: dolos-publisher
+description: one stele publisher — a Dolos backfill Job that restores, replays, publishes and prunes one network's stele repository
+type: application
+# Chart version: this chart. There is no appVersion on purpose: the dolos
+# build a release runs is image.tag in the network's values file, and the
+# three networks need not agree.
+version: 0.1.0
+home: https://github.com/txpipe/stelae
+keywords:
+  - stelae
+  - oci
+  - publisher

k8s/dolos-publisher/chart/templates/NOTES.txt

@@ -0,0 +1,17 @@
+{{ include "dolos-publisher.jobName" . }} in {{ .Release.Namespace }} — {{ .Values.network }}, run {{ .Values.run }}, concurrency {{ .Values.concurrency }}, {{ .Values.image.repository }}:{{ .Values.image.tag }}
+
+Follow the run:
+  kubectl -n {{ .Release.Namespace }} logs -f job/{{ include "dolos-publisher.jobName" . }}
+
+Lines worth reading:
+  restored own latest stele                            a pod start paid a full restore
+  no restorable stele — starting from genesis          the repository was empty
+  replaying toward the next epoch boundary target=N    replay started for N
+  sequence: N (tag epoch-N)                            publish window opened
+  the registry failed a round trip; making it again attempt=K
+                                                       a retry; attempt=3 is worth attention
+
+Re-run on purpose: bump `run` in the values file and upgrade. Any other
+change to the pod template without a bump fails the upgrade against the
+Job's immutable template — that failure is the guard, and the answer is
+to bump `run`. Never delete the Job by hand.

k8s/dolos-publisher/chart/templates/_helpers.tpl

@@ -0,0 +1,14 @@
+{{- define "dolos-publisher.jobName" -}}
+{{ required "network is required" .Values.network }}-backfill-{{ required "run is required" .Values.run }}
+{{- end -}}
+
+{{- define "dolos-publisher.configMapName" -}}
+{{ .Release.Name }}-config
+{{- end -}}
+
+{{- define "dolos-publisher.labels" -}}
+helm.sh/chart: {{ printf "%s-%s" .Chart.Name .Chart.Version }}
+app.kubernetes.io/name: {{ .Chart.Name }}
+app.kubernetes.io/instance: {{ .Release.Name }}
+app.kubernetes.io/managed-by: {{ .Release.Service }}
+{{- end -}}

k8s/dolos-publisher/chart/templates/configmap.yaml

@@ -0,0 +1,48 @@
+{{/*
+The restore-or-genesis entrypoint and the network's dolos.toml, mounted at
+/etc/publisher. The entrypoint is templated (concurrency, --insecure); the
+dolos.toml is the values string verbatim.
+*/}}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ include "dolos-publisher.configMapName" . }}
+  namespace: {{ .Release.Namespace }}
+  labels:
+    {{- include "dolos-publisher.labels" . | nindent 4 }}
+data:
+  entrypoint.sh: |
+    #!/bin/sh
+    # Restore-or-genesis, then run the backfill loop.
+    #
+    # /data is an emptyDir — every pod start begins empty. If the repo holds
+    # a stele, restore it (pruned by sync.max_history) and continue from
+    # there; an empty repo means genesis. A transient restore failure
+    # degrades to a genesis run whose publish lands on Standing::UpToDate
+    # and no-ops — wasted CPU, never a corrupt repo.
+    #
+    # Rendered by the dolos-publisher chart: the concurrency and the
+    # --insecure flag come from the network's values file, with their
+    # reasons.
+    set -eu
+
+    REPO="${PUBLISHER_REPO:?PUBLISHER_REPO is required}"
+    CONFIG=/etc/publisher/dolos.toml
+    DATA=/data/db
+
+    if [ ! -d "$DATA" ] || [ -z "$(ls -A "$DATA" 2>/dev/null || true)" ]; then
+        echo "fresh disk: attempting restore of own latest stele from $REPO"
+        if dolos --config "$CONFIG" bootstrap stelae --source "$REPO" --point latest{{ if .Values.insecure }} --insecure{{ end }}; then
+            echo "restored own latest stele"
+        else
+            echo "no restorable stele — starting from genesis"
+        fi
+    fi
+
+    exec dolos --config "$CONFIG" snapshot backfill --repo "$REPO" \
+        {{- if .Values.insecure }}
+        --insecure \
+        {{- end }}
+        --concurrency {{ required "concurrency is required" .Values.concurrency }}
+  dolos.toml: |
+    {{- required "dolosToml is required" .Values.dolosToml | nindent 4 }}

k8s/dolos-publisher/chart/templates/job.yaml

@@ -0,0 +1,59 @@
+{{/*
+The backfill Job. Its name carries the run counter: a Job's pod template is
+immutable, so a re-run is a new Job, and Helm removes the old one in the
+same upgrade. Everything else about the pod is a value.
+*/}}
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: {{ include "dolos-publisher.jobName" . }}
+  namespace: {{ .Release.Namespace }}
+  labels:
+    {{- include "dolos-publisher.labels" . | nindent 4 }}
+spec:
+  backoffLimit: {{ .Values.backoffLimit }}
+  template:
+    metadata:
+      labels:
+        app: {{ .Values.network }}-backfill
+    spec:
+      restartPolicy: OnFailure
+      {{- with .Values.nodeSelector }}
+      nodeSelector:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.tolerations }}
+      tolerations:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      terminationGracePeriodSeconds: {{ .Values.terminationGracePeriodSeconds }}
+      containers:
+        - name: publisher
+          image: {{ .Values.image.repository }}:{{ required "image.tag is required" .Values.image.tag }}
+          command: ["/bin/sh", "/etc/publisher/entrypoint.sh"]
+          env:
+            - name: PUBLISHER_REPO
+              value: {{ required "repo is required" .Values.repo | quote }}
+            - name: DOLOS_STELAE_REGISTRY_USER
+              valueFrom:
+                secretKeyRef:
+                  name: {{ .Values.publisherSecretName }}
+                  key: DOLOS_STELAE_REGISTRY_USER
+            - name: DOLOS_STELAE_REGISTRY_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: {{ .Values.publisherSecretName }}
+                  key: DOLOS_STELAE_REGISTRY_PASSWORD
+          resources:
+            {{- required "resources is required" .Values.resources | toYaml | nindent 12 }}
+          volumeMounts:
+            - name: data
+              mountPath: /data
+            - name: {{ .Values.network }}-config
+              mountPath: /etc/publisher
+      volumes:
+        - name: data
+          emptyDir: {}
+        - name: {{ .Values.network }}-config
+          configMap:
+            name: {{ include "dolos-publisher.configMapName" . }}

k8s/dolos-publisher/chart/values.yaml

@@ -0,0 +1,72 @@
+# One stele publisher: a Kubernetes Job running the official dolos image,
+# which restores its network's latest stele (or starts from genesis),
+# replays to the next epoch boundary, publishes, prunes, and repeats until
+# it reaches the aggregator tip. One release per network.
+#
+# These defaults are instance-agnostic — no network, registry, image pin or
+# credential is assumed. Everything a network must supply is empty here and
+# `required` in the templates, so a missing value fails at render rather
+# than starting a half-configured publisher. The operator supplies one values
+# file per network.
+#
+# Two values are deliberately without a default. `concurrency` was measured
+# per network against the registry it writes to. `dolosToml` carries the
+# `[snapshot] state_epochs` list, which is signed input frozen at the
+# network's first publish. Neither may be inherited from a chart.
+
+# The network name. Prefixes the Job name and labels the pod.
+network: ""
+
+# The deliberate re-run counter. The Job is named <network>-backfill-<run>,
+# so bumping it makes the upgrade create a new Job and remove the old one,
+# paying the restart cost on purpose. A pod template change without a bump
+# fails the upgrade against the Job's immutable template, which is the guard
+# against re-running by accident. See README.md.
+run: ""
+
+# Where the publisher writes: an oci:// URL to the network's stele
+# repository.
+repo: ""
+
+# Uploads in flight during a publish. Measured per network against the
+# registry it writes to; never lowered to make bursts smaller.
+concurrency: ""
+
+# The network's dolos.toml, carried verbatim into the ConfigMap — comments
+# included — so the frozen state_epochs list and its reasoning stay readable
+# in the values file and no template can change signed input.
+dolosToml: ""
+
+image:
+  # The official image: CI-built from every merge to main, every network's
+  # genesis files at /etc/genesis/<network>/, multi-arch.
+  repository: ghcr.io/txpipe/dolos
+  # Pin to the sha-<short> tag of a known commit — never latest. Each
+  # network's values file holds its own pin; there is no chart-wide dolos
+  # version.
+  tag: ""
+
+# The registry runs in-cluster and the write path is plain HTTP, so both the
+# restore and the backfill pass --insecure. Turn off only for a TLS registry.
+insecure: true
+
+# Secret holding DOLOS_STELAE_REGISTRY_USER and DOLOS_STELAE_REGISTRY_PASSWORD.
+# Referenced by name, never templated.
+publisherSecretName: stelae-registry-publisher
+
+# Restart budget. Every pod restart costs one epoch by design (/data is an
+# emptyDir; the pod restores from latest), and on spot capacity an eviction
+# counts here too.
+backoffLimit: 200
+
+# Covers the driver's SIGTERM handler finishing its chunk and flushing the
+# stores.
+terminationGracePeriodSeconds: 600
+
+# Placement. Empty means anywhere the scheduler likes, which is rarely right
+# on a tainted cluster.
+nodeSelector: {}
+tolerations: []
+
+# No default: the working set is a network fact.
+resources: {}

k8s/registry/.gitignore

@@ -0,0 +1 @@
+node_modules/