Add release-tag measurements and capture recovery evidence

.github/workflows/ci.yml

@@ -2,6 +2,8 @@ name: Benchmark CI
 
 on:
   pull_request:
+  push:
+    branches: [main]
   workflow_dispatch:
 
 permissions:
@@ -12,7 +14,7 @@ concurrency:
   cancel-in-progress: true
 
 env:
-  ARGON_ENGINE_REF: ba6e06a9a3b31124d6c37475b5667dd70ab42379
+  ARGON_ENGINE_REF: v2.1.2
   ARGON_SUITE_REF: ${{ github.event.pull_request.head.sha || github.sha }}
 
 jobs:
@@ -47,6 +49,7 @@ jobs:
       - name: Configure isolated Compose fixture
         run: |
           echo "ARGON_ENGINE_SOURCE=$GITHUB_WORKSPACE/engine" >> "$GITHUB_ENV"
+          echo "ARGON_ENGINE_SHA=$(git -C ../engine rev-parse HEAD)" >> "$GITHUB_ENV"
           mkdir -p results
 
       - name: Validate and build the Go 1.26.6 container recipe
@@ -62,7 +65,7 @@ jobs:
       - name: Run unit tests and one-cell correctness smoke
         run: |
           docker compose run --rm --no-deps bench \
-            --ref "$ARGON_ENGINE_REF" --results /suite/results/ci-smoke -- \
+            --ref "$ARGON_ENGINE_SHA" --results /suite/results/ci-smoke -- \
             -sizes 100 -concurrency 1 -depths 1 \
             -metadata-samples 2 -workflow-samples 2 -read-samples 2 \
             -divergence-docs 3 -divergence-rounds 2 -timeout 3m \
@@ -75,7 +78,7 @@ jobs:
           from pathlib import Path
           report = json.loads(Path('results/ci-smoke/raw.json').read_text())
           assert report['complete']
-          assert report['provenance']['engine']['git_head'] == os.environ['ARGON_ENGINE_REF']
+          assert report['provenance']['engine']['git_head'] == os.environ['ARGON_ENGINE_SHA']
           assert report['provenance']['suite']['git_head'] == os.environ['ARGON_SUITE_REF']
           assert not report['provenance']['engine']['dirty']
           assert not report['provenance']['suite']['dirty']
@@ -97,6 +100,7 @@ jobs:
 
       - name: Retain raw smoke samples, provenance, source archives and logs
         if: always()
+        continue-on-error: true # Optional evidence upload; smoke/provenance gates remain required.
         uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
         with:
           name: benchmark-smoke-${{ github.run_id }}-${{ github.run_attempt }}

README.md

@@ -2,7 +2,7 @@
 
 This suite measures an explicit Argon engine checkout. It distinguishes a metadata-only fork from a sandbox with a physical MongoDB copy and capture ready, then measures the first native query, captured-write visibility and storage after divergence. Every successful run writes all timing samples, nearest-rank p50/p95/p99, configuration, environment and source provenance.
 
-The workflow runner requires the engine's `StartCapture`, `SyncBranch` and `WaitAuto` APIs. The dependency in `go.mod` is the historical published baseline, **not** the code measured by this runner. Until these APIs are released, an explicit checkout containing the hardening changes is required. `scripts/run.py` uses a temporary module replacement, freezes the exact engine source, and records its Git ref, dirty status, tracked diff SHA256, complete source manifest/hash and build module information. It also archives the frozen engine and executable suite source, including untracked files; unpublished code is labelled accordingly. The suite's executable source hash identifies runner changes before a suite commit exists.
+The current runner targets Argon **v2.1.2+** and its `/v2` Go module. `scripts/run.py` uses a temporary module replacement, freezes the explicitly selected engine source, and records its Git ref, matching release tag when present, dirty status, tracked diff SHA256, complete source manifest/hash and build module information. It also archives the frozen engine and executable suite source, including untracked files; unpublished code is labelled accordingly. The suite's executable source hash identifies runner changes before a suite commit exists. Reproduce pre-v2.1.2 reports with their recorded historical suite commit; their imports and import contract differ from the current runner.
 
 ## Run locally
 
@@ -12,7 +12,7 @@ Requires Go 1.26.6+, Python 3.12+, Git and MongoDB 7+ configured as a replica se
 export MONGODB_URI='mongodb://localhost:27017/?replicaSet=rs0'
 export GOCACHE=/tmp/argon-benchmark-go-cache
 export GOTOOLCHAIN=go1.26.6
-python3 scripts/run.py --engine /absolute/path/to/argon -- \
+python3 scripts/run.py --engine /absolute/path/to/argon --ref v2.1.2 -- \
   -sizes 1000,10000 -concurrency 1,4 -depths 1,4 \
   -metadata-samples 100 -workflow-samples 20 -read-samples 20
 ```
@@ -29,7 +29,7 @@ docker compose up --build --abort-on-container-exit --exit-code-from bench
 docker compose down
 ```
 
-The Compose configuration pins MongoDB 7.0.14 and configures replica set `rs0`. Host-local and container measurements are separate environments and must not be compared as equivalent runs. The current local report was run on the host; the container recipe has not yet been exercised on this machine.
+The Compose configuration pins MongoDB 7.0.43 and configures replica set `rs0`. Host-local and container measurements are separate environments and must not be compared as equivalent runs. CI exercises the container recipe; recorded local reports state their own MongoDB version and environment. Fixture writes finish before import and the runner explicitly acknowledges the engine's quiesced-source requirement.
 
 ## Measurements and definitions
 
@@ -63,10 +63,30 @@ This runs 1k/50k/1m documents × 1/4/16 workers × 1/4/16 ancestry depth, with 1
 
 ## Pull-request CI
 
-The PR workflow checks out companion engine commit `ba6e06a9a3b31124d6c37475b5667dd70ab42379` beside the suite, builds the Go 1.26.6 Docker recipe, and starts the Compose MongoDB replica set. It runs the runner's Go unit tests followed by one 100-document / one-worker / one-level smoke cell, with two workflow samples and six required captured divergence updates. `--ref` is explicit; CI verifies the engine/suite refs and actual Go version in the generated provenance. Raw samples, reports, both source archives and logs are retained as a workflow artifact for 14 days, including available diagnostics on failure.
+The PR and main-branch workflow checks out companion engine release `v2.1.2` beside the suite, resolves its exact commit, builds the Go 1.26.6 Docker recipe, and starts the Compose MongoDB replica set. It runs the runner's Go unit tests followed by one 100-document / one-worker / one-level smoke cell, with two workflow samples and six required captured divergence updates. `--ref` is the resolved commit; CI verifies the engine/suite refs and actual Go version in the generated provenance. The job attempts to retain raw samples, reports, source archives and logs for 14 days. Optional artifact uploads can fail when account storage is full; test and provenance failures still fail CI. Published measurement bundles remain in this repository.
 
 This job checks correctness and reproducibility of the container workflow. Its tiny sample counts and shared CI runner are unsuitable for performance SLAs or comparisons with the recorded local matrix. The historical reports and their measured suite refs remain unchanged.
 
 ## Published results
 
-See [RESULTS.md](RESULTS.md). Historical numbers retain their original date and exact scope. New results include raw samples and source hashes; no dirty working tree is identified as a released engine version. Benchmark failure exits nonzero and does not publish a complete report.
+The [v2.1.2 release matrix](reports/2026-09-24-v2.1.2/README.md) covers 1k/10k/50k documents with 1/4 workers and 1/4 ancestry levels. See [RESULTS.md](RESULTS.md). Historical numbers retain their original date and exact scope. New results include raw samples and source hashes; no dirty working tree is identified as a released engine version. Benchmark failure exits nonzero and does not publish a complete report.
+
+## Sustained capture and process recovery
+
+Use a SHA256-verified release CLI with a disposable replica set. Install
+`pymongo` in an isolated Python environment, then run:
+
+```sh
+python3 scripts/capture_recovery.py --binary /absolute/path/to/argon \
+  --version 2.1.2 --duration-seconds 600 --output /new/recovery-results
+```
+
+The script creates unique metadata and sandbox databases, acknowledges native
+writes with majority write concern, and repeatedly verifies complete WAL counts
+and reconstructed values. Halfway through, it kills only its own API process,
+writes while capture is stopped, restarts, and checks catch-up without lost or
+duplicate writes. It also checks graceful shutdown after health-check cycles and
+removes only its fixture databases. The output records the release binary hash,
+MongoDB version, elapsed time, write counts and recovery duration. This tests API
+process recovery, not MongoDB failover, storage loss or a production availability
+SLA. Run it independently of timing matrices to avoid self-induced contention.

RESULTS.md

@@ -2,6 +2,31 @@
 
 Every number below is scoped to its recorded engine, suite and environment. Historical measurements are retained as history; current hardening measurements are separately labelled. A dirty source snapshot is not a released engine version. See [README.md](README.md) for methodology and reproducible commands.
 
+## 2026-09-24 · released v2.1.2 source, 12-cell matrix
+
+Engine [`v2.1.2`](https://github.com/argon-lab/argon/releases/tag/v2.1.2), commit `03559026972457ea663fdbc77d252e20ffb65b63`; suite `1e9fa82b30593beea851e2ff624eaf7d6cda0434`. Both measured trees were clean and frozen. All 12 cells completed: 1k/10k/50k documents × 1/4 workers × 1/4 ancestry levels, with 1,200 metadata forks, 240 sandbox/capture workflows and 15,000 verified captured divergence updates.
+
+Environment: shared Apple M6 / 24 GiB host, macOS 27.0 arm64, Go 1.26.6, native MongoDB 7.0.43 one-member replica set, WiredTiger 0.5 GiB cache and majority acknowledgment. Window: `2026-09-24T07:46:20.711026Z` → `2026-09-24T07:50:05.174369Z`. Background host activity was uncontrolled; no heavy Argon test suite or timing matrix ran concurrently. This environment differs from earlier reports, so the numbers are not a controlled comparison.
+
+Selected depth 4 observations, milliseconds (`p50 / p95 / p99`):
+
+| Documents | Workers | Metadata fork (n=100) | Sandbox + capture ready (n=20) | First native query (n=20) | Ack → WAL observed (n=20) |
+|---:|---:|---|---|---|---|
+| 1000 | 1 | 15.77 / 18.16 / 18.76 | 136.75 / 154.69 / 157.05 | 11.59 / 16.21 / 18.61 | 514.47 / 518.71 / 552.94 |
+| 1000 | 4 | 23.99 / 33.65 / 37.81 | 186.90 / 271.21 / 317.32 | 12.86 / 27.07 / 34.99 | 512.25 / 522.56 / 534.36 |
+| 10000 | 1 | 11.01 / 17.12 / 19.26 | 231.53 / 249.21 / 255.47 | 8.77 / 10.03 / 13.08 | 510.53 / 518.88 / 522.59 |
+| 10000 | 4 | 20.83 / 23.96 / 24.13 | 351.07 / 520.30 / 647.30 | 7.00 / 11.24 / 21.46 | 510.34 / 522.28 / 534.43 |
+| 50000 | 1 | 16.06 / 18.94 / 18.98 | 599.11 / 646.29 / 761.98 | 5.80 / 8.15 / 9.46 | 518.62 / 522.88 / 524.57 |
+| 50000 | 4 | 17.87 / 28.93 / 35.88 | 1425.89 / 1599.13 / 2122.08 | 8.46 / 20.00 / 22.29 | 510.28 / 518.61 / 524.15 |
+
+[Full generated report](reports/2026-09-24-v2.1.2/report.md), [all raw samples](reports/2026-09-24-v2.1.2/raw.json), and [provenance/reproduction instructions](reports/2026-09-24-v2.1.2/README.md). All summary quantiles were independently recomputed from raw samples. Twenty workflows per cell make p99 the observed maximum. Sandbox readiness includes a full physical copy; metadata latency does not describe total readiness. Ack-to-observed latency includes batching and polling, and is separate from native write acknowledgment. Storage data distinguishes logical document bytes from immediate allocated collection/index bytes.
+
+The separate published-binary recovery experiment completed 600.25 seconds with 1,865 acknowledged native writes and 1,866 exact history/state checks. Killing and restarting the API midway, with an additional write while it was stopped, recovered in 0.329 seconds; final graceful shutdown took 0.039 seconds (exit 0). [Summary](reports/2026-09-24-v2.1.2/recovery.json) and [all samples](reports/2026-09-24-v2.1.2/recovery-samples.json) are retained. This is a single-writer local correctness/soak result, not a production uptime or database failover claim.
+
+The 1m/16-worker/16-depth expansion remains an experiment plan; production tail latency and multi-node availability are not established by this run.
+
+---
+
 ## 2026-09-07 · committed hardening engine, Go 1.26.6
 
 This final local matrix uses exact engine commit `ba6e06a9a3b31124d6c37475b5667dd70ab42379` and suite commit `a05950938a8eae5ba8cd089a02d9b7207a641945`. Both were frozen from clean committed sources. **This identifies a local commit build, not a tag release.** The previous dirty-source diagnostic below remains a separate historical record.

docker-compose.yml

@@ -1,6 +1,6 @@
 services:
   mongo:
-    image: mongo:7.0.14
+    image: mongo:7.0.43
     command: ["mongod", "--replSet", "rs0", "--bind_ip_all"]
     healthcheck:
       test: ["CMD-SHELL", "mongosh --quiet --eval 'try { rs.status() } catch (e) { rs.initiate({_id: \"rs0\", members: [{_id: 0, host: \"mongo:27017\"}]}) }; if (!db.hello().isWritablePrimary) quit(1)' "]

go.mod

@@ -3,7 +3,7 @@ module github.com/argon-lab/benchmarks
 go 1.26.6
 
 require (
-	github.com/argon-lab/argon v1.0.2-0.20260707043331-8bf0f1e9dd85
+	github.com/argon-lab/argon/v2 v2.1.2
 	go.mongodb.org/mongo-driver v1.17.7
 )