Skip to content

feat(settle): lab 18 — zero-inference partial-observation controller - #75

Merged
bigboateng merged 2 commits into
mainfrom
feat/settle-lab
Jul 20, 2026
Merged

feat(settle): lab 18 — zero-inference partial-observation controller#75
bigboateng merged 2 commits into
mainfrom
feat/settle-lab

Conversation

@bigboateng

Copy link
Copy Markdown
Contributor

What

New lab 18 — settle: a standalone, compiled Go binary that wraps any agent's command execution and intervenes with minimum, nonblocking force when the loop stops making observable progress. Zero model inference — all detection is hashing, counting, diffing, and regex.

This is a new architecture, not lab 17 (f-prime-governor) shipped. It keeps lab 17's defensible ideas (the EV inequality, minimum-force under partial observation, weak-sensor agreement) and drops F' = p(1−p), which was decorative.

Theory (docs/01)

settle is the same Ramadge–Wonham supervisor as Boatstack, operating in the regime where the spec is unobservable:

  • Boatstack's spec is co-designed observable → no false-positive rate.
  • settle's spec ("do not remain in a non-progressing cycle") is provably not observable — a futile loop and slow-but-genuine progress can share the same projection P(s). By the Existence theorem, no exact supervisor exists, so settle runs an approximate belief-state supervisor and pays for the unobservability with an irreducible FP rate (stated as a theorem, not a defect).

Design consequences, aligned to repo-root supervisory-control.md:

  • minimum force = the minimally-restrictive supervisor (supC(K)), which is also the EV > 0 rule — two frames, one law.
  • nonblocking: even a deny withholds one command and leaves read-only moves open (fixes lab 17's swe_breaker.py exit(1) hard-block, which violated this).
  • fail-OPEN, inverting Boatstack's fail-closed: an optimizer must never block a healthy agent under sensor loss.

Detection (docs/02)

The real object is a non-marked recurrent class in the observation stream. One cycle detector subsumes the five sensors: state-repeat = period-1, oscillation = period-k, with edit-distance-floor and entropy-collapse as leading indicators. Recurrence keys on the joint (error signature, state hash), so a loop whose error repeats while the workspace keeps changing (hidden progress) is not flagged. Semantic-signature + command-normalization ported into a single Go source of truth (was duplicated across three drifting files in lab 17).

Surface

settle exec -- <command> [args...]   # supervise one command (atomic unit)
settle run  -- <agent> [args...]     # wrap an agent; shell shim routes through settle exec
settle doctor                        # self-check

Isolated — no runtime Pitot/Harbor dependency. Decision maps 1:1 to Pitot's control.response for a later bridge adapter.

Calibration (docs/04)

scripts/calibrate.py mines 1,890 Harbor trajectories for the recovery-probability curve. Finding: byte-identical repeated failing commands are rare (agents mutate commands) — which validates the joint/semantic sensors over lab 17's literal hash. Data hints k=2 on thin support (44 sequences); ships the conservative fallback k=5 transparently, with the derived hint recorded in calibration.json.

Claim discipline

Detection firing correctly on the golden fixtures = Observed. Net effect on task success = Being-evaluated until the pre-registered paired experiment (docs/04). No invented numbers, no cross-model uplift claims.

Verification

  • GOWORK=off go test ./... green — golden fixtures (always-fail, syntax-mutating, oscillation, genuine-progress, near-miss, hidden-progress), a determinism property test, and the canonicalization audit.
  • settle doctor passes; a live settle exec loop escalates inform → redirect → deny as designed.
  • scripts/check_import_boundary.sh holds (detector never imports exec).
  • Module added to go.work; go vet clean across all three workspace modules.

🤖 Generated with Claude Code

settle is a new architecture (not lab 17 shipped): a standalone, compiled Go
binary that wraps any agent's command execution and intervenes with minimum,
nonblocking force when the loop stops making observable progress. Zero model
inference — all detection is hashing, counting, diffing, and regex.

Theory (docs/01): settle is the same Ramadge-Wonham supervisor as Boatstack,
operating in the regime where the spec is UNOBSERVABLE. Boatstack's spec is
co-designed observable (no false-positive rate); settle's ("do not remain in a
non-progressing cycle") is provably not — a futile loop and slow-but-genuine
progress can share the same projection P(s) — so by the Existence theorem no
exact supervisor exists. settle runs an approximate belief-state supervisor and
pays for unobservability with an irreducible FP rate, stated as a theorem.

Design consequences, aligned to repo-root supervisory-control.md:
- minimum force = the minimally-restrictive supervisor (supC(K)), also the EV>0 rule
- nonblocking: even a deny withholds one command and leaves read-only moves open
  (fixes lab 17's swe_breaker exit(1) hard-block, which violated this)
- fail-OPEN, inverting Boatstack's fail-closed: an optimizer must never block a
  healthy agent under sensor loss

Detection (docs/02): drops lab 17's F'=p(1-p) (decorative Bernoulli variance).
The real object is a non-marked recurrent class in the observation stream; one
cycle detector subsumes state-repeat (period 1), oscillation (period k), with
edit-distance-floor and entropy-collapse as leading indicators. Recurrence keys
on the JOINT (error signature, state hash), so a loop whose error repeats while
the workspace changes (hidden progress) is not flagged. Semantic-signature and
command-normalization ported to a single Go source of truth (was duplicated
across three drifting files in lab 17).

Surface: settle exec -- <cmd> (atomic unit), settle run -- <agent> (shell-shim
wrapper), settle doctor. Isolated — no runtime Pitot/Harbor dependency; Decision
maps 1:1 to Pitot control.response for a later bridge adapter.

Calibration (docs/04): scripts/calibrate.py mines 1,890 Harbor trajectories for
the recovery-probability curve. Finding: byte-identical repeated failing
commands are rare (validates the joint/semantic sensors over lab 17's literal
hash). Data hints k=2 on thin support (44); ships the conservative fallback k=5
transparently. Net effect on task success is Being-evaluated pending the
pre-registered paired experiment — not claimed.

Verified: go test ./... green (fixtures + determinism + canonicalization),
detector-innocence import boundary holds, module added to go.work.
settle replay -- <trajectory.json>... drives the detector over recorded
ATIF-v1.7 agent trajectories and reports what settle would have done, without
running anything. The safe/free observe mode for seeing behavior on real runs
before any live wiring. Header caveats that offline replay has no workspace
state, so it runs error-signature-only (upper bound on interventions vs live).
@bigboateng
bigboateng merged commit 376e35c into main Jul 20, 2026
6 checks passed
@bigboateng
bigboateng deleted the feat/settle-lab branch July 20, 2026 22:39
bigboateng added a commit that referenced this pull request Jul 21, 2026
…e subcommand (#78)

* fix(settle): restore the 'decide' subcommand (missing from #75 merge)

decide.go (decision-only stdin/stdout mode) didn't land in the #75 squash, but
the Harbor integration depends on it. Restores the file + wires the 'decide'
case in main.go. Build/vet/test green.

* docs(settle): detection-quality report + honest claim-tier reframe

Adds docs/05-detection-quality.md: fixture correctness (0 FP/FN), ~93% precision
at the redirect threshold via the recovery curve, ~10% firing rate on real
traces, deny rung dormant. Reframes Status/docs-04 from Being-evaluated to
Evaluated: net-neutral on Qwen/terminal-bench (workload not loop-bound); causal
uplift on loop-prone workloads (small models, online evals) = next experiment.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant