feat(settle): lab 18 — zero-inference partial-observation controller - #75
Merged
Conversation
settle is a new architecture (not lab 17 shipped): a standalone, compiled Go
binary that wraps any agent's command execution and intervenes with minimum,
nonblocking force when the loop stops making observable progress. Zero model
inference — all detection is hashing, counting, diffing, and regex.
Theory (docs/01): settle is the same Ramadge-Wonham supervisor as Boatstack,
operating in the regime where the spec is UNOBSERVABLE. Boatstack's spec is
co-designed observable (no false-positive rate); settle's ("do not remain in a
non-progressing cycle") is provably not — a futile loop and slow-but-genuine
progress can share the same projection P(s) — so by the Existence theorem no
exact supervisor exists. settle runs an approximate belief-state supervisor and
pays for unobservability with an irreducible FP rate, stated as a theorem.
Design consequences, aligned to repo-root supervisory-control.md:
- minimum force = the minimally-restrictive supervisor (supC(K)), also the EV>0 rule
- nonblocking: even a deny withholds one command and leaves read-only moves open
(fixes lab 17's swe_breaker exit(1) hard-block, which violated this)
- fail-OPEN, inverting Boatstack's fail-closed: an optimizer must never block a
healthy agent under sensor loss
Detection (docs/02): drops lab 17's F'=p(1-p) (decorative Bernoulli variance).
The real object is a non-marked recurrent class in the observation stream; one
cycle detector subsumes state-repeat (period 1), oscillation (period k), with
edit-distance-floor and entropy-collapse as leading indicators. Recurrence keys
on the JOINT (error signature, state hash), so a loop whose error repeats while
the workspace changes (hidden progress) is not flagged. Semantic-signature and
command-normalization ported to a single Go source of truth (was duplicated
across three drifting files in lab 17).
Surface: settle exec -- <cmd> (atomic unit), settle run -- <agent> (shell-shim
wrapper), settle doctor. Isolated — no runtime Pitot/Harbor dependency; Decision
maps 1:1 to Pitot control.response for a later bridge adapter.
Calibration (docs/04): scripts/calibrate.py mines 1,890 Harbor trajectories for
the recovery-probability curve. Finding: byte-identical repeated failing
commands are rare (validates the joint/semantic sensors over lab 17's literal
hash). Data hints k=2 on thin support (44); ships the conservative fallback k=5
transparently. Net effect on task success is Being-evaluated pending the
pre-registered paired experiment — not claimed.
Verified: go test ./... green (fixtures + determinism + canonicalization),
detector-innocence import boundary holds, module added to go.work.
settle replay -- <trajectory.json>... drives the detector over recorded ATIF-v1.7 agent trajectories and reports what settle would have done, without running anything. The safe/free observe mode for seeing behavior on real runs before any live wiring. Header caveats that offline replay has no workspace state, so it runs error-signature-only (upper bound on interventions vs live).
bigboateng
added a commit
that referenced
this pull request
Jul 21, 2026
…e subcommand (#78) * fix(settle): restore the 'decide' subcommand (missing from #75 merge) decide.go (decision-only stdin/stdout mode) didn't land in the #75 squash, but the Harbor integration depends on it. Restores the file + wires the 'decide' case in main.go. Build/vet/test green. * docs(settle): detection-quality report + honest claim-tier reframe Adds docs/05-detection-quality.md: fixture correctness (0 FP/FN), ~93% precision at the redirect threshold via the recovery curve, ~10% firing rate on real traces, deny rung dormant. Reframes Status/docs-04 from Being-evaluated to Evaluated: net-neutral on Qwen/terminal-bench (workload not loop-bound); causal uplift on loop-prone workloads (small models, online evals) = next experiment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
New lab 18 —
settle: a standalone, compiled Go binary that wraps any agent's command execution and intervenes with minimum, nonblocking force when the loop stops making observable progress. Zero model inference — all detection is hashing, counting, diffing, and regex.This is a new architecture, not lab 17 (
f-prime-governor) shipped. It keeps lab 17's defensible ideas (the EV inequality, minimum-force under partial observation, weak-sensor agreement) and dropsF' = p(1−p), which was decorative.Theory (
docs/01)settleis the same Ramadge–Wonham supervisor as Boatstack, operating in the regime where the spec is unobservable:settle's spec ("do not remain in a non-progressing cycle") is provably not observable — a futile loop and slow-but-genuine progress can share the same projectionP(s). By the Existence theorem, no exact supervisor exists, sosettleruns an approximate belief-state supervisor and pays for the unobservability with an irreducible FP rate (stated as a theorem, not a defect).Design consequences, aligned to repo-root
supervisory-control.md:supC(K)), which is also theEV > 0rule — two frames, one law.swe_breaker.py exit(1)hard-block, which violated this).Detection (
docs/02)The real object is a non-marked recurrent class in the observation stream. One cycle detector subsumes the five sensors: state-repeat = period-1, oscillation = period-k, with edit-distance-floor and entropy-collapse as leading indicators. Recurrence keys on the joint
(error signature, state hash), so a loop whose error repeats while the workspace keeps changing (hidden progress) is not flagged. Semantic-signature + command-normalization ported into a single Go source of truth (was duplicated across three drifting files in lab 17).Surface
Isolated — no runtime Pitot/Harbor dependency.
Decisionmaps 1:1 to Pitot'scontrol.responsefor a later bridge adapter.Calibration (
docs/04)scripts/calibrate.pymines 1,890 Harbor trajectories for the recovery-probability curve. Finding: byte-identical repeated failing commands are rare (agents mutate commands) — which validates the joint/semantic sensors over lab 17's literal hash. Data hintsk=2on thin support (44 sequences); ships the conservative fallbackk=5transparently, with the derived hint recorded incalibration.json.Claim discipline
Detection firing correctly on the golden fixtures = Observed. Net effect on task success = Being-evaluated until the pre-registered paired experiment (
docs/04). No invented numbers, no cross-model uplift claims.Verification
GOWORK=off go test ./...green — golden fixtures (always-fail, syntax-mutating, oscillation, genuine-progress, near-miss, hidden-progress), a determinism property test, and the canonicalization audit.settle doctorpasses; a livesettle execloop escalates inform → redirect → deny as designed.scripts/check_import_boundary.shholds (detector never imports exec).go.work;go vetclean across all three workspace modules.🤖 Generated with Claude Code