Skip to content

README.md

Latest commit

 

History

History
327 lines (254 loc) · 14.2 KB

README.md

File metadata and controls

327 lines (254 loc) · 14.2 KB
Raw

intelligence-flow

Intelligence Flow studies harnesses as networks: how useful signal moves, where it degrades, and which boundaries limit accepted output.

A harness is not a wrapper around an agent. A harness is an inspectable flow network for routing, measuring, constraining, and improving useful intelligence through a system. The model is only one node. The real system is the graph around it.

intelligence-flow is the harness/network-flow layer built on top of modelgraph. It is small on purpose: the priority is a clear theory, minimal primitives, and deterministic labs that connect back to that theory.

The harness equation

H = (S, N, E, T, Σ, R, V, F)
Symbol Name Meaning
S source The origin of the flow (task, question, objective, failing test, user intent).
N nodes Transformation boundaries. Defined by the boundary they expose, not by a domain label.
E edges Measured flow between nodes (context, state, evidence, plan, diff, decision, error signal).
T trace The observed execution record.
Σ signals Measurements on nodes/edges/paths/runs (cost, latency, error, relevance, confidence, relation preservation).
R relations Relations that should be preserved through the graph. Public language: relation checks.
V verifier / sink The acceptance rule for useful output (tests pass, answer supported, PR approved, human accepts).
F feedback Paths that route observation or error back into the system.

Simplified: H = (nodes, edges, traces, signals, relations, verifier, feedback).

source → nodes → edges → sink
         ↓       ↓
       trace   signals
         ↓       ↓
      relation checks
         ↓
      feedback

Full treatment in docs/01-theory.md.

Useful intelligence flow

Useful intelligence flow is not raw output. It is the amount of useful signal that reaches an accepted sink under constraints:

Φ(H) = useful signal reaching the accepted sink

A practical per-path score:

Φ(path) = Q(path) / C(path)

C(path) = α·tokens + β·latency + γ·money + δ·retries + ε·human_input + ζ·complexity
Q(path) = w1·correctness + w2·relation_preservation + w3·scope_control
        + w4·confidence  + w5·completeness

This is a practical scoring lens, not a mathematically complete theorem. It makes harnesses comparable; it does not claim optimality.

The network-flow lens

For an edge e = (u -> v):

flow(e)     = useful signal transferred from u to v
capacity(e) = how much useful signal the boundary can carry under constraints
cost(e)     = the cost paid to move signal across the boundary
loss(e)     = how much signal is degraded, corrupted, or made ambiguous
error(e)    = distance between expected relation and observed relation

bottleneck(H) = argmin capacity(e) over the relevant path / cut

This borrows the network-flow lens from optimization theory to make harnesses inspectable. We do not claim every AI harness is literally a classical max-flow problem — it is a practical systems model first. See docs/04-network-flow-lens.md.

Harness graph patterns

Four recurring topologies. Each has a runnable, deterministic demo in labs/ and is explained in docs/03-harness-graph-patterns.md.

1 — Single-Agent Wrapper

[Task] -> [Agent / LLM] -> [Output] -> [Verifier] -> [Accept?]

One path. Few observable boundaries. Bottleneck hidden inside the agent. Low trace resolution.

2 — Context-Aware Harness

[Task] -> [Context] -> [Agent / LLM] -> [Verifier] -> [Accept?]
             ^
             |
      [Repo / Docs / Memory]

Adds an inspectable context edge. Can measure context relevance, loss, stale memory. Common bottleneck: Context -> Agent.

3 — Feedback Harness

[Task] -> [Context] -> [Agent] -> [Verifier] -> [Accept?]
                          ^            |
                          +---fail-----+

Adds a feedback edge. Error becomes signal. Can improve, oscillate, or saturate.

4 — Parallel / Multi-Path Harness

                /-> [Path A] -\
[Task] -> [Shared Context]      -> [Compare / Verify] -> [Accept?]
                \-> [Path B] -/

Multiple paths carry candidate flow. Compare quality/cost/error and route to the path with the highest useful-flow score.

What can be inferred before implementation

The graph exposes tradeoffs before any agent code is written:

Pattern Pros Cons / new failure mode
Single-agent wrapper simple, low coordination cost low observability, hidden bottlenecks
Context-aware harness better input shaping, measurable context quality wrong / stale context
Feedback harness enables improvement needs good error signals; can oscillate or saturate
Parallel harness explores alternatives, higher chance of accepted output higher comparison and coordination cost

Why this matters

Most AI systems are evaluated at the final output. But final output alone hides where useful intelligence was created, lost, or corrupted. A graph harness lets you ask: what entered the system, which path it took, which node transformed it, which edge degraded it, which relation was preserved or broken, which verifier allowed or blocked it, which feedback loop improved or saturated, and which path produced the most accepted output per cost.

Local output is not system intelligence

A single model call is just input → model → output. A harness is source → context → work node → verifier → feedback → accepted sink. The intelligence we care about is not only inside the node — it is in the flow through the graph.

Relationship to modelgraph

intelligence-flow is built above modelgraph, not as a replacement. Three layers:

modelgraph              (primitive)   transforms, traces, evaluators, graph runs
composable-model-graph  (proofs)      relation checks, emergent system failures, run comparison
intelligence-flow       (harness)     edges, path comparison, bottlenecks, feedback, useful-flow scoring
modelgraph answers intelligence-flow answers
What transforms ran? Which path carried useful signal?
What outputs did they produce? Where did useful signal degrade?
What trace exists? Which edge limited the system?
Did evaluation pass? Which feedback route improved the run? Which topology extracted more useful intelligence?

All modelgraph imports flow through one module — src/modelgraph-adapter.ts — so intelligence-flow stays a clean layer above it. See docs/06-relationship-to-modelgraph.md.

First principles

  1. A harness is a graph, not a wrapper.
  2. Nodes are transformation boundaries, not fixed categories.
  3. Edges are measured flow, not just connections.
  4. Traces make behavior inspectable.
  5. Signals quantify quality, cost, error, and degradation.
  6. Relations define what must be preserved.
  7. Verifiers define accepted output.
  8. Feedback turns error into a new path.
  9. Bottlenecks reveal where to improve.
  10. Better harnesses route more useful signal to the sink under constraints.

Layout

src/
  index.ts              public surface
  theory.ts             H = (S,N,E,T,Σ,R,V,F), Φ = Q/C, cost/quality weights
  flow.ts               nodes, linear paths, runs, the sink, signal lifting
  graph.ts              createFlowGraph — branch/merge harness spec (nodes + edges)
  edge.ts               FlowEdge, EdgeFlowSignal
  signals.ts            FlowSignal + extractors
  relations.ts          relation checks + trace-relation localization
  path.ts               path cost / score / signature
  compare.ts            comparePaths, bestPath
  bottleneck.ts         findBottleneck -> BottleneckResult
  feedback.ts           runWithFeedback
  diagram.ts            printFlowGraph + canonical pattern art
  modelgraph-adapter.ts the single modelgraph boundary
docs/                   01-theory … 19-product-value-projection
labs/                   01 single · 02 context · 03 feedback · 04 parallel ·
                        05 diagram · 06 terminal-runner · 07 terminal-live ·
                        08 nonlinear · 09 optimizer · 10–11 Terminal-Bench ·
                        12 evidence-engineered coding · 13 autoresearch ·
                        14 product-value-projection
labkit/                 repo-wide tooling: project a lab into its own public repo
python/                 the intelligence_flow Python package (flow algorithms)

Graduating a lab to a public repo. labkit/ is a standalone, config-driven toolkit that projects a labs/* project into its own public GitHub repository (surface + projection engine, workflow generator, scaffolder, and a drift doctor). One labs/<lab>/publish.config.json per lab is the single source of truth. Run python -m labkit doctor from the repo root; see labkit/README.md.

Execution model

A harness runs in three shapes: a linear path (default), a branch/merge graph (createFlowGraph with nodes + edges — several edges out of a node is a branch, several into a node is a merge), and a bounded loop on top via runWithFeedback. Branch/merge runs inside one FlowRun, so the diagram, scoring, and bottleneck tools work unchanged; loops stay in the harness layer so the modelgraph core stays barebones. Full treatment in docs/07-execution-model.md; runnable proof in labs/08-nonlinear-harness.

The harness-neutral evidence-engineered coding node is in labs/12-product-engineering-loop. It keeps implementation tactics open while converting product intent into a human-approved, hash-locked plan; compiles task/test/evidence artifacts; applies test, review, and ship gates; and exports thin Cursor, Claude Code, Codex, Gemini, and GitHub adapters from one canonical workflow.

The standalone Product Value Projection operator is in labs/14-product-value-projection. It turns an exact external message plus a minimal repository slice into a traceable Product Value Map. It is a research lab and command contract, not part of the Boatstack distribution.

Diagram printer

One of the first practical features in intelligence-flow is the ability to print the harness graph. This matters because harnesses become debuggable only when the structure is visible. A diagram can show paths, bottlenecks, feedback loops, measured edge signals, cost and latency, relation-check failures, and accepted/rejected sinks.

The point is not visualization for its own sake. The point is reasoning. From the graph alone, we can often infer tradeoffs before writing agent code: single-agent (simple, cheap, low observability), context-aware (better input shaping, but context quality becomes the bottleneck), feedback (can improve over retries, but can oscillate or saturate), parallel (can route around weak paths, but adds comparison and coordination cost).

A harness graph should be printable before it is executable.

import { printFlowGraph, fromFlowPath, printLinearPath } from "intelligence-flow";

printFlowGraph(graph);            // static architecture
printFlowGraph(fromFlowPath(path), run); // overlay measured signals + bottleneck
[context]
  |
  | cost=1200tok latency=0ms relevance=0.61
  v
[agent]
  |
  | cost=40tok accepted=true
  v
[Accept]

Bottleneck:
  context
  reason: highest cost (1200tok) on the path

Runnable demo of all four patterns: labs/05-diagram-printer.

Run the labs

pnpm install        # resolves the modelgraph github dep and builds it
pnpm typecheck
pnpm labs           # runs the deterministic labs; each self-verifies

Each lab is an executable research project: it states the hypothesis and boundary, prints its graph pattern, path, signals, accepted/rejected verdict, inferred bottleneck, and a short explanation, then runs a self-check. Expected output is committed next to each lab as expected-output.txt.

Development & Release Standards

Documentation Writing Standard

Documentation and website copy follow Simplified Technical English (ASD-STE100) — a controlled-language standard for clear, unambiguous, translatable technical writing. The house rules distilled from it (prefer one meaning per term, short active-voice sentences, one instruction per step) live in simplified-technical-english.md. The full ASD-STE100 specification is copyrighted by ASD and is not reproduced here; get the authoritative document from https://asd-ste100.org.

Published Artifacts Must Speak Software

All published and projected artifacts (including the boatstack and pitot distributions) must "speak software." This means they must be accompanied by precise, programmatically verifiable, and append-only release note fragments (stored under their respective release-notes/ directories).

  • Structured Fragments: Release notes are not fuzzy human logs; they are structured software files requiring level-three Markdown headings (### <heading>) on the first line and structured description of impact.
  • Strict Compliance: Any change that modifies a published module or lab requires a new, dated, append-only release note fragment. This rule is programmatically enforced by release_notes.py check-policy and preflight checks in CI.
  • Downstream Fidelity: This ensures that downstream synchronization workflows and final tagged GitHub releases can present a high-fidelity, machine-readable record of changes without manual reconstruction or developer guesswork.

What this is not

  • Not a claim that every harness is a classical max-flow problem.
  • Not a large orchestration framework — the primitives stay small.
  • Not a reimplementation of modelgraph.
  • Not domain-locked: node kinds are not hardcoded, and the core attaches no meaning to signal keys.