test(guards): make the guards prove what they claim about themselves

CHANGELOG-INTERNAL.md

@@ -31,6 +31,53 @@ a `### BREAKING` section placed FIRST in that version, and each such line is pre
 
 ### Tests
 
+- **The hot path's "zero allocations" rule got a gate.** New
+  `tests/test_hot_path_allocations.py` pins that `decide()` RETAINS nothing per packet: at most
+  64 blocks and 4096 bytes over 5000 calls, against a measured floor of 13 blocks and 608 bytes
+  that stays flat with duplication, latency, the NAT flow table and 200 ports. `test_hot_path.py`
+  guards the neighbouring rule (no syscalls on the packet threads); this catches the other class,
+  where something starts being kept once per packet.
+  🔴 **It reads two meters because one of them has a hole, and the hole was found by mutation
+  rather than by reasoning.** The first version counted `sys.getallocatedblocks()` only, and a
+  mutation making `decide()` append to an ever-growing list **survived**: the appended value was
+  a cached small int, so no object was created and blocks moved from 6 to 7. `tracemalloc`
+  current sees the same case as 42 032 bytes against 208. The meter-canary test now proves both
+  meters can rise, including the references-only case specifically. Still not caught, stated in
+  the file: transient garbage, since both meters are net.
+- **Size ceilings, as a ratchet.** New `tests/test_code_shape.py` caps a function at 167 logic
+  lines and a module at 1299 - today's maxima (`theme.py::init_style`, `gui/app.py`), so nothing
+  had to be rewritten to make it pass. Lowering them is routine work, raising either is the
+  owner's call. **Comments and docstrings do not count**, and that is measured rather than
+  assumed: 9 776 of the package's 17 770 lines are logic, so 45% is explanation, and a raw-line
+  cap would have been a cap on explaining. A second test pins that property directly (ninety
+  lines of comment measure the same as none). Both mutation-checked, plus the empty-scan canary.
+- **"Verified by mutation" became checkable.** New `tests/test_mutation_registry.py` holds the
+  claim as data in three states that say three different things: `MUTATIONS` (re-runnable now,
+  five entries), `PROVEN_BY_HAND` (a dated session did it, no patch was written down, so no
+  machine repeats it) and `NOT_PROVEN` (no mutation, said out loud). Until now the phrase
+  appeared over twenty times in the notes and in a dozen docstrings with nothing behind it -
+  convention 5's own evidence was the prose convention 5 warns about. The suite checks the
+  bookkeeping only: every named test exists, none is filed under two states, and every search
+  pattern still occurs exactly once, so a registry entry rots the day the code moves rather
+  than the day someone runs it. The mutations themselves run from `internal_tools/mutate.py`
+  (outside git, one subprocess suite run each), with a mandatory canary entry that must report
+  BROKEN - a tree that fails to compile also exits non-zero, so without it a whole run can
+  report "all caught" and mean nothing. First full run: 5 caught, 0 survived, canary BROKEN.
+- **The repository scanners now prove they read something.**
+  `test_repo_conventions.py::test_the_repository_scanners_actually_read_files` names one file
+  each collector must return plus a floor on the count, for all seven collectors across
+  `test_repo_conventions`, `test_code_hygiene`, `test_layering` and `test_readme_guards`. A
+  glob or walk that comes back EMPTY satisfies every assertion built on it and looks exactly
+  like a working guard, so renaming `beantester/` would have silenced four guards at once in
+  silence. Mutation-checked: emptying a collector and pointing the walk at a missing root each
+  turn it red.
+- **The whole-tree scans stopped measuring files that are not in the repository.**
+  New `repo_text_files()` collector skips `internal_tools/`, `.claude/`, `crashes/` and the
+  private notes, and `::test_the_repository_scanners_stay_out_of_what_is_not_in_the_repository`
+  keeps them out. Measured before the change: the dash scan covered 183 files, 12 of them
+  git-ignored, so the same test measured a different set locally than in CI - including
+  `crashes/latest-crash.txt`, whose text comes from OS exceptions. Convention 33's coverage of
+  the private notes moves to the Stop hook, which runs where those files exist.
 - `tests/test_gui_release_fixes.py::test_start_only_fields_are_locked_while_a_session_runs`
   rewritten to DERIVE its subjects from `fields.FIELD_DEFS` instead of naming `duration` and the
   filter combobox by hand, and to resolve each field to the surface that renders it

tests/test_code_shape.py

@@ -0,0 +1,136 @@
+"""Size ceilings for the package, as a RATCHET: the numbers may only go down.
+
+Why a ceiling here, when nothing has actually gone wrong yet
+-----------------------------------------------------------
+Nobody reads this code line by line. There is no reviewer to notice that a function
+reached three hundred lines over four sessions, each of which added twenty and each
+of which was reasonable on its own. A ceiling is the only mechanism that notices,
+and it costs nothing while nothing grows.
+
+Why LOGIC lines and not lines
+-----------------------------
+Measured before choosing the metric (2026-08-02): of 17 770 lines in ``beantester/``,
+only 9 776 are logic - **45% of this package is comments, docstrings and blank
+lines**. That is deliberate and it is where the measurements and the reasons live. A
+raw-line ceiling would therefore be a ceiling on EXPLAINING, pushing the next session
+to delete the paragraph that says why a constant is 0.30 rather than to simplify the
+function. So comments and docstrings are free; only executable lines count.
+
+Where the numbers come from
+---------------------------
+They are today's maxima, not a textbook figure: ``init_style`` at 167 logic lines and
+``gui/app.py`` at 1299. Nothing has to be rewritten to make this pass, which is the
+point - a ceiling picked out of the air either fails on day one or is set so loose it
+never fires.
+
+**Lowering these is ordinary work. Raising either is the owner's decision, not a way
+to get unblocked** - a threshold bent to fit the code has stopped being a threshold.
+When a function crosses it, the answer is to split the function.
+
+What this does NOT check
+------------------------
+Whether a function does one thing (a tidy twenty-line function doing three things
+passes exactly like a good one), whether its name is honest, or whether splitting it
+scattered the logic across ten places - that last one has its own cost and no metric.
+Nesting depth is not measured either. This guard buys one thing only: nothing grows
+past what a person can follow without somebody deciding that it should.
+"""
+import ast
+import os
+
+from fakes import ROOT, check
+
+# Today's maxima, measured 2026-08-02. Ratchet: down is routine, up is a decision.
+FUNCTION_CEILING = 167          # beantester/gui/theme.py::init_style
+FILE_CEILING = 1299             # beantester/gui/app.py
+
+
+def _logic_lines(source):
+    """Line numbers that carry executable code: no blanks, comments or docstrings.
+
+    Docstrings are found through the AST rather than by matching quotes, so a string
+    that merely LOOKS like one (a multi-line literal assigned to a name) still counts
+    as logic - which is right, because it is.
+    """
+    tree = ast.parse(source)
+    lines = source.splitlines()
+    doc = set()
+    for node in ast.walk(tree):
+        if (isinstance(node, ast.Expr) and isinstance(node.value, ast.Constant)
+                and isinstance(node.value.value, str)):
+            doc.update(range(node.lineno, (node.end_lineno or node.lineno) + 1))
+    live = set()
+    for number in range(1, len(lines) + 1):
+        text = lines[number - 1].strip()
+        if text and not text.startswith("#") and number not in doc:
+            live.add(number)
+    return tree, live
+
+
+def _package_files():
+    out = []
+    for dirpath, dirnames, filenames in os.walk(os.path.join(ROOT, "beantester")):
+        dirnames[:] = [d for d in dirnames if d != "__pycache__"]
+        out += [os.path.join(dirpath, n) for n in filenames if n.endswith(".py")]
+    return out
+
+
+def _measure():
+    """(worst function, worst file) as (name, count) pairs, plus how many files."""
+    worst_function = ("", 0)
+    worst_file = ("", 0)
+    paths = _package_files()
+    for path in paths:
+        tree, live = _logic_lines(open(path, encoding="utf-8").read())
+        rel = os.path.relpath(path, ROOT).replace(os.sep, "/")
+        count = len(live)
+        if count > worst_file[1]:
+            worst_file = (rel, count)
+        for node in ast.walk(tree):
+            if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
+                continue
+            span = sum(1 for n in live if node.lineno <= n <= (node.end_lineno or 0))
+            if span > worst_function[1]:
+                worst_function = (f"{rel}::{node.name}", span)
+    return worst_function, worst_file, len(paths)
+
+
+def test_no_function_or_file_has_grown_past_the_ratchet():
+    """Nothing in the package is longer than the longest thing was on 2026-08-02.
+
+    Crossing this is not a licence to raise the number. A function over the ceiling
+    gets split - which is exactly what happened to the one case another project hit
+    on the day it introduced the same guard.
+    """
+    worst_function, worst_file, seen = _measure()
+
+    # The canary from test_repo_conventions, applied here: a walk that finds nothing
+    # satisfies both ceilings perfectly and looks like a guard that works.
+    check("the shape scan actually read the package "
+          "(an empty scan passes every ceiling ever set)", seen >= 30, f"({seen} files)")
+
+    check(f"no function exceeds {FUNCTION_CEILING} logic lines "
+          f"(split it - do not raise the ceiling)",
+          worst_function[1] <= FUNCTION_CEILING,
+          f"(worst: {worst_function[0]} at {worst_function[1]})")
+    check(f"no module exceeds {FILE_CEILING} logic lines",
+          worst_file[1] <= FILE_CEILING,
+          f"(worst: {worst_file[0]} at {worst_file[1]})")
+
+
+def test_the_ratchet_measures_logic_and_not_explanation():
+    """Comments and docstrings must stay free, or the ceiling punishes the thing
+    this project is built on.
+
+    Without this, the cheapest way to get back under the limit would be to delete
+    the paragraph explaining why a constant is what it is - the exact opposite of
+    what convention 5 asks for. Checked directly rather than assumed: a body padded
+    with ninety lines of comment measures the same as the body alone.
+    """
+    bare = "def f():\n" + "    x = 1\n" * 5
+    padded = "def f():\n" + '    """Doc."""\n' + "    # note\n" * 90 + "    x = 1\n" * 5
+    _, live_bare = _logic_lines(bare)
+    _, live_padded = _logic_lines(padded)
+    check("ninety lines of comment and a docstring do not count as logic",
+          len(live_padded) == len(live_bare),
+          f"(bare {len(live_bare)}, padded {len(live_padded)})")

tests/test_hot_path_allocations.py

@@ -0,0 +1,164 @@
+"""The decision path must not RETAIN anything per packet.
+
+What this guards
+----------------
+"Zero allocations" has been written in the hot-path section of the notes for a long
+time and nothing checked it. ``test_hot_path.py`` guards the neighbouring rule - no
+syscalls on the packet threads - which is a different failure. This one catches the
+class where ``decide()`` starts holding on to something once per packet: a cache
+added for speed, a list that only ever grows, a per-flow object nobody frees. At the
+measured real rate of ~14k packets a second that is tens of megabytes a minute, and
+the symptom the user sees is not "slow" but a session that dies after an hour.
+
+Two meters, because one of them has a hole
+------------------------------------------
+🔴 **Blocks alone are not enough, and this was found by mutation, not by thinking.**
+The first version of this file counted ``sys.getallocatedblocks()`` only. A mutation
+that made ``decide()`` append to an ever-growing list **SURVIVED it**: the appended
+value was a small cached int, so no object was created and the list's array growth
+cost about one block. Blocks see a retained NEW OBJECT; they are blind to a container
+filling up with references to objects that already exist - which is the shape most
+real leaks in this codebase would take.
+
+So the gate reads both, and each covers the other's blind spot:
+
+* ``sys.getallocatedblocks()`` - net objects retained. Measured for the mutation:
+  7 against a baseline of 6, i.e. invisible.
+* ``tracemalloc`` current (not peak) - bytes still held. Same mutation: **42 032
+  against 208**, i.e. unmissable.
+
+What it still does NOT catch, said out loud
+-------------------------------------------
+**Transient garbage.** Both meters are net, so an object allocated and freed inside
+the same call nets to nothing - measured, not assumed. A future ``f"{a}:{b}"`` built
+per packet would pass here. CPython has no cheap deterministic counter of total
+allocations (Go's ``AllocsPerRun`` has no equivalent), and pretending otherwise would
+be the guard that promises more than it measures. Churn stays a job for measurement
+(rule 5), not for this gate.
+
+Why the numbers are ceilings and not zero
+-----------------------------------------
+Measured across configurations (2026-08-02, three runs each, gc collected then
+disabled, values identical every time): the pass-through path costs **13 blocks and
+608 bytes per 5000 calls** and does not move - not with 200 ports instead of 50, not
+with the NAT flow table armed, not with latency. Duplication takes it to 15 blocks
+and 656 bytes. Those are interpreter bookkeeping, not per-packet retention, so the
+ceilings sit far above them and far below a real regression: one retained reference
+per packet is 42 kB, one retained object is +5000 blocks.
+"""
+import gc
+import random
+import sys
+import tracemalloc
+
+from fakes import check
+
+from beantester.core import BeanCore
+
+CALLS = 5000
+BLOCK_CEILING = 64      # measured floor 13, flat across configurations
+BYTE_CEILING = 4096     # measured floor 608; a one-reference-per-packet leak is 42k
+
+
+def _decide_many(core, rng, count):
+    for i in range(count):
+        core.decide(100, True, 5000 + (i % 50), i * 0.001, rng,
+                    remote_ip="1.2.3.4", remote_port=443, is_tcp=True)
+
+
+def _cost_of(core, count=CALLS):
+    """(net blocks, net bytes) retained by ``count`` decisions, warmed and gc-quiet."""
+    rng = random.Random(7)
+    _decide_many(core, rng, 200)          # fill every lazy structure first
+    gc.collect()
+    gc.disable()
+    tracemalloc.start()
+    try:
+        blocks_before = sys.getallocatedblocks()
+        bytes_before = tracemalloc.get_traced_memory()[0]
+        _decide_many(core, rng, count)
+        return (sys.getallocatedblocks() - blocks_before,
+                tracemalloc.get_traced_memory()[0] - bytes_before)
+    finally:
+        tracemalloc.stop()
+        gc.enable()
+
+
+def test_the_meter_can_actually_see_retention():
+    """Prove the instrument before believing a zero from it.
+
+    A gate whose measurement is stuck at zero passes forever and reads exactly like
+    a clean hot path. This is the same reason the mutation runner carries a canary:
+    an instrument that cannot report failure is not evidence. Retaining 5000 objects
+    must show up as thousands of blocks.
+    """
+    if not hasattr(sys, "getallocatedblocks"):        # non-CPython: say so, loudly
+        raise AssertionError("this interpreter has no getallocatedblocks; the hot "
+                             "path allocation gate cannot run and must not be "
+                             "reported as passing")
+    gc.collect()
+    gc.disable()
+    tracemalloc.start()
+    try:
+        blocks_before = sys.getallocatedblocks()
+        bytes_before = tracemalloc.get_traced_memory()[0]
+        kept = [object() for _ in range(5000)]
+        blocks = sys.getallocatedblocks() - blocks_before
+        # The case that defeated the block meter: a container filling with
+        # references to an object that ALREADY exists. No object is created, so
+        # blocks barely move - only the bytes do.
+        references = []
+        bytes_before_refs = tracemalloc.get_traced_memory()[0]
+        blocks_before_refs = sys.getallocatedblocks()
+        for _ in range(5000):
+            references.append(100)          # a cached small int: nothing new is made
+        ref_blocks = sys.getallocatedblocks() - blocks_before_refs
+        ref_bytes = tracemalloc.get_traced_memory()[0] - bytes_before_refs
+        retained = tracemalloc.get_traced_memory()[0] - bytes_before
+    finally:
+        tracemalloc.stop()
+        gc.enable()
+
+    check("the block meter reports thousands when 5000 objects are retained "
+          "(a meter stuck at zero would pass the gate below forever)",
+          blocks >= 4000, f"(saw {blocks} for {len(kept)} objects)")
+    check("the byte meter reports thousands when 5000 objects are retained",
+          retained >= 4000, f"(saw {retained} bytes)")
+    check("the byte meter catches a container of REFERENCES, which the block meter "
+          "cannot see - this is the hole a surviving mutant exposed on 2026-08-02",
+          ref_bytes >= 4000 and ref_blocks < 100,
+          f"(refs: {ref_blocks} blocks, {ref_bytes} bytes)")
+
+
+def test_the_decision_path_retains_nothing_per_packet():
+    """A default engine judging 5000 packets holds on to nothing.
+
+    Pass-through is the configuration the tool spends most of its life in and the
+    one "collect, do not damage" promises is free (see test_passthrough.py).
+    """
+    blocks, retained = _cost_of(BeanCore())
+    check(f"decide() retains at most {BLOCK_CEILING} blocks over {CALLS} packets "
+          f"(one retained object per packet would be {CALLS})",
+          blocks <= BLOCK_CEILING, f"(retained {blocks} blocks)")
+    check(f"decide() retains at most {BYTE_CEILING} bytes over {CALLS} packets "
+          f"(a container growing by one reference per packet is about 42 kB, and "
+          f"the block count above cannot see it)",
+          retained <= BYTE_CEILING, f"(retained {retained} bytes)")
+
+
+def test_the_armed_gates_do_not_retain_per_packet_either():
+    """Impairment on is still not a licence to keep a copy of every packet.
+
+    Duplication is the one gate that legitimately hands a second packet onward, so
+    it is the honest worst case to point this at.
+    """
+    for label, arm in (("duplication", lambda c: setattr(c, "dup", 1.0)),
+                       ("latency", lambda c: setattr(c, "latency_s", 0.05)),
+                       ("nat flow table", lambda c: setattr(c, "nat_timeout_s", 30))):
+        core = BeanCore()
+        arm(core)
+        blocks, retained = _cost_of(core)
+        check(f"with {label} armed, decide() retains at most {BLOCK_CEILING} blocks",
+              blocks <= BLOCK_CEILING, f"(retained {blocks} blocks)")
+        check(f"with {label} armed, decide() retains at most {BYTE_CEILING} bytes",
+              retained <= BYTE_CEILING, f"(retained {retained} bytes)")