Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion agent_assembly/adapters/_shared/tool_governance.py
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ async def _record_async_tool_result(
HTTP boundaries, this function therefore finds no hook and emits nothing on
the shipped path β€” for allowed calls as much as denied ones.

Under ADR 0033 Β§6 that makes SDK-side recording **Planned** (AAASM-5731),
Under ADR 0033 Β§6 that makes SDK-side recording **Planned** (AAASM-5750),
not *Unmeasured*: Β§6 reserves ``Unmeasured`` for an action no control
inspected, where nothing is known, and here exactly where the record stops
has been measured. It is certainly not *Observed*, which needs a durable
Expand Down
2 changes: 1 addition & 1 deletion agent_assembly/core/audit_sink.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
declares its disposition; :func:`resolve_audit_sink` reads it; ``init_assembly``
warns about it and reports it on the returned context.

Under ADR 0033 Β§6 this makes SDK-side recording **Planned** (AAASM-5731), not
Under ADR 0033 Β§6 this makes SDK-side recording **Planned** (AAASM-5750), not
*Observed* β€” *Observed* requires a durable event attributed to the action, and
there is none. It is deliberately not *Unmeasured*: Β§6 reserves that for an
action no control inspected, where nothing is known, and here exactly where the
Expand Down
187 changes: 187 additions & 0 deletions test/unit/core/test_planned_referent.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,187 @@
"""AAASM-5750 β€” AAASM-5731 must not be the referent of a forward-looking claim.

The rule this gate enforces comes from **AAASM-5750's own description**, not from
ADR 0033 Β§6. Β§6 requires that ``Planned`` carry *a* ticket reference and says
nothing about which ticket; naming the right one is 5750's decision. Stating the
source precisely matters, because a failure message citing an ADR for a rule the
ADR does not contain sends the next reader to the wrong document.

The defect: AAASM-5731 measured that no interceptor this SDK ships resolves an
audit hook. It never intended to build a sink, and it is closed. A
forward-looking claim pointing at it reads as a live commitment while resolving
to finished work. So the invariant is narrow and permanent β€” **AAASM-5731 may be
cited as the ticket that measured the gap, never as the ticket that will fix
it.**

The assertion is two-tier, because one tier alone fails in one direction or the
other and review caught both:

* a **guarded** site (one of :data:`EXPECTED_SITES`) must name AAASM-5750
exactly. Without this the gate stops asserting the thing the change made true
β€” repointing a guarded site to any other live ticket passed green, which is
precisely the drift the gate exists to catch.
* **any other** site must merely not name a stale referent. Asserting
AAASM-5750 repository-wide was the first version's defect: Β§6 scopes
``Planned`` to any decided-but-unbuilt capability with any ticket, so an
unrelated roadmap row β€” including
``docs/examples/framework-support.md``'s docs-area maturity label, a
different axis entirely β€” is legitimate and must not fail this gate.

Two limits are disclosed rather than fixed, both measured as currently
unreachable:

* the reachability check is per **file**, not per site. A guarded file that
reflowed its real site out of the scan's reach *and* gained a second, correct
claim would keep its entry. Requires two coordinated edits; today no file in
this repository carries more than one site.
* the gate file is excluded from its own scan, so it is a hiding place for a
stale referent. It is a test file that documents no SDK behaviour, and the
exclusion matches one exact path rather than a prefix.
"""

from __future__ import annotations

import re
from pathlib import Path

#: The ticket that owns building the SDK-side audit sink. Guarded sites must
#: name it exactly.
CAPABILITY_REFERENT = "AAASM-5750"

#: Tickets that *measured* the absence of an SDK-side audit sink. Backward
#: citations to them are correct and are left alone; what this gate forbids is
#: either one appearing as the ticket a forward-looking claim defers to.
STALE_REFERENTS = frozenset({"AAASM-5731", "AAASM-5681"})

#: The two shapes a deferral takes: the ADR 0033 Β§6 term, and the plain
#: "tracked as" pointer used where no term is stated.
_FORWARD_CLAIM = re.compile(r"\bPlanned\b|tracked as")
_TICKET_REF = re.compile(r"AAASM-\d+")
_SCANNED_SUFFIXES = frozenset({".py", ".md"})
_SKIPPED_DIRS = frozenset({".git", ".venv", "node_modules", "__pycache__", "build", "dist"})

#: Excluded from its own scan. Including it was the first version's defect: this
#: file names AAASM-5750 in its own docstring, which padded the site count and
#: let a floor be satisfied by the gate quoting itself.
_GATE_FILE = "test/unit/core/test_planned_referent.py"

#: Audit-sink deferrals that must remain reachable by the scan. A fixture
#: compared against a walk of the tree, not a constant compared against another
#: constant: if a site is deleted, renamed, or reflowed out of the scan's reach,
#: the walk stops finding it and this fails.
EXPECTED_SITES = (
"agent_assembly/core/audit_sink.py",
"agent_assembly/adapters/_shared/tool_governance.py",
"test/unit/test_quickstart_negative_control.py",
)


def _repo_root() -> Path:
"""Walk up to the directory holding ``pyproject.toml``.

Resolving the root rather than assuming a relative path keeps the scan
repository-wide regardless of the directory pytest is invoked from β€” a
package-relative path silently narrows the scan to whatever is below the
caller.
"""
for candidate in [Path(__file__).resolve(), *Path(__file__).resolve().parents]:
if (candidate / "pyproject.toml").is_file():
return candidate
raise AssertionError(
"no pyproject.toml found above this test; the scan would cover nothing and pass for the wrong reason"
)


def _ends_sentence(line: str) -> bool:
"""Whether a comment line closes a sentence.

A wrapped sentence (``… Planned under ADR 0033 Β§6`` / ``(AAASM-5750) …``)
does not; a complete one does.
"""
trimmed = line.strip().rstrip("#* ")
return bool(trimmed) and trimmed[-1] in ".!?"


def _deferral_sites() -> list[tuple[str, int, str, str]]:
"""Every forward-looking claim paired with a ticket.

The ticket is looked for on the claim's own line **and the line after it**.
One line is not enough: ``test/unit/test_quickstart_negative_control.py``
wraps ``Planned`` and its ticket onto separate lines, and the first version
of this scan silently skipped it while the PR claimed it was covered.
Whether a site is checked must not depend on where a comment happens to
wrap.
"""
root = _repo_root()
sites: list[tuple[str, int, str, str]] = []

for path in sorted(root.rglob("*")):
if not path.is_file() or path.suffix not in _SCANNED_SUFFIXES:
continue

rel = path.relative_to(root)
if _SKIPPED_DIRS.intersection(rel.parts) or str(rel) == _GATE_FILE:
continue

lines = path.read_text(encoding="utf-8", errors="replace").splitlines()
for index, line in enumerate(lines):
if not _FORWARD_CLAIM.search(line):
continue

# Extend to the next line only when this line carries no ticket of
# its own AND does not end a sentence. Without the sentence guard
# the window pairs a claim with a ticket belonging to the *next*
# sentence β€” review produced a real case where an inserted line of
# forward-looking prose was blamed for a correct backward citation
# beneath it. There are 33 such backward citations in this repo.
window = line
if not _TICKET_REF.search(line) and not _ends_sentence(line) and index + 1 < len(lines):
window = f"{line}\n{lines[index + 1]}"

ticket = _TICKET_REF.search(window)
if ticket is None:
continue

sites.append((str(rel), index + 1, ticket.group(0), line.strip()))

return sites


def test_forward_claims_name_the_right_ticket() -> None:
guarded = set(EXPECTED_SITES)
problems = []

for path, lineno, ticket, text in _deferral_sites():
if path in guarded:
if ticket != CAPABILITY_REFERENT:
problems.append(
f"{path}:{lineno} is a guarded audit-sink deferral and must "
f"name {CAPABILITY_REFERENT}, not {ticket} β€” this is the site "
f"the referent change corrected, and letting it drift to any "
f"other ticket is what this gate exists to prevent: {text}"
)
elif ticket in STALE_REFERENTS:
problems.append(
f"{path}:{lineno} defers to {ticket}, which measured the gap and "
f"will not fix it β€” use the ticket that builds the sink "
f"(AAASM-5750, per its own description): {text}"
)

assert not problems, "\n".join(problems)


def test_every_expected_site_is_still_reachable() -> None:
"""Anti-vacuity, and the reason it names paths rather than counting.

A count can be held up by an unrelated site appearing as a real one is
deleted. Naming them makes that substitution visible.
"""
seen = {site[0] for site in _deferral_sites()}
missing = [path for path in EXPECTED_SITES if path not in seen]
assert not missing, "\n".join(
f"{path} carries no forward claim the scan can pair with a ticket; it "
f"was deleted, renamed, or reflowed so the term and the ticket are more "
f"than one line apart β€” in which case a stale referent there would no "
f"longer be checked"
for path in missing
)
2 changes: 1 addition & 1 deletion test/unit/test_quickstart_negative_control.py
Original file line number Diff line number Diff line change
Expand Up @@ -300,7 +300,7 @@ def test_a_denied_call_emits_an_audit_record_carrying_the_agent_and_tool(
# (RuntimeQueryInterceptor + GatewayClient expose neither
# record_result nor on_tool_end), so tool outcomes produce no audit
# evidence on the shipped path β€” Planned under ADR 0033 Β§6
# (AAASM-5731), not Unmeasured, since where the record stops has been
# (AAASM-5750), not Unmeasured, since where the record stops has been
# measured. What this pins is the governance flow's call β€” the part
# fixable without wiring a sink.
assert len(quickstart.interceptor.records) == 1
Expand Down