Skip to content

[AAASM-5528] 📝 (docs): Bound absolute enforcement claims to verified paths - #135

Merged
Chisanan232 merged 5 commits into
mainfrom
v0.0.1-rc.7/AAASM-5528/remove_absolute_claims
Aug 6, 2026
Merged

[AAASM-5528] 📝 (docs): Bound absolute enforcement claims to verified paths#135
Chisanan232 merged 5 commits into
mainfrom
v0.0.1-rc.7/AAASM-5528/remove_absolute_claims

Conversation

@Chisanan232

@Chisanan232 Chisanan232 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Description

Corrects public claims on the two hub pages this ticket owns — the landing page and the competitor comparison — so they stop asserting guarantees the implementation does not support.

Four clusters:

  1. "It works across your whole fleet of agents and does not require you to rewrite your existing agent code." Governance is per agent and per path. An agent launched outside a governed path is not governed at all — a demonstrated bypass, not a theoretical one.
  2. Layer descriptions. The SDK does not decide "before any network request leaves the process" (it gates wrapped framework tool calls; raw HTTP, subprocess spawns and file access are not intercepted); the proxy is not "no code changes" unconditionally; eBPF detects rather than catches, on Linux x86_64 with OpenSSL.
  3. "A security checkpoint in front of every agent action." Scoped to governed actions.
  4. The audit row and differentiator. "Immutable audit log with tamper-evident signatures ✓ 🚧 (HMAC-SHA256)" was wrong in both directions — see below.

Type of Change

  • 🐛 Bug fix (correcting inaccurate or missing content)

Related Issues

  • Related Jira ticket: AAASM-5528 (P0 Bug)
  • Parent Epic: AAASM-5526
  • Attached Goal: CBLPCRLM-13 — Verified Product Truth and Protection Boundaries
  • Fix version: agent-assembly v0.0.1-rc.7
  • Related: AAASM-5626 (best-effort audit emission)

Documentation Checklist

  • cd docs && mdbook build passes locally with no new warnings
  • All internal cross-links verified
  • Page registered in docs/src/SUMMARY.md — N/A, no new pages
  • Self-hosted instructions not included (SaaS-only scope) — none added
  • "Last reviewed" footer updated with today's date — comparison.md updated to 2026-08-06
  • Commits follow GitEmoji convention and are small / atomic — 2 commits, one page each

The audit claim — corrected in both directions

This is the part most worth a careful read, because the previous text was simultaneously an over-claim and an under-claim.

Over-claimed. The chain is an unkeyed SHA-256 hash chain (aa-core/src/audit.rs:10,713use sha2::{Digest, Sha256}; there is no hmac import anywhere in aa-core, and the only HMAC in the repo is unrelated JWT/webhook signing). "HMAC-SHA256" is factually wrong and "signed" is wrong: an unkeyed chain is tamper-evident, not a signature — anyone able to rewrite the log can recompute the chain. "Immutable" is also false: retention pruning deletes rows (aa-gateway/src/storage/sqlite.rs:715, postgres.rs:854). The chain does not cover the database sink at all — the conversion explicitly drops seq/previous_hash/entry_hash (aa-gateway/src/storage/audit_bridge.rs:10-12). Emission is best-effort, and seq/last_hash commit before the send, so a dropped entry is indistinguishable from tampering.

Under-claimed. The 🚧 marker presented this as a planned Enterprise capability. It is not — AuditWriter::verify_chain (aa-gateway/src/audit.rs:142) ships today as aasm audit verify-chain (aa-cli/src/commands/audit/mod.rs:14,31,44), in the open-source build. Understating is as inaccurate as overstating, so the capability is stated plainly rather than deleted.

Wording is deliberately aligned with what AAASM-5612 is publishing on docs/src/security-model.md in #134 ("per-session JSONL event log with an unkeyed SHA-256 hash chain … append-only by convention and best-effort on emission"), so the hub does not ship two different descriptions of one mechanism. The comparison page links to security-model.md#audit-log for the full bounds.

Architecture implications

None. No decision is recorded or changed; the corrections align the hub with ADR 0002 and ADR 0030, which already establish the correct guarantee level. ADR 0033 (canonical architecture) is open under independent review in agent-assembly #1951 — this PR deliberately does not restructure anything around it.

Product-truth implications

  • The matrix keeps its tick marks where the capability genuinely exists, but two enforcement rows now carry footnotes stating their assumptions rather than an unqualified ✓.
  • "Kernel-level bypass detection (eBPF)" already said detection and was left as ✓; the differentiator prose beneath it, which claimed "kernel-level enforcement", was corrected.
  • The audit row moved from ✓ 🚧 to partial, matching the legend's "limited" sense, with the reasoning in a footnote.
  • Deep detail was added, not removed: the footnotes carry the transport, host-selection, CA and platform constraints that the one-line rows cannot.

Security considerations

Strictly conservative — every edit narrows a claim a security reviewer might otherwise rely on. The audit correction is the most consequential: a reader evaluating PCI-DSS / SOC 2 posture would previously have believed the log was signed and immutable. It is neither.

Scope / out of scope

  • In scope, owned: docs/src/README.md, docs/src/comparison.md.
  • Not touched, owned by AAASM-5612 concurrently: docs/src/quickstart-saas.md, docs/src/cloud-deployment.md, docs/src/open-core-boundary.md, docs/src/security-model.md.
  • Not touched: docs/src/compatibility.md (generated).

Dependency mapping

No file overlap with AAASM-5612's PR #134 on this repo — that PR owns security-model.md, which this PR only links to. Verified by reading #134's branch before writing the audit wording.

Testing / CI / Sonar / coverage expectations

Gate Command Result
Book build mdbook build docs pass
Compatibility drift python3 docs/scripts/generate_compatibility.py --check pass
Hub-component drift python3 docs/scripts/generate_hub_components.py --check pass
Stale repo names python3 docs/scripts/check_repo_names.py pass
Footnote rendering grep built comparison.html 7 footnote nodes render

Re-grep of the built book (docs/book/index.html, docs/book/comparison.html) for catch everything, nowhere to hide, whole fleet, no code changes, every agent action, cannot bypass, HMAC-SHA256, immutable audit, signed with returns zero hits.

This repo has no markdownlint or lychee workflow (aggregate.yml, hub-metadata-check.yml, modules-ref-check.yml only) and no Node toolchain, so mdbook build plus the three drift checks are the complete local gate set. Sonar and coverage: not configured for this repo; no change expected.

Jira links validated

AAASM-5528, AAASM-5526 and AAASM-5626 resolve.

Migration behavior / rollback

No migration. git revert of either commit is independent and safe.

Known limitations of this PR

  • The claim inventory artifact lives in the agent-assembly repo, not here, because that repo owns the capability matrix and the protection ladder the corrected copy is measured against. Path: verification-reports/AAASM-5528-public-claim-inventory.md.
  • The landing page and comparison page link to the core limitations page by absolute URL (https://docs.agent-assembly.com/core/latest/devtools/limitations.html) because the hub mounts core under /core/latest/ via the aggregation pipeline rather than as a relative mdBook page. Verified live with lychee — 0 errors.

Related PRs — merge order

Same ticket, three repositories:

  1. ai-agent-assembly/agent-assembly[AAASM-5528] 📝 (docs): Bound absolute enforcement claims to verified paths agent-assembly#1952 (merge first; carries the claim inventory and owns the limitations page this PR links to)
  2. This PRai-agent-assembly/docs
  3. ai-agent-assembly/official-website[AAASM-5528] 📝 (website): Bound absolute enforcement claims to verified paths official-website#90

Order is a preference, not a hard dependency — the linked page is already live.


Review round 2 — changes applied

Independent review returned CHANGES-REQUIRED. Every blocker and major is addressed; the corrections are summarised here so a re-reviewer can check them without re-reading the diff.

Two new falsehoods this ticket had introduced, in the understating direction

  1. "CA trust-store install is macOS-only" was false. aa-cli/src/commands/proxy/ca.rs:79-82 dispatches to install_linux, and :150-188 is a complete implementation — it copies to /usr/local/share/ca-certificates/aa-proxy.crt and runs update-ca-certificates; uninstall_linux is at :191, both wired at proxy/mod.rs:26,45. Windows is the unsupported platform (ca.rs:87). What is macOS-only is the automatic install at proxy start (aa-proxy/src/lib.rs:64-69). I had searched only the aa-proxy crate and generalised from its cfg(target_os) guards — precisely the defect class this ticket exists to remove, and ADR 0033 §5.3 already had it right. Corrected in every location, across all three repos.
  2. eBPF "Linux x86_64 only" over-generalised. There is no cfg(target_arch) or runtime arch check anywhere in the eBPF crates. TLS uprobes attach by symbol from /proc/<pid>/maps and exec tracepoints resolve offsets from live BTF — both work on aarch64. Only the file-I/O kprobes are x86_64 (14 hardcoded __x64_sys_*, kprobe.rs:145-160). requirements.md had also become self-contradictory (:24 vs :80).

Both mattered: a Linux evaluator reading "macOS only" would discard the most deployable enforcement mechanism in the product.

The SDK is advisory, not an enforcement point

Copy said "the SDK and proxy can deny an action before it runs". Only the proxy does. aa-sdk-client has no in-tree caller that refuses (decision.rs:32-33: "The SDK remains advisory … not the primary gate") and query_policy is a voluntary UDS call a non-cooperating process never makes (client.rs:247-279). ADR 0002 is the precedent; ADR 0033 restates it.

This reconciles with — rather than contradicts — the earlier finding that the language wrapper raises before the wrapped body: it does, for a caller that chooses to honour the verdict. Recorded as evidence block E7. Public register is now: proxy denies before execution · SDK evaluates (advisory) · eBPF observes/detects.

"Immutable audit" survived on pages this ticket rewrote

Five sentences kept the word while this PR's own evidence block E5 records it as false (retention pruning deletes rows — sqlite.rs:715, postgres.rs:854). concepts.md was internally inconsistent: :46 said "every decision" twelve lines above the :58 line already corrected. All now match PR #134's wording.

How it got past the first pass, and what changed: the built-output re-grep reported clean because immutable was not on the banned-word list. A token list only finds tokens you already thought of. Re-verification is now a class-based scan — absolute quantifiers, crypto/permanence, observe-as-prevent, no-code-change, platform overreach, unshipped features — run over the built artifact.


Review round 3 — changes applied

Three wrong public claims remained; all three are fixed, plus the reporting defects.

  • M1 — eBPF still "catches" on the security page. three-layer-defense.md:26 said the kernel layer "catches actions the higher layers never saw". The identical sentence had been corrected on the sibling three-layer-model.md and missed here — on the page a security evaluator actually reads. Now "can report". Neither copy contains a banned token, which is exactly why no vocabulary or class scan surfaced it. Row A36.
  • M2 — surviving absolute on the landing blockquote. introduction/overview.md:19 still said "review exactly what every agent did and why", three lines below the corrected statement that unrouted paths need their own controls, and the same claim already removed at A9/A21/A25. Now "review exactly what was observed and decided." Row A37.
  • M3 — the macOS CA claim was wrong on all 8 surfaces, and it was my round-2 fix that made it wrong. aa-proxy/src/lib.rs:64-69 attempts the install at start, gated only on !ca.is_installed()?; it calls tls/keychain.rs:16-42, which shells out to security add-trusted-cert and requires admin authorization — macOS prompts, and because the call propagates with ?, a refusal fails proxy startup. "Installed automatically" reads as silent and unattended; a macOS CI runner or non-admin developer following the docs got a non-zero exit and a keychain error the docs gave no reason for. Corrected in the same substance on all eight surfaces. Row A38, evidence E2 rewritten.

This is the same defect as B2, committed inside the fix for B2 — a platform claim generalised from a cfg(target_os) guard without reading what the guarded call requires. It is now written into the artifact's method note so the next person does not repeat it. ADR 0033 made the mirror-image error ("opt-in … not automatic"); that is AAASM-5638 and is being corrected in parallel — I did not touch the ADR.

Reporting and hygiene

  • Inventory recounted and all three PR bodies brought into line. The artifact is now 69 rows · 8 remove · 61 qualify, split 18 website / 38 core / 13 docs. Every number in every body is regenerated from that count, and the self-assertion that the bodies already matched is deleted rather than re-asserted.
  • Link-check exits 1 on six changed pages, not five — docs/src/README.md joined the changed set. Two distinct targets across the six: architecture/index.md (11 references) and introduction/index.md (2). Verified pre-existing at the merge base a1711c7a and on remote/main; out of scope, and my new introduction/concepts.md#audit link resolves.
  • Rows added during review now carry (branch position) — A35, A37, A38, W18, D13 — because the Method section says line numbers are base-commit, and their subject text did not exist at base.
  • W11–W13 now cite the full blog slug instead of a literal ellipsis.
  • Out-of-scope findings recorded, not edited: governance-limits/claude-code.md calls eBPF uprobes "the only enforcement path" for file read/write (same class as A7/A36 — needs a follow-up under AAASM-5526), and protocol/CHANGELOG.md's historical "immutable audit record" stays, because a changelog records what was said at the time.

Method note extended

Three failure modes from this round are now recorded in the artifact: a cfg guard tells you where code compiles, not what it needs; correcting one instance of a sentence does not correct its copies; and assert over the rendered artifact, not the source string.

Re-verification

Class scan of built output: 0 hits in every class across all three repos — the single immutable is the corrective "not immutable". A new stray-markup scan over rendered text returns 0 segments. Browser re-run: 19 rows, 0 problems, 16/16 page-viewport-theme combinations resolve exactly one limitations.html link, 14 tab stops with 0 missing focus indicators, no console errors. All three branches advanced as fast-forwards; no force-push.

M3 follow-up — aligned with ADR PR #1955

The ADR half of this correction (AAASM-5638, agent-assembly #1955) and these eight surfaces now say the same thing. I re-derived the chain from source rather than from the paraphrase: pub async fn run (aa-proxy/src/lib.rs:41) executes the macOS block unconditionally at :62-67, gated only on !ca.is_installed()?; CaStore::install (tls/ca.rs:215) no-ops if already trusted, else calls keychain::add_trusted_cert (:219); that shells out to security add-trusted-cert (tls/keychain.rs:18, Command::new("security") at :23-32) and requires admin authorization (:16); and ca.install()? propagating out of run is why a refused prompt fails proxy startup.

All eight surfaces already carried the three non-negotiables — automatic attempt, admin authorization required, refusal fails startup — so no wording changed this round. The two website strings state them without file:line, at marketing altitude; the pages that already cite code keep their citations.

On the "does not rely on" / "does not use" distinction: no surface of mine made that claim about the trust store, so nothing needed changing. The two do not use matches in the docs repo are about the SDK ("agents that do not use the SDK"), which is unrelated and correct.

Checking it did, however, surface a new finding, recorded but not edited: HOST_ENFORCEMENT_REASON (aa-devtool-claude-code/src/lifecycle.rs:657-659), which is user-visible CLI output, states that Agent Assembly "never adds its certificate authority to the macOS system trust store". That is true of the integration path — which establishes trust per-launch via NODE_EXTRA_CA_CERTS — and false at product scope, because the proxy binary attempts exactly that install at startup. It is Rust source outside this ticket's changed set, so it is in the inventory's out-of-scope table and needs a follow-up under AAASM-5526.

Citation-provenance audit. Prompted by the ADR audit's find that a pre-existing citation pointed at aa-proto/_embedded/proto/audit.proto — gitignored, produced by build.rs, resolving only on a tree where someone had built — I checked every source path cited in the inventory against git ls-files and git check-ignore. All are committed; none is generated or build-produced, so there is nothing to mark. The method note now records the rule, because whether a generated path resolves depends on what you ran, not on what is committed.

The landing page claimed the product "works across your whole fleet of
agents and does not require you to rewrite your existing agent code", that
the SDK decides "before any network request leaves the process", and that
the proxy needs "no code changes".

Each overstates a real capability:

- Governance is per agent and per path. An agent launched outside a governed
  path is not governed at all — a measured bypass, not a theoretical one.
- The SDK gate fires on wrapped framework tool calls after its initializer
  runs; raw HTTP, subprocess spawns and file access are not intercepted.
- The proxy needs the process to honour HTTP_PROXY/HTTPS_PROXY and to trust
  its local root CA (trust-store install is macOS-only), speaks HTTP/1.1
  only, and by default decrypts only the built-in LLM hosts.
- eBPF is observe-only on Linux x86_64; it detects, it does not block.

Adds links to the core known-limitations page from both claim clusters.

Refs AAASM-5528
Replaces "a security checkpoint in front of every agent action" with the
governed-action scope, and footnotes the two enforcement matrix rows so the
ticks carry their assumptions: the proxy row now states the routing, CA
trust, transport and default-host limits, and the eBPF row states that it
detects rather than prevents and needs Linux x86_64 with OpenSSL.

The tick marks themselves are unchanged — both capabilities exist; only the
unstated preconditions were missing.

Refs AAASM-5528
The matrix row and differentiator 5 were wrong twice over, and this change
was written but never committed in the previous push.

Over-claimed: the chain is an unkeyed SHA-256 digest chain
(aa-core/src/audit.rs:10,713 — no hmac import exists anywhere in aa-core;
the only HMAC in the repo signs REST JWTs and webhooks). "HMAC-SHA256" is
wrong and "signed" is wrong — an unkeyed chain is tamper-evident, not a
signature. "Immutable" is wrong too: retention pruning deletes rows
(aa-gateway/src/storage/sqlite.rs:715, postgres.rs:854), and the DB mirror
carries no chain metadata at all (storage/audit_bridge.rs:10-12).

Under-claimed: the 🚧 marker presented this as planned Enterprise work. It
is not — AuditWriter::verify_chain (aa-gateway/src/audit.rs:142) ships today
as `aasm audit verify-chain` (aa-cli/src/commands/audit/mod.rs:14,31,44) in
the open-source build. Understating is as inaccurate as overstating, so the
capability is stated plainly rather than deleted.

Also corrects differentiator 2 ("kernel-level enforcement" -> visibility;
eBPF returns no verdict) and differentiator 3 (per-host CA -> per-host certs
from a local root CA, plus the launch/routing precondition).

Wording matches AAASM-5612 PR #134 on security-model.md so the hub does not
ship a page making this claim and a page declaring it contradicted.

Refs AAASM-5528, AAASM-5626
Three review findings, all in the hub landing and comparison pages:

- "trust-store installation is macOS-only" is false. install_linux is a
  complete implementation (aa-cli/src/commands/proxy/ca.rs:150-188): it
  copies to /usr/local/share/ca-certificates/ and runs
  update-ca-certificates. Only the AUTOMATIC install at proxy start is
  macOS-only; Windows is the unsupported platform. A Linux evaluator
  reading the old text would discard the most deployable enforcement
  mechanism in the product.
- eBPF is not "Linux x86_64 only" — there is no cfg(target_arch) in the
  eBPF crates; only the file-I/O kprobes are x86_64. The footnote also now
  names the loader-daemon requirement.
- The SDK does not block. aa-sdk-client has no in-tree caller that refuses
  (decision.rs:32-33) and query_policy is a voluntary call, so the SDK is
  advisory and the proxy is the enforcement point (ADR 0002).

Refs AAASM-5528
The round-2 fix replaced a false "macOS-only" claim with a false
"installed automatically" one. aa-proxy/src/lib.rs:64-69 attempts the
install at start, gated only on !ca.is_installed()?; it calls
tls/keychain.rs:16-42, which shells out to security add-trusted-cert and
needs admin privileges — macOS prompts, and because the call propagates
with ?, a refusal fails proxy startup.

"Installed automatically" reads as silent and unattended; a macOS CI runner
or non-admin developer following the docs gets a non-zero exit with a
keychain error the docs gave no reason for.

Refs AAASM-5528, AAASM-5638
@sonarqubecloud

sonarqubecloud Bot commented Aug 6, 2026

Copy link
Copy Markdown

@Chisanan232

Copy link
Copy Markdown
Contributor Author

Claude Code review complete — APPROVE-READY

Four independent review rounds. Recording the verdict here because GitHub does not permit the author account to approve its own pull request.

Requirements. This is the docs repo slice of AAASM-5528. It corrects comparison.md and README.md against the source-backed claim inventory committed in agent-assembly PR #1952 (merged, 8d29531a).

The blocker this round caught. At round 3 the four corrected comparison.md verdicts (inventory rows D9–D12) existed only as an uncommitted worktree modification while the PR head still shipped "Immutable audit log with tamper-evident signatures ✓ 🚧 (HMAC-SHA256)" and "audit log entries are signed with HMAC-SHA256, making post-hoc alteration detectable". The PR body and the inventory both asserted the work was done. Independently verified against the code: there is no HMAC anywhere in aa-core — the chain is unkeyed SHA-256 — and aa-gateway/src/storage/{sqlite,postgres}.rs delete audit rows on retention, so "immutable" is false too. Now committed and shipping the corrected matrix row and item text.

Claim accuracy. The macOS CA wording matches ADR 0033 §5.3 as corrected by AAASM-5638 (merged, 80e0e619) and agent-assembly #1952 — attempted automatically at proxy start, requires admin authorization, macOS prompts, a refusal fails proxy startup, Windows unsupported. Footnote integrity verified in built HTML: refs == defs {audit, ebpf, proxy}, 0 dangling, 0 orphan, 0 literal [^ leftovers, and the security-model.html#audit-log anchor resolves on main independent of merge order.

Merge ordering — this PR is a prerequisite, not a preference. docs PR #134 (AAASM-5612) rewrites security-model.md and glossary.md to state that no HMAC covers audit records. Until this PR lands, comparison.md and the concatenated print.html assert the opposite. Merging #134 first would ship a hub that self-contradicts on a disproved cryptographic claim.

Tests/CI. All 5 commits build; mdbook build, generate_hub_components.py --check, check_repo_names.py --check and generate_compatibility.py --check pass at every commit. Class scan of rendered output: 0 real defects. CI 3 passed / 0 failed, zero unresolved threads, no merge commits, no force-push.

Sonar/coverage: not configured for this repository; no analysable source touched.

Security: no secrets; no private-repo content — cloud and saas-infra are private and the diff was swept clean.

@Chisanan232
Chisanan232 merged commit aa07694 into main Aug 6, 2026
4 checks passed
@Chisanan232
Chisanan232 deleted the v0.0.1-rc.7/AAASM-5528/remove_absolute_claims branch August 6, 2026 04:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant