Skip to content

[AAASM-5681] 🐛 (gateway): Declare and surface that shipped clients discard audit events - #363

Merged
Chisanan232 merged 12 commits into
mainfrom
v0.0.1/AAASM-5681/fix/node_audit_sink_signal
Aug 13, 2026
Merged

[AAASM-5681] 🐛 (gateway): Declare and surface that shipped clients discard audit events#363
Chisanan232 merged 12 commits into
mainfrom
v0.0.1/AAASM-5681/fix/node_audit_sink_signal

Conversation

@Chisanan232

@Chisanan232 Chisanan232 commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Description

Both gateway clients this SDK ships discard hook-layer audit events — record,
recordResult, scanPrompts — and the only signal was a one-shot note gated on
AA_DEBUG=1. A caller had to already suspect the problem in order to discover
it. Enforcement is unaffected, which is exactly why the drop is easy to miss:
denies still deny, and every audit call resolves successfully.

This PR does not build an audit sink. It makes the absence declared, surfaced
and tested
, and corrects the documentation that claimed otherwise.

The measured absence

Run against the shipped clients (through createClient, the same factory
initAssembly uses), with a stub placed at the native boundary — not
substituted for the code under test:

recordResult return: undefined   crossings: []
stderr writes with AA_DEBUG unset: 0
noop client stderr writes: 0

A payload of "SECRET-PAYLOAD" reached nothing, and with AA_DEBUG unset all
three audit methods on both clients produced zero signal.

The positive control, which is what makes that zero admissible

POSITIVE CONTROL crossings: ["queryPolicy:{\"agent_id\":\"\",\"action_type\":\"tool_call\",\"tool_name\":\"bash\"}"]

check() crosses the same boundary the audit methods do not. Without this,
crossings: [] would be indistinguishable from a probe that never ran — which is
how this defect survived its original tests, whose assertions were over mocks of
the thing under test.

Structure verified independently of the ticket: two shipped factories, and
init-assembly.ts:217/226 are the only non-test construction sites.

Claim vocabulary: Planned, not Unmeasured

ADR 0033 §6 reserves Unmeasured for "no control inspected this action or
payload; nothing is known about it."
That is not this case. The event is
constructed, passed, and dropped, and the measurement above says so exactly. A
measured zero is not "nothing is known", so Unmeasured would be wrong here —
noted explicitly because it is the kind of label that gets "corrected" later.

Planned is the fit: "Decided but not implemented — a ticket reference; no
capability claim."
The FFI record path is a separate new capability and
AAASM-5681 is the reference. The type documents it in those terms.

Observed for the hook layer holds only on the caller-supplied branch. That
is why AuditSinkDisposition is two-valued rather than a boolean:
"caller-supplied" is documented as the absence of a claim, not an
assurance
— this SDK cannot speak for a client it did not build. Same
discipline as asserted versus bound in the gateway's agentId attribution
(AAASM-5665 / #2016): the receiving side may record what was claimed, and may
record that it could not corroborate it, but may not round the two together.

Type of Change

  • ✨ New feature
  • 🐛 Bug fix
  • ♻️ Refactoring
  • 🍀 Performance improvement
  • 📝 Documentation update
  • 🔧 Configuration / CI change
  • ⬆️ Dependency upgrade
  • 🚀 Release

Breaking Changes

Does this PR introduce any breaking changes to public APIs or behaviour?

  • No
  • Yes (describe below)

Two additive fields, both optional, and one new behaviour that does not fail
init:

  • GatewayClient.auditSink? — optional so a caller's own client keeps compiling.
  • AssemblyContext.auditSink?optional deliberately. AssemblyContext is
    exported from this package's public entry (exports["."].types
    dist/types/index.d.ts) and is a plain interface with no brand or private
    member, so a consumer can construct one in a test fixture or custom
    integration. Requiring the field would be a compile break for published
    consumers, introduced by a diagnostic field, in a ticket about truthfulness.
    initAssembly always populates it.
  • initAssembly now writes one warning to stderr when the resolved client is
    known to discard. It does not fail init: the proxy and eBPF layers are
    unaffected and a caller may not need SDK-side audit.

The guarantee does not rest on either optional field. It rests on every
shipped client declaring its disposition
, which the test suite enforces.

Related Issues

  • Related Jira ticket: AAASM-5681
  • Related GitHub issues: N/A

Testing

  • Unit tests added / updated
  • Integration tests added / updated
  • Manual testing performed
  • No tests required (explain why)

pnpm test 659 passed / 2 skipped, exit 0 · pnpm lint exit 0 ·
pnpm typecheck exit 0 · pnpm build exit 0. pnpm 10.33.2, verified from
inside the repo.

tests/audit-sink-disposition.test.ts pins three things separately, because any
one of them alone passes while the defect is present:

  1. every shipped client declares a disposition;
  2. the declaration matches behaviour — a client saying "discarded" reaches
    nothing, measured against a boundary a positive control proves is reachable;
  3. initAssembly surfaces the drop on the default path with AA_DEBUG
    unset, on stderr and as context.auditSink.

It is structural, not behavioural-only — and that is now true rather than
asserted. The suite reads the module's own create*GatewayClient exports rather
than a hand-maintained literal, so a new factory is covered the moment it exists.
Review demonstrated the original claim was false: an exported, discarding,
undeclared createThirdGatewayClient passed at 9 passed (9). Reproduced here
as N7 — both exhaustiveness tests now fail and name the offending client.

Mutation evidence

Six mutations, run serially, each in one foreground command with a trap so the
restore is guaranteed, each verified by SHA-256 against the pre-mutation
baseline. Every one reddened its test; final residue check 0 in both files.

# What was broken Result
N1 no-op client drops its declaration 1 failed | 1 passed
N2 native client drops its declaration 1 failed | 1 passed
N3 declares "discarded" but actually crosses the boundary 1 failed
N4 no init-time warning 2 failed | 9 passed
N5 flag always reports "caller-supplied" 2 failed | 9 passed
N6 warn unconditionally 1 failed
N7 a third exported, discarding, undeclared client 2 failed | 9 passed
N8 same, named createGatewayClientFromEnv to dodge the name pattern 2 failed | 11 passed
N9 the no-op enforcement clause used in every mode (the round-2 defect) 1 failed | 12 passed
N10 the check-capable clause used in every mode (the round-1 defect) 1 failed | 12 passed

N1 and N2 are the discriminating pair. The parameterised test has one row per
shipped client, and each mutation reddens only its own row — neither client
passes on the other's coverage.

N3 is the mutation that makes the suite worth having. It pins declaration
against behaviour, so a client cannot claim "discarded" while leaking, nor
claim it records while dropping.

N6 kills the negative control. Without it, the warning assertions would still
pass on a build that warns unconditionally — a different untruth.

Documentation — the product-truth half

Three claims in this repo asserted that audit evidence is produced by default.
Measured against both shipped clients, none holds on the default path. Corrected
separately from the code change:

  • docs/01-introduction/index.md — "every governance-relevant action is recorded
    in an audit trail".
  • docs/02-quick-start/index.md — an allowed call's governance event "is
    recorded in the audit trail".
  • README.md — the LangChain callback layer "records denials and results for the
    audit trail".

Each now states that the event is emitted, that retention depends on the
gateway client, and how to tell which one you have.

Design note

The init-time warning plus programmatic flag is not a new pattern — it mirrors
warnAgentUnregistered, which pairs an unconditional stderr write with the
registered field for exactly this reason. Written to process.stderr rather
than console/a swappable logger so it cannot be silenced by log configuration,
and fired at init rather than per audit call so it cannot become steady-state
noise.

Checklist

  • Code follows project style guidelines (pnpm lint, pnpm typecheck)
  • Self-review of the diff completed
  • Documentation updated if behaviour changed
  • All CI checks passing
  • Commits are small and follow the Gitmoji convention
  • Commits are signed off (git commit -s) — DCO sign-off is optional/advisory here, not enforced

Deliberately not done

  • No FFI audit sink is built. That is a new capability needing its own
    scoping, not a bug fix. It is why the honest term is Planned.
  • go-sdk and python-sdk are untouched. Both share the defect per this
    ticket's own comments, and Python is worse — its hooks do not resolve at all,
    so nothing is emitted for allowed calls either. Reported rather than widened;
    filed separately.
  • The capability manifest rows that must carry the caller-supplied-client
    condition live in agent-assembly/governance/**, owned by a concurrent change.
  • AAASM-5683 — the LangChain wrapToolWithAssembly deny path — is separate
    and not folded in.

Round 2 — review fixes

1. The warning shipped a false claim on the default path. It said
"Enforcement is unaffected — allow/deny still applies" on every default init.
On auto / sdk-only / grpc-sidecar createClient returns the no-op client,
whose check() is allow-all — measured: {"denied":false,"pending":false} for a
tool named rm-rf-slash. It also landed immediately after the AAASM-1847 line
saying a policy DENY will not block a tool, contradicting it in consecutive
lines; and since that line only fires when a framework was patched, in a bare
consumer app mine was the only posture claim printed. Corrected in the warning,
gateway-governance.ts, client.ts and the introduction to say what is true:
the audit drop does not change the enforcement posture, which is not the same as
enforcement applying.

2. A fourth uncorrected retention claim. README.md"denies are still
recorded but not enforced"
— in the same LangChain section, seven lines below
the bullet I had just corrected. Missed because I enumerated three claims and
stopped rather than re-reading the section I had edited.

3. The structural claim. Made true rather than softened; see N7 above.

Finding 4 — why auditSink is optional when three siblings are required

detectedAdapters, unpatchedAdapters and registered were all added to
AssemblyContext as required, all diagnostic, same rationale. The
inconsistency is real and worth stating rather than leaving unremarked.

The reason for diverging: review verified against the published npm tarball
that AssemblyContext is publicly constructible, and ran the counterfactual
compile — exactly one differing line, TS2741: Property 'auditSink' is missing.
So the compile break is demonstrated, not hypothesised. The three siblings were
added before that was established; whether they were breaks was never measured.

Given a proven break against a published package, in a ticket whose subject is
truthfulness, the smaller harm is the inconsistency. The guarantee does not live
on this field in either case — it lives on shipped clients declaring, which N1,
N2, N3 and N7 enforce. If the maintainers prefer consistency with the three
siblings, making it required is a one-line change and I will take that direction.

Finding 5 — which remaining doc claims are hook-layer and which are runtime

Checked, and they split:

  • Hook-layer, corrected. docs/01-introduction/index.md offered "an audit
    trail without rewriting your agent code"
    as a default-path capability thirty
    lines below the corrected paragraph. It is the SDK's own claim and it is now
    narrowed.
  • Runtime, left alone. docs/02-quick-start/index.md and
    docs/05-configuration/index.md describe observe mode recording shadow audit
    events
    (aa audit list --dry-run-only). Those are written by the gateway —
    transform_for_observe_mode produces a ShadowEvent that flows into the
    gateway's record_audit — on a different channel from the SDK's hook-layer
    record / recordResult. They are enforcementMode semantics, not an SDK
    retention claim, so correcting them here would introduce an error rather than
    remove one.

Ticket status

ACs 3 and 4 — the documentation estate across all three SDKs, and the
capability-manifest rows carrying the caller-supplied condition — are not met
by this PR and are out of its scope. AAASM-5681 stays open after this merges.

Round 3 — review fixes

1. The corrected warning was false in napi-inprocess. Attempt 1 removed the
claim on three modes and created its inverse on the fourth: the clause
interpolated ${mode} but asserted the allow-all no-op client, and the warning
fires for both shipped clients. So the one mode that genuinely enforces was
told it could not. The clause now branches on CHECK_CAPABLE_MODE, and the
dangling `Supply ` + fragment is gone.

This claim had shipped wrong twice with nothing pinning it, so it now has a
discriminating pair: N9 forces the no-op clause everywhere and reddens the
napi-inprocess test; N10 forces the check-capable clause everywhere and
reddens the auto test. Neither could pass on the other's coverage. The
napi-inprocess test drives the real createClient routing through a mocked
native binding, following create-client-mode-routing.test.ts rather than
asserting on a hand-built client.

2. The "who this is for" list. Carving out only audit implied by contrast that
enforcement and redaction are default-path properties. Neither is — enforcement
only in a check-capable run, and redaction is applied by the runtime/proxy, with
this layer treating a redact verdict as allow under enforce. All three items
now state their real scope. Same shape as the previous round's finding, one
bullet over: when I correct one item of a list, the list is the unit.

3. Detection is now by shape, not by name. exportedClientFactoryNames tests
the returned object for check / record / recordResult / scanPrompts
instead of matching /^create[A-Za-z]*GatewayClient$/. Reproduced review's
escape as N8: createGatewayClientFromEnv passed at 11 passed (11) under
the name pattern and now fails, named in the message.

Commit-hygiene disclosure: fix 3 landed inside 0c67f548e, whose message
describes only fix 1 — both touch the same test file and I staged them together.
Two logical units in one commit, and the message under-describes it. Flagging
rather than leaving it for a reader to notice.

…s with audit events

`record` / `recordResult` / `scanPrompts` all return `Promise<void>`, so a
client that retains a hook-layer audit event and one that drops it are
indistinguishable to the caller. This type gives the difference a name.

Two-valued rather than boolean on purpose: the SDK can only speak for the
clients it builds, so `"caller-supplied"` is the absence of a claim, not an
assurance that events are retained.

Refs AAASM-5681
Measured on remote/main: with a stub at the native boundary, `check()`
crosses it and `record` / `recordResult` / `scanPrompts` cross nothing,
while the only signal was a one-shot note gated on AA_DEBUG=1 — so a
caller had to already suspect the drop to discover it.

Both shipped clients now declare `auditSink: "discarded"`. The field is
optional on the interface so a caller's own client keeps compiling, but
a client this package ships must never be silent about it.

Refs AAASM-5681
A caller configuring governance should not have to opt in to learn that
their audit trail does not exist. `initAssembly` now warns once on stderr
when the resolved client is known to discard, and reports `auditSink` on
the returned context as the programmatic counterpart — mirroring how
`registered` pairs with its init-time warning.

Does not fail init: the proxy and eBPF layers are unaffected and a caller
may not need SDK-side audit. A caller-supplied client is reported as
`"caller-supplied"` and does not warn, because the SDK cannot speak for it.

Refs AAASM-5681
Pins three things separately, because any one alone passes while the
defect is present: every shipped client declares a disposition; a client
declaring "discarded" reaches nothing, measured against a boundary a
positive control proves is reachable; and initAssembly surfaces the drop
with AA_DEBUG unset, on stderr and as context.auditSink.

The caller-supplied case is a negative control, so the warning assertions
cannot pass on a build that warns unconditionally.

Refs AAASM-5681
…ault

The introduction said governance-relevant actions are "recorded in an
audit trail", the quick-start said an allowed call's event "is recorded
in the audit trail", and the README said the LangChain callback layer
"records denials and results for the audit trail". Measured against both
shipped clients, none of the three holds on the default path.

Each now states that the event is emitted, that retention depends on the
gateway client, and how to tell which one you have.

Refs AAASM-5681
…ible

`AssemblyContext` is exported from this package's public entry
(`exports["."].types` → dist/types/index.d.ts) and is a plain interface,
so a consumer can construct one in a test fixture or custom integration.
A required field would be a compile break for published consumers,
introduced by a diagnostic field.

The guarantee does not weaken: it rests on every shipped client declaring
its disposition, not on contexts carrying it. `initAssembly` still always
populates the field.

Refs AAASM-5681
@codecov

codecov Bot commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

The new warning fired on every default init saying "Enforcement is
unaffected — allow/deny still applies". On auto / sdk-only / grpc-sidecar
`createClient` returns the no-op client, whose check() is allow-all, so
that is false exactly where the warning is loudest. It also landed
immediately after the AAASM-1847 line saying a policy DENY will NOT block
a tool — and that line only fires when a framework was patched, so in a
bare consumer app mine was the only posture claim printed.

The true statement is that the audit drop does not change the enforcement
posture, not that enforcement applies. Corrected in the warning, the
disposition type, the client comment and the introduction.

Refs AAASM-5681
README's "denies are still recorded but not enforced" sits in the same
LangChain section as the bullet I corrected, and asserts exactly the
retention documented as absent seven lines above it. Missed because I
enumerated three claims and stopped, rather than re-reading the section I
had just edited.

Also narrows the introduction's "who this is for" bullet, which offered
an audit trail as a default-path capability.

Refs AAASM-5681
The PR body claimed a future third shipped client that discards without
declaring would fail the suite. It would not: SHIPPED_CLIENTS was a
hand-maintained literal with no exhaustiveness mechanism, and review
demonstrated an exported, discarding, undeclared client sailing through
at 9 passed (9).

The suite now reads the module's own `create*GatewayClient` exports, so a
new factory is covered the moment it exists. Reproduced review's exact
counter-example: both new tests fail, naming the offending client.

Refs AAASM-5681
…i-inprocess

Attempt 1 removed the claim on three modes and created its inverse on the
fourth: the warning interpolated ${mode} into a clause asserting the
allow-all no-op client, but it fires for BOTH shipped clients, so in
napi-inprocess it told operators running the documented in-process
enforcement mode that their client cannot block — while createClient
returns the native client there, which denies against a denying runtime.

The clause now branches on CHECK_CAPABLE_MODE. Also removes the dangling
`Supply ` + fragment left by that attempt.

Two tests pin the text per mode, driving napi-inprocess through the mocked
native binding the way create-client-mode-routing.test.ts does. Both
inversions are now caught: forcing the no-op clause everywhere reddens the
napi test, forcing the check-capable clause everywhere reddens the auto
test. Nothing pinned this before, which is why it shipped wrong twice.

Refs AAASM-5681
… one

Carving out only audit implied by contrast that enforcement and redaction
ARE default-path properties. Neither is: auto / sdk-only / grpc-sidecar
route checks through the allow-all no-op client, and redaction is applied
by the runtime/proxy — under enforce this layer treats a redact verdict as
allow, per docs/05-configuration.

Same shape as the previous round's finding, one bullet over: correcting one
item of a list inside a line I was already editing, and leaving its
neighbours contradicted by the same page. The list was the unit to check.

Refs AAASM-5681
…s build

The cross-link added for the redaction bullet omitted `index.md`, so
Docusaurus resolved `../05-configuration/` to `/node-sdk/05-configuration/`
and failed the build with a broken link. Every other cross-link in docs/,
including one on this same page, uses `../<section>/index.md`.

Verified against the gate rather than by inspection: the site builds clean
locally, and reverting the link reproduces CI's exact error before the
build passes again.

Refs AAASM-5681
@sonarqubecloud

Copy link
Copy Markdown

@Chisanan232
Chisanan232 merged commit 9d3e9a8 into main Aug 13, 2026
27 checks passed
@Chisanan232
Chisanan232 deleted the v0.0.1/AAASM-5681/fix/node_audit_sink_signal branch August 13, 2026 06:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant