Skip to content

[AAASM-5664] 🐛 (adapters): Report only successfully patched adapters as active - #354

Merged
Chisanan232 merged 10 commits into
mainfrom
v0.0.1-rc.7/AAASM-5664/fix/active_adapters
Aug 7, 2026
Merged

[AAASM-5664] 🐛 (adapters): Report only successfully patched adapters as active#354
Chisanan232 merged 10 commits into
mainfrom
v0.0.1-rc.7/AAASM-5664/fix/active_adapters

Conversation

@Chisanan232

@Chisanan232 Chisanan232 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Target

  • Task summary:

    AssemblyContext.activeAdapters reported a framework as active in the same run
    in which that framework's patch was warned to have failed. buildActiveAdapters
    (src/core/init-assembly.ts:540-555 on main) unioned the detection list with
    a set of patch-success flags; since every non-LangChain flag is itself gated on
    detection, the union could only ever reproduce the detection list. activeAdapters
    was therefore a pure detection list wearing a governance name.

    This PR derives the active list from the patch results alone and splits the
    outcome into three explicit fields, so a detected-but-ungoverned framework can
    never present as protected.

  • Key point change (optional):

    The defect was reproducible on main, in the repo's own test suite.
    tests/architecture/coverage-regression.test.ts:39 asserted
    activeAdapters === ["vercel-ai-sdk"] in a run whose stderr read:

    [agent-assembly] WARNING: the Vercel AI SDK ("ai") is installed as a frozen ES module namespace … Vercel AI SDK tool calls will NOT be governed by this SDK

    A green suite was pinning the defect as the contract, which is why it survived.

    Two source comments already claimed the behaviour that was never implemented,
    which is the clearest evidence the union was an oversight rather than a design:

    • patchDetectedVercelAiSdk — "this auto-detected path never surfaces
      vercelAiSdkPatched / an active adapter for an ungoverned Vercel install …
      It warns and is excluded from the active adapters."
    • tests/conformance/fail-open-conformance.test.ts — "excluded from
      activeAdapters (asserted in the unit / init suites)."

    Both are true as of this PR, and now actually asserted.

    As-is → to-be for a real ai install (the common case, frozen ESM):

    Field Before After
    activeAdapters ["vercel-ai-sdk"] (false) []
    detectedAdapters ["vercel-ai-sdk"]
    unpatchedAdapters ["vercel-ai-sdk"]

    Why three fields rather than filtering one. "The framework is not installed"
    and "the framework is installed but this SDK is not governing it" are different
    facts with different remedies. Collapsing them into an omission from
    activeAdapters would trade one misleading signal for another.

    The dead-code claim held only in part — worth a reviewer's attention. Four of
    the six patch flags (vercelAiSdkPatched, openAIAgentsPatched,
    langGraphPatched, mastraPatched) are strictly gated on frameworks.includes(...)
    and were genuinely dead. The two LangChain flags are not: registerLangChainHandler
    and wrapLangChainTools both bail only on !frameworks.includes("langchain-js") && !config.langchain,
    so an explicit config.langchain governs callbacks/tools whether or not
    @langchain/core resolves. langchain-js is the one id that can legitimately be
    active without being detected, and that half of the union is preserved deliberately.

Effecting Scope

  • Action Types:

    • ✨ Adding new something
      • 🟢 No breaking change
      • 🟠 Has breaking change
    • ✏️ Modifying existing something
      • 🟢 No breaking change
      • 🟠 Has breaking change
    • 🚮 Removing something
    • 🔧 Fixing bug
    • ♻️ Refactoring something
    • 🍀 Improving something (performance, code quality, security, etc.)
    • 🚀 Release
  • Scopes:

    • 🧩 SDK public API
    • 🪡 API client and transport
    • 🤖 Framework hooks
    • 🫀 Data model and types
    • ⛑️ Error handling
    • 🧪 Testing
      • 🧪 Unit testing
      • 🧪 Integration testing
      • 🧪 End-to-end testing
    • 🚀 Building
      • 🤖 CI/CD
      • 🔗 Dependencies
      • 📦 Project configurations
    • 📚 Documentation
  • Additional description:

    Behavioural change, called out deliberately. activeAdapters narrows to its
    documented meaning. A caller who read it as "which frameworks did you detect"
    will now see fewer entries — most visibly, a real ai install yields []. That
    narrowing is the fix, not a side effect: the old value was not true. Anyone who
    genuinely wanted detection should read the new detectedAdapters, so no
    information is lost, only correctly labelled.

    AssemblyContext gains two required readonly fields. Because the SDK is the only
    producer of that interface and consumers read it rather than construct it, this
    is additive for every realistic caller; a caller who hand-rolls an AssemblyContext
    for a test double would need to add them.

    No consumer inside src/ reads activeAdapters — the only references are the
    producer (src/core/init-assembly.ts) and the type declaration
    (src/types/assembly-context.ts), so nothing internal changes behaviour as a
    result of the new semantics.

    Versioned Docusaurus snapshots under website/versioned_docs/ are intentionally
    left untouched — they describe already-shipped releases.

Description

  • 🐛 (adapters) — replace buildActiveAdapters with buildAdapterStates, which
    derives active from patchedAdapterIds(patches) and returns detected / active /
    unpatched. initAssembly surfaces all three on the returned context. The two
    LangChain flags are preserved for the explicit-config.langchain path; the four
    detection-gated flags no longer contribute.
  • 🐛 (adapters) — document the three states on the public AssemblyContext,
    including why an omission from activeAdapters alone is not a sufficient signal.
  • ✅ (test) — new tests/active-adapters-truth.test.ts: two negative controls
    covering the two distinct routes to the defect (patch fails — frozen-ESM Vercel;
    patch skipped — LangGraph with no agentId), each paired with a positive control
    over the same framework and code path. The Vercel case asserts the omission against
    the un-silenceable stderr warning captured from the same run, so it measures the
    exact co-occurrence the ticket reported rather than the omission alone. A guard
    assertion fails the test if that warning ever stops being emitted, so the fixture
    cannot go vacuous silently.
  • ✅ (test) — retarget coverage-regression.test.ts off the defect it was pinning.
  • 📝 (docs) — correct docs/03-core-concepts, docs/04-guides, and
    docs/05-configuration. The guides printed ["vercel-ai-sdk"] as sample
    activeAdapters output, which is provably wrong for a real install — the same
    over-claim as the code defect, one output surface over.

Verification

The test was written failing first, and proven to bite by reverting the fix:

Stage Result
New test, before fix AssertionError: expected [ 'vercel-ai-sdk' ] to not include 'vercel-ai-sdk' — positive control already green
After fix 4 passed
Fix reverted to the old union (mutation) 3 failed / 5 passed — expected [ 'vercel-ai-sdk' ] to not include 'vercel-ai-sdk', expected [ 'langgraph-js' ] to not include 'langgraph-js', expected [ 'vercel-ai-sdk' ] to deeply equal []. Both positive controls stayed green, so the suite cannot pass by emptying the list.
Mutation restored verified by a positive grep for the real predicate, not merely absence of the marker

Gate exit codes on the final tree:

Gate Exit
pnpm test (645 passed, 2 skipped, 70 files) 0
pnpm typecheck 0
pnpm lint 0
prettier --check on touched files 1pre-existing

prettier --check flags src/core/init-assembly.ts and the three docs/* files.
All four fail identically on pristine remote/main (measured in a detached worktree
so the repo's own .prettierrc resolved), so this PR neither introduces nor fixes
them; pnpm lint is eslint-only and is clean. The two files this PR adds or rewrites
wholesale are prettier-clean.

Round 2 — independent review (CHANGES-REQUIRED) addressed

All four blocking findings reproduced independently before fixing. Head at review was
a55d243d1; the round-2 work is a8dba5506, 1fc22c707, d8e58358f, b306f1f15.

B1 — langchain-js reported active with zero governance. Fixed.
initAssembly patches a shallow copy of the caller's config, so config.langchain is
shared by reference only when the caller supplied it. registerLangChainHandler reached
its body on detection alone, and ensureLangChainCallbacks's ??= minted a fresh object
on the copy — the handler went into an array the caller can never read, then was
discarded. Reproduced:

Probe before after
@langchain/core installed, no langchain config active=["langchain-js"], unpatched=[], caller cfg undefined active=[], unpatched=["langchain-js"], + stderr warning

Both LangChain paths are now gated on caller reachability rather than presence, and the
detected-but-unwired case warns on stderr like every sibling framework already does. This
was the ticket's own defect surviving in the one branch that is not detection-gated.

B2 — new false capability claim. Fixed. The round-1 JSDoc "Treat presence here, and
only here, as the signal that in-process governance is in force"
was false three ways:
LangGraph/Mastra are NON_ENFORCING_MODULES lineage-only (AAASM-4830 — the SDK's own
runtime NOTE says so), the LangChain callback layer is audit-only (AAASM-4799), and every
enforcing path degrades to a non-blocking check under the allow-all no-op client. Replaced
with a mechanism statement plus the per-framework breakdown and the mode caveat, per ADR
0033 §6's prohibition on undifferentiated verbs. The LangGraph positive control that
pinned the over-claim now asserts the mechanism changed and states that it is not an
enforcement claim.

B3 / M1 — arrays not mutually consistent. Fixed (documented, not "corrected"). The
active ⊄ detected case is real and intended: an explicitly-configured adapter is active
without being detected. Verified active ∩ detected = ∅ with an inert ai install plus
explicit langchain config. The tautological "always a superset" sentence is gone; the real
invariant (unpatched === detected \ active) is stated, along with the
detected.filter(d => active.includes(d)) idiom that silently drops the one genuinely
wrapped adapter. A caution admonition now carries this to consumers, not just maintainers.

B4 — contradicting prose left standing. Fixed. docs/04-guides no longer says
installing a package is "all that is required to activate governance", nor that init
"patches its execution surface". Added a "What an applied patch actually does" column
separating lineage-only from denied-before-execution, with the check-capable-mode footnote.

M2 — addressed with B2. M3 — fixed: the explicit-config.langchain-without-install
case, the sole justification for keeping the two LangChain flags, had no test anywhere.
It now has one that asserts the wrapped tool's invoke rejects under a denying gateway
and the original implementation is never called — not that a string appears in a list.

M4 — decision recorded, not silence. website/versionChannels.json pins
lastVersion: 0.0.1-rc.6, so the default-served page is the rc.6 snapshot with the old
prose. Decision: do not edit the frozen snapshot. It correctly documents rc.6, where
activeAdapters really did mean "detected"; editing it would make the archive lie about
the code it describes. This ticket's fixVersion is rc.7, and cutting rc.7 snapshots the
corrected tree and moves lastVersion forward, closing the gap. The exposure is the
window between this merge and the rc.7 cut; a follow-up ticket tracks adding an
admonition to the rc.6 snapshot if that window turns out to be long.

Round-2 mutation testing (both restored, verified by positive grep):

Mutation Result
Undo the B1 reachability gate 1 failed — expected [ 'langchain-js' ] to not include 'langchain-js'
Delete the two LangChain flags 2 failed — the new explicit-config control and the pre-existing detected case. Before this round the same mutation failed only the detected case.

Round-2 gate exit codes: pnpm test 0 (647 passed / 2 skipped) · pnpm typecheck
0 · pnpm lint 0 (it caught a genuinely dead frameworks parameter, now removed)
· prettier --check on the files this PR authors 0.

Prettier baseline re-measured the reviewer's way — git archive of remote/main into a
tree with its own .prettierrc, repo-wide, both directions: 95 at base, 95 at HEAD,
zero new, zero gone.
The round-1 wording "prettier fails on 4 files" understated it:
four of the files this PR touches were already failing; the repo-wide pre-existing count
is 95 and prettier gates nothing here (no lefthook/husky/pre-commit hook, no CI step).

Round 3 — the one open review clause, plus a sibling instance it surfaced

Round-2 review verified all four blockers and three of four majors resolved, leaving one
open clause. Commits 12be7ec6f, 12232ad77.

Open finding — *Observed* unearned for langgraph-js/mastra. Fixed. ADR 0033 §6
Observed requires "a durable event attributed to the action". These patches emit none:
wrapCompiledGraph is runWithAgentId(...) wrapping the original call, and
runWithAgentId is AsyncLocalStorage.run. Re-derived with a discriminating probe:

0   src/hooks/langgraph.ts
0   src/hooks/mastra.ts
13  src/hooks/ai-sdk.ts         <- positive control
26  src/hooks/openai-agents.ts  <- positive control

No §6 term describes what these adapters do, so none is used. They are stated plainly —
they bind the agent id so that other layers' evidence is attributed to it — with the
in-process call itself called §6 Unmeasured, and the ADR's own scoping caveat that
this says nothing about the proxy/eBPF layers.

Sibling instance found while checking the other labels to the same standard. The
degradation clause claimed an enforcing path "degrades to Evaluated" under the
allow-all no-op client. §6 Evaluated requires a decision record from the control
plane; createNoopGatewayClient.check returns a bare literal, so no decision is
produced and the call is Unmeasured, not evaluated-then-ignored. Corrected.

Applying the standard to the remaining labels also tightened the LangChain callback
claim: it reaches Observed only when the client persists what it records, and
record/recordResult are deliberate no-ops on both the no-op client and the
napi-inprocess client (AAASM-4847) — so it requires a caller-supplied client, which
the docs now say. Denied before execution is retained for the three paths that earn
it; verified that createWrappedExecute throws before executeOriginal() is invoked
and openai-agents returns before originalInvoke.call.

This mattered because docs/06-api-reference generates the public API reference from
src/index.ts TSDoc — an unearned term would have shipped as product documentation.

Both review nits taken. The ensureLangChain* ??= footgun is now documented where
a future caller would hit it, and the M4 follow-up is tracked as
AAASM-5676.

Round-3 gate exit codes: pnpm test 0 (647 passed / 2 skipped) · pnpm typecheck
0 · pnpm lint 0 · prettier --check on PR-authored files 0; repo-wide
set-difference against remote/main empty both directions (95 → 95).

Out of scope and filed separately: AAASM-5663 (README non-functional at both
entrypoints) and AAASM-5665 (no audit evidence for denied calls, no agent identity
on the check path — touches a wire contract, owner decision).

buildActiveAdapters unioned the detection list with a set of patch-success
flags. Every non-LangChain flag is itself gated on detection, so the union
could only ever reproduce the detection list: a framework found on disk was
reported in activeAdapters even when its patch had just warned it was inert.
The frozen-ESM Vercel shape (AAASM-4842) hit this on every real install —
activeAdapters said "vercel-ai-sdk" in the same run stderr said Vercel tool
calls would NOT be governed.

Derive active from the patch results alone, and split the outcome into three
explicit fields rather than one filtered array — "not installed" and
"installed but ungoverned" are different facts with different remedies, and
collapsing them into an omission from activeAdapters would trade one
misleading signal for another.

The two LangChain flags are preserved: an explicit config.langchain wraps
callbacks/tools whether or not @langchain/core resolves, so langchain-js is
the one id that can legitimately be active without being detected.

Refs AAASM-5664, blocks AAASM-5529
Pins the AAASM-5664 contract over the artifact a caller actually reads
(ctx.activeAdapters / detectedAdapters / unpatchedAdapters) rather than an
internal patch flag.

Two negative controls, because there are two distinct routes to reporting a
detected-but-ungoverned framework as active: a patch that fails (the frozen-ESM
Vercel shape) and a patch that is skipped for a missing prerequisite (LangGraph
without an agentId). The Vercel case asserts the omission against the
un-silenceable stderr warning captured from the SAME run, so the test measures
the exact co-occurrence the ticket reported rather than the omission alone.

Each is paired with a positive control over the same framework and code path —
without them an always-empty activeAdapters would satisfy every negative
assertion, which is the vacuous-evidence shape Epic AAASM-5526 exists to
eliminate.

Refs AAASM-5664
…efect

This guard resolved `ai` without mocking it, so the real frozen ES module
namespace rejected the governed tool factory and the patch warned-and-failed —
yet the case asserted activeAdapters === ["vercel-ai-sdk"]. It was pinning the
AAASM-5664 defect as if it were the contract, which is why the bug survived a
green suite.

Assert the truthful split instead: detected and unpatched, not active. The
shutdown-iterates-adapters half of the guard is unchanged, since shutdown
still walks every registered adapter regardless of patch outcome.

Refs AAASM-5664
The guides printed `ctx.activeAdapters` with `["vercel-ai-sdk"]` as an example
output. That is now provably wrong for a real install: `ai` ships as a frozen
ES module namespace, so its patch cannot take effect and it is reported
ungoverned. Documenting it as active is the same over-claim as the code defect,
one output surface over — AAASM-5529 requires that a failed adapter cannot
present as protected anywhere.

Adds the two new context fields to the published AssemblyContext shape and
states the detection/governance distinction where each doc introduces it.
Versioned docs snapshots are left alone; they describe shipped releases.

Refs AAASM-5664
@codecov

codecov Bot commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Review finding B1. `initAssembly` patches a SHALLOW COPY of the caller's config,
so `config.langchain` is shared by reference only when the caller supplied it.
`registerLangChainHandler` reached its body on detection alone, and
`ensureLangChainCallbacks`'s `??=` then minted a fresh object on the copy — the
handler was pushed into an array the caller can never read and was discarded
when `initAssembly` returned. `AssemblyCallbackHandler`'s constructor only calls
`super()`, so no global registration rescued it.

`patches.langChainHandler` was truthy regardless, so `langchain-js` was reported
active with zero governance: the ticket's own defect surviving in the one branch
that is not detection-gated. Reproduced — with `@langchain/core` installed and no
`langchain` config, `active = ["langchain-js"]`, `unpatched = []`, and the
caller's `config.langchain` is still `undefined` after init. On a LangGraph app
that reads `active = ["langchain-js","langgraph-js"]` with `unpatched = []`,
i.e. an affirmative all-clear for a run governing nothing.

Gate both LangChain paths on caller reachability instead of presence, and warn
on stderr like every sibling auto-detected framework already does. The tools path
returned an empty list here anyway, but only after pointlessly importing the
optional LangChain adapter; its now-unused `frameworks` parameter is dropped.

Refs AAASM-5664
…nce"

Review findings B2, B3, M1 and M2. The previous round asserted "Treat presence
here, and only here, as the signal that in-process governance is in force for
that framework". That is false three ways and was newly introduced: LangGraph
and Mastra are NON_ENFORCING_MODULES lineage-only (AAASM-4830, and the SDK's own
runtime NOTE says so), the LangChain callback layer is audit-only (AAASM-4799),
and every enforcing path degrades to a non-blocking check under the allow-all
no-op client. Fixing a false claim by writing a stronger false claim one layer in
is the exact shape this Epic exists to remove.

ADR 0033 §6 forbids undifferentiated verbs like "protects"/"enforces". State the
mechanism instead — the patch was applied and is reachable — and give the
per-framework breakdown plus the mode caveat, so a reader can tell Observed from
Denied-before-execution rather than being handed one crisp word that is wrong for
half the adapters.

Also replaces the `detectedAdapters` "always a superset" sentence. It was a
tautology that read like a guarantee of `detected ⊇ active`, which does not hold:
an explicitly-configured adapter is active without being detected, so with an
inert `ai` install plus explicit langchain config the intersection is empty. The
invariants that do hold are stated, along with the filter idiom that silently
drops the one genuinely-wrapped adapter.

Refs AAASM-5664
…able

Review findings B4, B2, M1. The guides said "With @openai/agents, ai,
@langchain/langgraph, or @mastra/core installed, this is all that is required to
activate governance for it" — three lines above the previous round's own comment
saying `ai` is reported ungoverned, inside the same fenced block. The sentence
above it ("detects it and patches its execution surface") had the same problem.

Installing a package is not sufficient, and an applied patch buys different
things per framework, so the Status column alone could not carry the truth. Adds
a "What an applied patch actually does" column separating lineage-tagging-only
(LangGraph, Mastra) from denied-before-execution (Vercel, OpenAI Agents, and
LangChain's wrapped tools but not its audit-only callbacks), with the
check-capable-mode caveat as a footnote.

Core concepts gains the matching mechanism-not-enforcement framing and a caution
admonition for the `active ⊄ detected` case, which the previous round documented
for maintainers in a code comment but not for consumers anywhere.

Frozen `website/versioned_docs/` snapshots are deliberately untouched — see the
M4 decision recorded on the ticket.

Refs AAASM-5664
… paths

Review findings B1, B2 and M3.

Adds a negative control for the B1 shape: `@langchain/core` detected with no
`langchain` config must not be active. It guards its own guard by asserting the
caller's `config.langchain` is still `undefined` after init, so the test proves
the handler really is unreachable rather than trusting the shallow-copy reasoning
to keep holding.

Adds the positive control for the decision the whole PR rests on — explicit
`config.langchain` with `@langchain/core` absent — which had no test anywhere in
the repo. Mutation testing showed deleting the two LangChain flags failed only a
test that installs the package, i.e. the load-bearing case was unguarded. This
one asserts over the artifact that decides: the wrapped tool's `invoke` rejects
under a denying gateway and the original implementation is never called, not
merely that a string appears in a list.

The LangGraph positive control previously pinned membership alone while the
surrounding prose called it governance, locking in the B2 over-claim. It now
asserts the mechanism actually changed (the StateGraph prototype was replaced)
and documents that this is not an enforcement claim.

Refs AAASM-5664
Round-2 review. `langgraph-js`/`mastra` were labelled *Observed*, but §6 requires
"a durable event attributed to the action" and these patches emit none:
`wrapCompiledGraph` is `runWithAgentId(...)` around the original call and
`runWithAgentId` is `AsyncLocalStorage.run`. A discriminating probe counts 0
gateway/record/check references in langgraph.ts and mastra.ts against 13 in
ai-sdk.ts and 26 in openai-agents.ts. No §6 term fits what they do, so none is
used: they are described plainly as binding the agent id so OTHER layers'
evidence is attributed to it, with the in-process call stated as *Unmeasured*.

Checking the sibling labels to the same standard turned up a second unearned
term the review did not flag. The degradation clause said an enforcing path
"degrades to *Evaluated*" under the allow-all no-op client, but §6 *Evaluated*
requires a decision record from the control plane and `createNoopGatewayClient`
returns a bare literal — no decision is produced, so the call is *Unmeasured*,
not evaluated-then-ignored.

Same standard applied to the LangChain callback layer: it reaches *Observed*
only if the client persists what it records, and `record`/`recordResult` are
deliberate no-ops on both the no-op client and the napi-inprocess client
(AAASM-4847), so that needs a caller-supplied client and now says so.
*Denied before execution* is kept for the three paths that earn it — the refusal
provably precedes the effect.

This text ships to the public API reference (docs/06-api-reference generates from
src/index.ts), so an unearned term would have been published product documentation.

Refs AAASM-5664
…tgun

Round-2 review nit. Both `ensure*` helpers run against the shallow copy
`initAssembly` makes of the caller's config, so their `??=` writes into a
discarded object whenever the caller did not supply `langchain`. That is what
made `activeAdapters` credit a handler nobody could reach (AAASM-5664).

The reachability guards that fix it live in the two callers, and nothing in the
type system connects them to the helpers — a new call site, or a well-meaning
simplification of a guard, silently re-arms it. Document the trap where someone
would actually hit it.

Refs AAASM-5664
@sonarqubecloud

sonarqubecloud Bot commented Aug 7, 2026

Copy link
Copy Markdown

@Chisanan232
Chisanan232 merged commit a9bc52b into main Aug 7, 2026
27 checks passed
@Chisanan232
Chisanan232 deleted the v0.0.1-rc.7/AAASM-5664/fix/active_adapters branch August 7, 2026 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant