Fix the two real-history failure modes: action_id crash degrade + config-bound capability detection

docs/checks.json

@@ -1221,6 +1221,57 @@
       "requires_human_review_regardless_of_patch": false,
       "suggested_patch_kind": "manual"
     },
+    {
+      "autofix_safe": false,
+      "category": "verify",
+      "default_severity": "medium",
+      "description": "The PR changes the config file that binds a dynamic toolkit's authority; static analysis cannot diff the effective tool list.",
+      "docs_url": "https://github.com/ThreeMoonsLab/agents-shipgate/blob/main/docs/checks.md#ship-cap-config-binding-changed",
+      "dynamic_default": false,
+      "evidence_fields": [
+        "kind",
+        "provider",
+        "constructor",
+        "binding",
+        "config_path",
+        "changed_file",
+        "source_ref"
+      ],
+      "fires_when": "A VerificationContext and a base report are both present, the same factory site is config-bound on both sides, and the bound config file is in the PR's changed files.",
+      "floor_severity": null,
+      "id": "SHIP-CAP-CONFIG-BINDING-CHANGED",
+      "mvp_tier": "lifecycle",
+      "rationale": "A factory whose authority-bearing argument is bound from a config read hides its effective tool surface from static enumeration on both sides of the diff, so a config edit (e.g. adding \"refund\" to an actions list) produces no capability change in the generic diff. The config delta must be reviewed instead of passing invisibly.",
+      "recommendation": "Review the config delta for added capabilities, or declare an explicit local tool inventory for the toolkit so future changes are statically comparable.",
+      "requires_human_review": true,
+      "requires_human_review_regardless_of_patch": false,
+      "suggested_patch_kind": "manual"
+    },
+    {
+      "autofix_safe": false,
+      "category": "verify",
+      "default_severity": "high",
+      "description": "The PR removes the config binding from a dynamic toolkit factory present on both sides of the diff, so the toolkit may fall back to its everything-enabled defaults.",
+      "docs_url": "https://github.com/ThreeMoonsLab/agents-shipgate/blob/main/docs/checks.md#ship-cap-config-binding-removed",
+      "dynamic_default": false,
+      "evidence_fields": [
+        "kind",
+        "provider",
+        "constructor",
+        "binding",
+        "base_config_path",
+        "source_ref"
+      ],
+      "fires_when": "A VerificationContext and a base report are both present, the base bound the factory's authority-bearing argument to a config read, and the head constructs the same factory site with no binding at all. An unreadable (unknown) head binding never fires this check.",
+      "floor_severity": "high",
+      "id": "SHIP-CAP-CONFIG-BINDING-REMOVED",
+      "mvp_tier": "lifecycle",
+      "rationale": "A restriction-bearing config binding is the only statically-provable restraint on a toolkit whose tools load through a runtime factory. Removing the binding expands authority while looking like cleanup, and neither side's tools are statically enumerable, so the generic capability diff sees nothing. A coding agent must not self-approve that expansion; without this check the diff degrades to silent insufficient_evidence parity between base and head.",
+      "recommendation": "A human must review the removal; restore the config binding or an explicit literal allowlist, or declare an explicit local tool inventory for the toolkit so the mounted surface is statically enumerable.",
+      "requires_human_review": true,
+      "requires_human_review_regardless_of_patch": false,
+      "suggested_patch_kind": "manual"
+    },
     {
       "autofix_safe": false,
       "category": "codex_boundary",
@@ -2693,7 +2744,7 @@
       "id": "SHIP-SCOPE-TOOLKIT-UNBOUNDED",
       "mvp_tier": "core",
       "rationale": "A recognized toolkit constructor (e.g. stripe_agent_toolkit) called with no\nconfiguration allowlist mounts its full tool surface \u2014 refund/cancel/dispute \u2014\nwhich the static extractor cannot enumerate. The unbounded grant must be\nreviewed rather than pass silently into insufficient_evidence.",
-      "recommendation": "Pass an explicit configuration allowlist (resource:verb actions) to the toolkit constructor so the agent mounts only the tools it needs.",
+      "recommendation": "Pass an explicit configuration allowlist (resource:verb actions) to the toolkit constructor so the agent mounts only the tools it needs. If the toolkit must stay dynamic, declare an explicit local tool inventory for the tools it mounts so the surface is statically reviewable.",
       "requires_human_review": true,
       "requires_human_review_regardless_of_patch": true,
       "suggested_patch_kind": "manual"

docs/checks.md

@@ -154,6 +154,8 @@ baseline summary and do not fail CI.
 | `SHIP-VERIFY-AGENT-INSTRUCTIONS-WEAKENED` | medium | The PR edits agent-instruction trust roots and weakening cannot be statically disproven; routed to human review. |
 | `SHIP-VERIFY-TRIGGER-CATALOG-DRIFT` | medium | The PR changes the trigger catalog that decides when Shipgate runs; routed to human review to rule out gate evasion. |
 | `SHIP-VERIFY-CAPABILITY-SCOPE-BROADENED` | critical | The PR removes or broadens a dynamically-loaded toolkit's least-privilege configuration bound (e.g. a `stripe_agent_toolkit` allowlist), silently expanding the toolkit surface; blocks rather than degrading to insufficient_evidence. |
+| `SHIP-CAP-CONFIG-BINDING-REMOVED` | high | The PR removes the config binding from a dynamic toolkit factory present on both sides of the diff, so the toolkit may fall back to its everything-enabled defaults; routed to human review instead of silent insufficient_evidence parity. |
+| `SHIP-CAP-CONFIG-BINDING-CHANGED` | medium | The PR changes the config file that binds a dynamic toolkit's authority; static analysis cannot diff the effective tool list, so the config delta is routed to review. |
 
 ## Check Details
 
@@ -948,6 +950,30 @@ coding agent cannot self-approve the broadening; a human must re-apply a
 least-privilege configuration or explicitly approve the expanded surface. A
 narrowing (bound added or tightened) emits nothing.
 
+### SHIP-CAP-CONFIG-BINDING-REMOVED
+
+Config-bound capability detection: a dynamic toolkit factory whose
+authority-bearing constructor argument is *bound from a config read*
+(`json`/`yaml`/`toml` load, `os.environ`, pydantic settings) hides its
+effective tool surface from static enumeration on both sides of a verify
+diff. When the same factory site appears on both sides and the head removes
+the binding entirely, the toolkit may fall back to its everything-enabled
+defaults — authority expands while looking like cleanup, and the generic
+capability diff sees nothing. Fires `high` in verify mode and routes to
+human review instead of silent `insufficient_evidence` parity. Fail-safe by
+design: a head binding that is merely *unreadable* (`unknown`) never fires
+this check — it degrades to a source warning; a head that moves to a
+literal allowlist is the safe direction and emits nothing.
+
+### SHIP-CAP-CONFIG-BINDING-CHANGED
+
+Companion review item: the same factory site is config-bound on both sides
+of the diff and the PR changes the bound config file itself (e.g. adding
+`"refund"` to an actions list). Static analysis cannot diff the effective
+tool list, so the config delta is routed to review at `medium` with the
+inventory remedy. Fires only when the binding's literal config path matches
+a changed file — env- or settings-bound factories never guess a match.
+
 ### SHIP-MCP-ENV-SECRET-PASSTHROUGH
 
 Fires when a statically parsed MCP server passes through secret-like

docs/checks/scope.yaml

@@ -57,5 +57,7 @@ checks:
   - binding
   - source_ref
   recommendation: Pass an explicit configuration allowlist (resource:verb actions)
-    to the toolkit constructor so the agent mounts only the tools it needs.
+    to the toolkit constructor so the agent mounts only the tools it needs. If the
+    toolkit must stay dynamic, declare an explicit local tool inventory for the
+    tools it mounts so the surface is statically reviewable.
   requires_human_review_regardless_of_patch: true

docs/checks/verify.yaml

@@ -163,3 +163,56 @@ checks:
   recommendation: A human must review and re-apply a least-privilege toolkit
     configuration or explicitly approve the broadened scope; do not remove the
     allowlist to expand agent capability without review.
+- id: SHIP-CAP-CONFIG-BINDING-CHANGED
+  default_severity: medium
+  mvp_tier: lifecycle
+  requires_human_review: true
+  description: The PR changes the config file that binds a dynamic toolkit's
+    authority; static analysis cannot diff the effective tool list.
+  rationale: A factory whose authority-bearing argument is bound from a config
+    read hides its effective tool surface from static enumeration on both sides
+    of the diff, so a config edit (e.g. adding "refund" to an actions list)
+    produces no capability change in the generic diff. The config delta must be
+    reviewed instead of passing invisibly.
+  fires_when: A VerificationContext and a base report are both present, the same
+    factory site is config-bound on both sides, and the bound config file is in
+    the PR's changed files.
+  evidence_fields:
+  - kind
+  - provider
+  - constructor
+  - binding
+  - config_path
+  - changed_file
+  - source_ref
+  recommendation: Review the config delta for added capabilities, or declare an
+    explicit local tool inventory for the toolkit so future changes are
+    statically comparable.
+- id: SHIP-CAP-CONFIG-BINDING-REMOVED
+  default_severity: high
+  floor_severity: high
+  mvp_tier: lifecycle
+  requires_human_review: true
+  description: The PR removes the config binding from a dynamic toolkit factory
+    present on both sides of the diff, so the toolkit may fall back to its
+    everything-enabled defaults.
+  rationale: A restriction-bearing config binding is the only statically-provable
+    restraint on a toolkit whose tools load through a runtime factory. Removing
+    the binding expands authority while looking like cleanup, and neither side's
+    tools are statically enumerable, so the generic capability diff sees nothing.
+    A coding agent must not self-approve that expansion; without this check the
+    diff degrades to silent insufficient_evidence parity between base and head.
+  fires_when: A VerificationContext and a base report are both present, the base
+    bound the factory's authority-bearing argument to a config read, and the head
+    constructs the same factory site with no binding at all. An unreadable
+    (unknown) head binding never fires this check.
+  evidence_fields:
+  - kind
+  - provider
+  - constructor
+  - binding
+  - base_config_path
+  - source_ref
+  recommendation: A human must review the removal; restore the config binding or
+    an explicit literal allowlist, or declare an explicit local tool inventory
+    for the toolkit so the mounted surface is statically enumerable.

llms-full.txt

@@ -1665,6 +1665,8 @@ baseline summary and do not fail CI.
 | `SHIP-VERIFY-AGENT-INSTRUCTIONS-WEAKENED` | medium | The PR edits agent-instruction trust roots and weakening cannot be statically disproven; routed to human review. |
 | `SHIP-VERIFY-TRIGGER-CATALOG-DRIFT` | medium | The PR changes the trigger catalog that decides when Shipgate runs; routed to human review to rule out gate evasion. |
 | `SHIP-VERIFY-CAPABILITY-SCOPE-BROADENED` | critical | The PR removes or broadens a dynamically-loaded toolkit's least-privilege configuration bound (e.g. a `stripe_agent_toolkit` allowlist), silently expanding the toolkit surface; blocks rather than degrading to insufficient_evidence. |
+| `SHIP-CAP-CONFIG-BINDING-REMOVED` | high | The PR removes the config binding from a dynamic toolkit factory present on both sides of the diff, so the toolkit may fall back to its everything-enabled defaults; routed to human review instead of silent insufficient_evidence parity. |
+| `SHIP-CAP-CONFIG-BINDING-CHANGED` | medium | The PR changes the config file that binds a dynamic toolkit's authority; static analysis cannot diff the effective tool list, so the config delta is routed to review. |
 
 ## Check Details
 
@@ -2459,6 +2461,30 @@ coding agent cannot self-approve the broadening; a human must re-apply a
 least-privilege configuration or explicitly approve the expanded surface. A
 narrowing (bound added or tightened) emits nothing.
 
+### SHIP-CAP-CONFIG-BINDING-REMOVED
+
+Config-bound capability detection: a dynamic toolkit factory whose
+authority-bearing constructor argument is *bound from a config read*
+(`json`/`yaml`/`toml` load, `os.environ`, pydantic settings) hides its
+effective tool surface from static enumeration on both sides of a verify
+diff. When the same factory site appears on both sides and the head removes
+the binding entirely, the toolkit may fall back to its everything-enabled
+defaults — authority expands while looking like cleanup, and the generic
+capability diff sees nothing. Fires `high` in verify mode and routes to
+human review instead of silent `insufficient_evidence` parity. Fail-safe by
+design: a head binding that is merely *unreadable* (`unknown`) never fires
+this check — it degrades to a source warning; a head that moves to a
+literal allowlist is the safe direction and emits nothing.
+
+### SHIP-CAP-CONFIG-BINDING-CHANGED
+
+Companion review item: the same factory site is config-bound on both sides
+of the diff and the PR changes the bound config file itself (e.g. adding
+`"refund"` to an actions list). Static analysis cannot diff the effective
+tool list, so the config delta is routed to review at `medium` with the
+inventory remedy. Fires only when the binding's literal config path matches
+a changed file — env- or settings-bound factories never guess a match.
+
 ### SHIP-MCP-ENV-SECRET-PASSTHROUGH
 
 Fires when a statically parsed MCP server passes through secret-like

src/agents_shipgate/checks/_verify_common.py

@@ -25,11 +25,15 @@
 
 from __future__ import annotations
 
+from collections import defaultdict
+
 from agents_shipgate.core.context import ScanContext
+from agents_shipgate.core.domain import ToolkitScopeBound
 from agents_shipgate.core.globbing import glob_match
 from agents_shipgate.core.lenses.effective_policy import (
     build_effective_policy_snapshot,
 )
+from agents_shipgate.core.toolkit_scope import bound_from_policy_fact, policy_key_for
 from agents_shipgate.schemas.capability_change import EffectivePolicy
 from agents_shipgate.schemas.common import (
     SourceReference,
@@ -100,6 +104,67 @@ def base_effective_policy(context: ScanContext) -> EffectivePolicy | None:
     return getattr(reference, "effective_policy", None)
 
 
+def base_toolkit_bounds(context: ScanContext) -> dict[str, ToolkitScopeBound]:
+    """BASE-side toolkit scope bounds, decoded from the ``--diff-from``
+    reference report's carried ``toolkit_scope_bound`` policy facts.
+
+    Empty when there is no diff reference or the base carried no bounds —
+    callers degrade safely (no base, no comparison).
+    """
+    reference = context.diff_reference
+    facts = getattr(reference, "facts", None) if reference is not None else None
+    if facts is None:
+        return {}
+    bounds: dict[str, ToolkitScopeBound] = {}
+    for fact in facts.policies:
+        bound = bound_from_policy_fact(fact)
+        if bound is not None:
+            bounds[policy_key_for(bound)] = bound
+    return bounds
+
+
+def head_toolkit_bounds(context: ScanContext) -> dict[str, ToolkitScopeBound]:
+    """HEAD-side toolkit scope bounds from the live extraction."""
+    return {policy_key_for(bound): bound for bound in context.toolkit_bounds}
+
+
+def pair_toolkit_bounds(
+    base: dict[str, ToolkitScopeBound], head: dict[str, ToolkitScopeBound]
+) -> list[tuple[ToolkitScopeBound, ToolkitScopeBound]]:
+    """Pair base↔head bounds for comparison, keeping instances distinct.
+
+    First by exact ``provider:binding`` key, so multiple toolkit instances of a
+    provider do not collapse. Then a leftover fallback: for a provider with
+    exactly one unmatched bound on each side, pair them — this re-pairs a
+    *renamed* single toolkit so a rename-plus-weaken cannot slip through.
+    Ambiguous leftovers (more than one unmatched per side) are left unpaired
+    (fail safe).
+    """
+    pairs: list[tuple[ToolkitScopeBound, ToolkitScopeBound]] = []
+    matched_head: set[str] = set()
+    base_left: list[ToolkitScopeBound] = []
+    for key in sorted(base):
+        if key in head:
+            pairs.append((base[key], head[key]))
+            matched_head.add(key)
+        else:
+            base_left.append(base[key])
+    head_left = [head[key] for key in sorted(head) if key not in matched_head]
+
+    base_by_provider: dict[str, list[ToolkitScopeBound]] = defaultdict(list)
+    head_by_provider: dict[str, list[ToolkitScopeBound]] = defaultdict(list)
+    for bound in base_left:
+        base_by_provider[bound.provider].append(bound)
+    for bound in head_left:
+        head_by_provider[bound.provider].append(bound)
+    for provider in sorted(base_by_provider):
+        bl = base_by_provider[provider]
+        hl = head_by_provider.get(provider, [])
+        if len(bl) == 1 and len(hl) == 1:
+            pairs.append((bl[0], hl[0]))
+    return pairs
+
+
 def verify_finding(
     context: ScanContext,
     *,
@@ -138,8 +203,11 @@ def verify_finding(
 __all__ = [
     "SEVERITY_RANK",
     "base_effective_policy",
+    "base_toolkit_bounds",
     "changed_files",
     "head_effective_policy",
+    "head_toolkit_bounds",
+    "pair_toolkit_bounds",
     "touched",
     "verification_active",
     "verify_finding",

src/agents_shipgate/checks/registry.py

@@ -31,6 +31,7 @@
     verify_baseline_waiver,
     verify_capability_scope,
     verify_ci_gate,
+    verify_config_binding,
     verify_policy,
     verify_trigger_drift,
 )
@@ -77,6 +78,7 @@
     verify_agent_instructions.run,
     verify_trigger_drift.run,
     verify_capability_scope.run,
+    verify_config_binding.run,
 ]
 
 

src/agents_shipgate/checks/toolkit_bounds.py

@@ -33,6 +33,13 @@ def run(context: ScanContext) -> list[Finding]:
             # An explicit resource:verb allowlist was declared — least
             # privilege, exactly what we want. Nothing to flag.
             continue
+        if bound.config_binding in {"config", "unknown"}:
+            # A configuration IS passed — it lives in config (or is not
+            # statically readable), so "mounted without a scope bound" would
+            # be a false claim. The config-binding detections
+            # (SHIP-CAP-CONFIG-BINDING-*) and the unknown-binding source
+            # warning own those states.
+            continue
         findings.append(
             Finding(
                 check_id=CHECK_ID,
@@ -62,7 +69,11 @@ def run(context: ScanContext) -> list[Finding]:
                 recommendation=(
                     f"Pass an explicit `configuration` allowlist (resource:verb "
                     f"actions) to the {bound.constructor} constructor so the agent "
-                    "mounts only the tools it needs, not the full toolkit surface."
+                    "mounts only the tools it needs, not the full toolkit surface. "
+                    "If the toolkit must stay dynamic, declare an explicit local "
+                    "tool inventory for the tools it mounts at "
+                    f"{bound.source_ref}:{bound.source_line} so the surface is "
+                    "statically reviewable."
                 ),
             )
         )