[runyourempire]

What

Adversarial audit 2026-10-12, finding L2. Every Preemption refresh (about every 10 minutes) asked the cloud model again about the same handful of signal chains and LLM-judged items. The live ai_usage table (read-only) shows it:

day	adversarial calls	cost
10-11	334	$2.49
10-12	190	$1.41
10-13	198	$1.43
10-14	114	$0.81

That is against a $1.50/day cap. Once the cap was spent, three things followed:

• the user's own "Assess with AI" was refused with "Assessment failed — try again";
• the pass failed open, so chains the model had rejected came back;
• the fast cache-miss feed never applied verdicts at all.

1. Verdict cache (adversarial_cache.rs)

• Key: SHA-256 of exactly what the model judges. That is the item's title, kind, urgency, explanation, deps and projects, plus PROMPT_VERSION, the system prompt text, the provider and the model id.
  • The item id is excluded because chain ids are minted fresh on every refresh.
  • The user-context summary is excluded because its topic order churns between cycles; the TTL re-judges against a changed profile.
  • Changed material gets a new verdict, and a prompt edit or model switch re-judges.
• Stored: the full verdict (should_surface, confidence, grounded explanation, arguments, reasoning chain) and judged_at.
• TTL: 7 days, pruned on every save and capped at 512 entries.
• Persistence: one JSON map in the existing kv_store table. It survives restarts and is shared across processes on the same DB, with no schema migration. A bump would force fourda and fourda-engine to be rebuilt together before either opens the DB, and would collide with parallel branches on the version number. A lost race between two writers costs one re-judge, never an unjudged item shown.

2. Fail-closed (adversarial_batch.rs, with the decision table in its module doc)

• A cached rejection stays suppressed while it is valid, whatever the budget or the provider is doing. The Critical/High safety floor is unchanged.
• Never-judged items when the model can't judge them (background budget share spent, provider error, unreadable answer): unvetted heuristic items are held back. In Preemption those are exactly the signal chains (ConfidenceProvenance::Heuristic, chain-*).
  • Decision: doctrine principle 5, "never show intelligence the system can't stand behind". Chains are the one item class whose only backing is a weighted formula, and the model is the standard 4DA applies to them when one is configured.
  • What still surfaces: OSV-verified items (bypass, as before), LLM-assessed llm-* items (already vetted by a model upstream), corroborated Critical/High items (safety floor) and the deterministic upgrade plan (appended after the filter). So the surface is never emptied of grounded items; the test unjudged_heuristics_are_held_back_but_grounded_items_still_surface pins this.
  • When no deliberation is configured at all (a Basic-tier model or no key), behaviour is unchanged: the deterministic escalation gate runs and items pass. Holding chains back there would make them unreachable for a user who has chosen no cloud model, which is a product decision beyond L2. Flagged for the lead.
• A budget stop or provider failure is not retried for every item in the batch.
• The cache-miss fast feed now applies stored verdicts too (apply_stored_verdicts), so a rejected chain cannot flash there while the deliberated refresh runs.

3. Background passes no longer starve user calls

• BACKGROUND_BUDGET_SHARE_PERCENT = 60. Every background pass stops at 60% of either daily cap, leaving the rest for calls a user is waiting on:
  • LLMClient::budget_refused applies the share only to BACKGROUND_PURPOSES = adversarial, rerank_judge, ingest_judge, verdict_drain, channel_render_auto. It does this in complete, complete_structured and the streaming call.
  • User-initiated purposes keep the whole cap: Assess with AI, search synthesis, a channel render the user asks for (channel_render), connection tests and translation. The test background_share_gates_only_background_purposes pins the classification.
  • The scheduled stale-channel pass is background (round 3). channel_render::auto_render_stale_channels runs after every monitoring cycle (app_setup.rs) and on the first-run "render all". It now uses its own purpose, channel_render_auto. Opening a channel, creating one and "render now" keep channel_render and the full cap.
  • The scheduled monitoring_brief deliberately keeps the full cap. It is the brief the user reads, one call per window, and starving it would hide the day's findings behind background reranking.
  • local_judge::budget_blocks, called only by the background judge passes, uses the share. This is a three-line change outside G6's local_judge.rs hunks.
• Across processes (chosen over a per-process reserve): background_llm_budget_reached() re-seeds the counters from the persisted ai_usage ledger, at most once a minute. That is one indexed SUM, applied with fetch_max, so nothing recorded in-process is lost. The reserve therefore holds across the GUI and fourda-engine on the same database; before, each process only saw the other's spend from its startup seed.
• Localized refusal: a refused metered call keeps the English sentence for raw callers, "Daily AI budget reached — resets at HH:MM local time (in X h Y min)". It now ends with a machine token, [budget_reset:HH:MM:MINUTES], covering both the cap and the preflight cost wall.
  • The preflight wording is now "This call would exceed today's AI budget …".
  • parseBudgetReset() and budgetResetVars() in error-messages.ts read the token. The UI renders blindspots.ai.budgetReached in Blind Spots and errors:errorMsg.budgetReached in translateError, with {time} and {in} placeholders. {in} is a localized "in 3 hours" from Intl.RelativeTimeFormat.
  • Both keys are translated into all 13 locales, following the repo convention since fix(advisories): day-one advisory truth: sync without a restart, honest readiness, real dates #961.
  • The wait reads "in N hours" only within 5 minutes of a whole hour; otherwise it gives exact minutes ("in 89 minutes"), never "in 1 hour" for 1 h 29 min.
  • The token never renders verbatim (round 3):
    • digest_commands::provider_error_message passes a budget refusal through whole, so the "connect" and "model" checks can never re-classify it (test a_budget_refusal_passes_through_whole).
    • Every frontend site that shows a raw error from an LLM-backed command now renders a budget refusal localized through budgetRefusalMessage(): the briefing slice (shown in UnifiedAppBar), the graph detail summary, channel renders and the model eval. Any other error keeps the backend's own message.
    • Blind Spots, the item summary and the connection test already went through the budget-aware translateError.
    • I grepped every frontend caller of generate_ai_briefing, generate_item_summary, render_channel_now, run_model_eval, assess_blind_spots_with_ai, test_llm_connection, trigger_translation and auto_render_all_channels. synthesize_search has no frontend caller; translation and auto-render failures are silent.

Review nits taken

• The fast feed with no readable database fails closed: unvetted chains are held back (settle_stored, test fast_feed_without_a_database_holds_back_unvetted_chains).
• build_user_message carries a comment: changing the template requires bumping PROMPT_VERSION, because the template is part of what the cache key stands for.
• Preemption's comment now describes the fail-closed behaviour. The warn! "dropped ALL … possible LLM failure" is gone: drops are routine once the share is spent, so an info line now names the cause (filtered_by_verdict, held_back_unjudged, model_unavailable). filter_batch returns the BatchOutcome with its stats.

Expected cost effect

A refresh now calls the model only for material it has not judged in the last 7 days. A chain's explanation changes only when its source count or day span changes, so that is a few calls per chain per day at most, instead of one per chain per 10-minute cycle. Even before the cache is warm, background passes together cannot spend more than 60% of the cap.

Evidence

• Rust, round-3 scoped run (llm::tests, digest_commands, channel_render, adversarial, state_llm_limits, local_judge): 142 passed, 0 failed. Earlier scoped run (adversarial, state_llm_limits, llm::tests, local_judge, preemption, llm_judg*): 293 passed, 0 failed. The new tests in adversarial_cache_tests.rs use an injected model that counts calls:
  • cache_hit_makes_no_model_call (same material, new chain id: 0 calls);
  • changed_material_is_judged_again;
  • key_covers_model_and_prompt_but_not_the_item_id;
  • rejected_chain_stays_suppressed_when_the_budget_is_spent, also through a provider error;
  • unjudged_heuristics_are_held_back_but_grounded_items_still_surface;
  • verdicts_expire_after_the_ttl;
  • fast_feed_applies_stored_verdicts_without_a_model;
  • fast_feed_without_a_database_holds_back_unvetted_chains;
  • verdicts_persist_across_loads, on a real DB;
  • corrupt_store_starts_empty.
• State and LLM tests:
  • background_share_reserves_headroom_for_user_calls;
  • background_share_follows_the_live_counters, where the background stops while is_llm_limit_reached is still false;
  • budget_message_names_the_local_reset_time, which checks the token;
  • background_share_gates_only_background_purposes.
• Frontend:
  • error-messages: the token is parsed; other failures are null; the wait is localized (en/de); translateError never says "try again";
  • Blind Spots renders blindspots.ai.budgetReached.
  • budgetRefusalMessage replaces only budget refusals;
  • the rounding rule.
  • Round 3: 409 tests pass across error-messages, Blind Spots, the store slices and GraphDetailPanel, and tsc is clean.
• Not live-verified: this needs the model and a running app to watch the cache warm. The live kv_store has no adversarial_verdicts_v1 yet. ai_usage grouped by task_type per day is the metric to watch after merge.

Notes

• G6 (fix(g6): hidden windows stop rendering; judge leaves VRAM between cycles; DB budget #967) overlap: llm.rs is also edited by G6, in complete() around the local-judge block and at about line 1203 and later. My hunks are in the three budget gates, a new budget_refused/BACKGROUND_PURPOSES block, format_limit_error, the preflight message and a test. None overlap textually, but the gate at complete() sits a few lines above G6's local-judge hunk, so whichever lands second should rebase cleanly or with a trivial context fix. The local_judge.rs change is three lines in budget_blocks, outside G6's hunks.
• adversarial.rs is past the 700-line warning and under the 1000 error. The new logic lives in adversarial_batch.rs and adversarial_cache.rs.

🤖 Generated with Claude Code

https://claude.ai/code/session_018hjX3pEMmC7CHfG41PiPj7