Skip to content

fix(brief): wait for the startup lockfile rescan; strip meta-commentary about the brief's own structure - #990

Merged
runyourempire merged 3 commits into
mainfrom
fix/brief-startup-staleness
Oct 18, 2026
Merged

runyourempire merged 3 commits into
mainfrom
fix/brief-startup-staleness

Conversation

@runyourempire

@runyourempire runyourempire commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

What

Two defects found in a live check of the rebuilt operator app (main 4d257b9).

1. The brief stated facts from the previous session's inventory

Brief #421 (claude-sonnet-5, 09:51:10 UTC) said "ammonia (4da/src-tauri) also newly flagged: bump to >= 4.2.4 (cargo add ammonia@4.2.4)". Cargo.lock has carried 4.2.4 since #980.

The actual ordering, from data/logs/fourda.log.2026-10-18:

UTC event
09:50:46 Preemption startup warm. The feed was computed from the stored inventory; ammonia 4.2.0 → 4.2.4 is still in it.
09:50:56 Frontend auto brief (trigger="auto", from the post-analysis effect). "Auto-trigger regenerating — the facts changed".
09:51:10 Brief #421 written.
09:51:40 Startup ACE scan rewrites user_dependencies.
09:53:49 Lockfile walk report.
09:54:16 "ACE context scan complete".

The startup ACE scan (initialize_ace_on_startup) waits for first light and a grace period, then walks every project, which took about 8 minutes here. Nothing told the readers of dependency facts that the stored inventory was stale until it finished. The same applies to the startup catch-up morning briefing (app_setup, 3 s after start), which builds its notification from the Preemption feed.

Fix: a real "inventory fresh" signal (inventory_freshness.rs, a tokio::sync::watch gate; no polling):

  • The gate can never stay shut (review B1). begin_startup_refresh() shuts it when the startup ACE scan is scheduled and returns a RefreshGuard, which is moved into the scan task. The gate reopens, exactly once, on whichever comes first:

    • the guard drops: the scan finished, failed, or its task panicked (release builds unwind);
    • a hard 15-minute deadline passes (a hung walk, such as an unreachable network share), with a warn log.

    Reopening rebuilds the Preemption feed (refresh_preemption_cache_after_cycle) and emits inventory-fresh.

  • Brief (generate_briefing_internal, which covers the IPC command and the background auto-brief job): while the gate is shut it builds no facts and returns {success:false, error:"inventory_refreshing", inventory_refreshing:true} at once (review S6). Nothing is taken from the model call's share of the UI's 120 s timeout.

    • The UI shows a neutral pending state (review S5): role=status, a spinner and secondary text, no Retry, not the red error banner or "generate failed". The text is "Your projects are being rescanned. The brief will be written as soon as the scan finishes." (13 locales).
    • The UI generates the brief once, on inventory-fresh. The listener asks once more right after it registers, because the event can fire before it exists, and removes itself after firing (review S4).
  • Background auto-brief job: when deferred, it logs and generates once the gate reopens, instead of waiting for the next scheduler pass.

  • Startup catch-up morning briefing: waits for the gate (bounded at the deadline + 1 min), otherwise skips.

  • Scheduler tick (review S3): the check runs only when a briefing is due, before the stale-results reset (so that wait isn't restarted every tick), and logs at debug. The briefing is not marked sent, so the next tick delivers it.

  • With no context directories, no scan is scheduled and nothing waits. Only the startup rescan of the current session is gated.

2. The model's own working leaked into the brief

The same brief ended a bullet with "— medium severity, listed under Also Open rather than critical, noting it here since it's NEW."

  • Prompt (one line, kept separate from fix(advisories): a Maven-only project is not ready; the Brief names what it did not check (W5a) #987's coverage wording): never write about the brief itself (where a line is listed, why it is or is not in a section) or use the FACTS block's labels (ACT NOW, ALSO OPEN, NEW, UNCHANGED). An ALSO OPEN fact goes only in the "Lower severity" line, even when it is NEW.
  • Post-filter briefing_meta::strip_meta_commentary, applied after the dev-tag filter. It is conservative (review B2), because a false positive deletes a finding:
    • a leading FACTS label ("Also open: braces (navcal), …") loses only the label, never the bullet;
    • a clause is removed only when it carries a marker that only ever describes the brief itself ("listed under", "noting it here", "also open", "facts block", "this brief", "since it's NEW", …) and names no package from the brief's facts;
    • markers match whole words ("also open" never matches inside "also openssl").

Tests

  • inventory_freshness:
    • a fresh gate never waits;
    • a pending refresh reopens on drop and runs its finish once;
    • a waiter wakes when the refresh finishes;
    • a guard dropped by a panic reopens the gate;
    • a hung scan releases the gate at the deadline, and the late drop is a no-op;
    • a_brief_requested_during_the_rescan_states_no_facts.
  • briefing_meta:
    • the exact live sentence becomes "- ammonia (4da/src-tauri) also newly flagged: bump to >= 4.2.4 (cargo add ammonia@4.2.4) — medium severity.";
    • real findings and headings are untouched;
    • a meta-only line is dropped;
    • "as listed in the FACTS block" is removed;
    • false positives:
      • "- Also open: braces (navcal), uuid (4da-ledger)" keeps both packages;
      • "- ammonia 4.2.3 (listed under RUSTSEC-2026-0373): bump to >= 4.2.4" is kept whole;
      • "New minor release", openssl / openapi-types / newrelic.
  • briefing_prompt_tests::prompt_forbids_talking_about_the_briefs_own_structure.
  • Frontend:
    • the briefing slice sets the pending state, not an error;
    • briefing-inventory-retry.test.ts: the listener is armed once, re-asks on registration, fires once, and removes itself.
  • Runs:
    • Rust: inventory_freshness briefing digest_commands monitoring gives 264 passed, 0 failed.
    • Frontend: the touched suites give 34 passed; tsc is clean.

Not done / notes

  • The Preemption tab can still show the startup-warm feed (or a restored snapshot) for the few minutes before the rescan finishes. It is now rebuilt as soon as the rescan finishes, where before it waited for the next engine cycle (09:58 in the log). No notification or brief is built from it in that window.
  • Not verified live in the app: that needs a cold start of the operator app with a changed lockfile. The ordering above is from the live log, and the gates are unit-tested.
  • briefing_prompt.rs is also claimed by fix/severity-parity, and fix(advisories): a Maven-only project is not ready; the Brief names what it did not check (W5a) #987 edits the same prompt. This PR adds one rule line next to the existing meta-commentary rule.

🤖 Generated with Claude Code

https://claude.ai/code/session_018hjX3pEMmC7CHfG41PiPj7

runyourempire and others added 2 commits October 18, 2026 22:42
…ry about the brief's own structure

Live 2026-10-18 on the rebuilt operator app: brief #421 (09:51:10) told the
user to bump ammonia to 4.2.4 although Cargo.lock had carried 4.2.4 since
#980. The frontend's auto brief ran at 09:50:56, while the startup ACE scan
was still re-reading the lockfiles (user_dependencies rewritten 09:51:40,
lockfile walk report 09:53:49, scan complete 09:54:16), so the facts came
from the previous session's inventory. The same brief also said "listed
under Also Open rather than critical, noting it here since it's NEW".

Inventory freshness (new `inventory_freshness`, a tokio watch gate):
- begin_refresh when the startup ACE scan is scheduled; finish_refresh when
  it ends, which also rebuilds the Preemption feed and emits
  `inventory-fresh`.
- generate_briefing_internal waits for it (bounded, 45 s, event-driven, no
  polling). Still pending: no facts are built; it returns
  `inventory_refreshing` and the UI says the projects are being rescanned
  and generates the brief on `inventory-fresh`. Covers the background
  auto-brief job too.
- The startup catch-up morning briefing waits up to 15 min and otherwise
  skips; the scheduler tick skips while the refresh runs.
- With no context directories nothing is scheduled and nothing waits.

Meta-commentary: one prompt rule (never write about the brief itself or the
FACTS labels; ALSO OPEN only in the "Lower severity" line) plus
`briefing_meta::strip_meta_commentary`, which removes such clauses and keeps
the finding. Pinned with the exact live sentence.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hjX3pEMmC7CHfG41PiPj7
…meta filter, neutral pending state

- B1: begin_startup_refresh returns a RefreshGuard the scan task owns; the
  gate reopens when it drops (scan done, failed, or a panic's unwind) or at
  a hard 15-minute deadline (a hung walk), whichever comes first, exactly
  once. Reopening rebuilds the Preemption feed and emits inventory-fresh.
  Tests: a guard dropped by a panic reopens the gate; a hung scan releases
  it at the deadline (and the late drop is a no-op).
- B2: strip_meta_commentary removes only a leading FACTS label ("Also open:")
  or a marker clause that names no package from the brief's facts; markers
  match whole words. False-positive tests: the "Also open:" bullet keeps its
  packages, "ammonia 4.2.3 (listed under RUSTSEC-…)" is kept whole, "New
  minor release", openssl / openapi-types / newrelic.
- S3: the scheduler checks freshness only when a briefing is due, before the
  stale-results reset, at debug.
- S4: the retry listener re-asks once right after registering (the event can
  fire before it exists) and removes itself; tested once-only/self-removing.
- S5: a deferred brief is a neutral `pending` state (role=status, spinner,
  no Retry), not the red error banner or generateFailed.
- S6: the brief is deferred at once instead of waiting 45 s, so the model
  call keeps the UI timeout's full budget.
- Nits: the background auto-brief retries on inventory refresh instead of
  waiting for the next pass; the app_setup comment matches the code.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hjX3pEMmC7CHfG41PiPj7
@runyourempire
runyourempire force-pushed the fix/brief-startup-staleness branch from a2e5eab to c750786 Compare October 18, 2026 12:54
….95 duration_suboptimal_units)

CI's stable clippy (1.95) rejects `Duration::from_secs(60)` and
`from_secs(15 * 60)` under -D warnings. Same values, written in minutes,
as the codebase already does elsewhere (auto_discover.rs RESULT_TTL).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hjX3pEMmC7CHfG41PiPj7
@runyourempire
runyourempire added this pull request to the merge queue Oct 18, 2026
Merged via the queue into main with commit 8850793 Oct 18, 2026
24 checks passed
@runyourempire
runyourempire deleted the fix/brief-startup-staleness branch October 18, 2026 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant