Skip to content

feat(agent): stream analysis and add incident count windows - #349

Merged
hoalongnatsu merged 1 commit into
mainfrom
feature/update
Aug 26, 2026
Merged

feat(agent): stream analysis and add incident count windows#349
hoalongnatsu merged 1 commit into
mainfrom
feature/update

Conversation

@hoalongnatsu

Copy link
Copy Markdown
Member

What does this PR do?

Makes on-demand incident analysis observable while it runs and makes incident
counts describe a deliberate time window instead of an ever-growing all-time
total.

  • Streams model and tool lifecycle events from the analyze ReAct loop over SSE,
    then returns the persisted analysis ID on the terminal event.
  • Shows human progress such as "I'm checking metrics to understand what
    happened..." while preserving stable tool identifiers for dispatch and audit.
  • Adds an admin-configurable incident count window: 24h / 7d / 30d / 90d /
    all
    , defaulting to 7d.
  • Couples every authoritative count response with its effective count_window,
    so UI labels and numbers come from the same request.
  • Consolidates the repeated admin gateway-secret guard and hardens the admin UI:
    wider shared previews, wrapping for long IDs/payloads, inert table rows with
    explicit actions, and one-time recovery from stale lazy-loaded chunks.

Why?

Analysis previously had a long silent wait: operators clicked Run analysis
and saw no evidence that the model was working, which tools it used, or whether
closing the dialog would lose the run. Tool names were implementation identifiers
such as query_metrics, not readable activity.

Incident counts had the opposite problem: they counted the complete history, so
the number only grew and stopped describing current operational load. The UI also
had no contract tying a count to the window used to compute it.

The supporting UI fixes close problems found while exercising these flows:
long metric/pattern IDs forced cards and overlays wider than the viewport,
Decision rows behaved differently from other dense tables, and a deployment
could leave an open browser requesting an obsolete hashed route chunk.

How to test

Automated

cd versus-incident

gofmt -l pkg cmd              # no output
go vet ./...
go test ./...

cd ui
npx tsc -b
npm run lint
npm test                      # 704 tests / 54 files
npm run build

Manual: streamed analysis

  1. Start the OSS or enterprise stack and enable the analyze agent.
  2. Open Incidents, use the eye action on a row, and click Run analysis.
  3. Confirm the modal shows human progress (model text and activity such as
    "I'm checking related logs...") rather than raw snake-case tool names.
  4. Expand a completed tool step and confirm arguments/output wrap without a
    horizontal scrollbar.
  5. Close with Run in background while it is running. Confirm the analysis
    still finishes and is saved.
  6. Let a run finish in the modal. Confirm the parsed AnalysisCard replaces the
    transcript and View full analysis opens the persisted record.
  7. Simulate a stream/proxy failure and confirm the non-streaming fallback reports
    its own error if it also fails.

Manual: incident count window

  1. Open Settings → System (?tab=tuning).
  2. Set Incident count window to 24h, 7d, 30d, 90d, and all; save each value.
  3. Confirm the top bar, Now KPIs, and Incidents status/origin tabs refresh from
    the same windowed count response.
  4. Confirm Now and Incidents show counts over <window> in the left subtitle and
    do not duplicate the window beside the right-side AI/Webhook badge.
  5. Search for an incident older than the configured window. It must remain
    discoverable; search responses intentionally report count_window: "all".
  6. Restart the process and confirm the selected window persists.

Manual: UI hardening

  1. On Decisions, Incidents, Analyses, Patterns, Services, metrics/traces, and
    alert-fatigue lists, confirm a plain row click is inert and the eye button
    opens the shared preview panel.
  2. Confirm j/k still moves the active row and Enter opens its preview.
  3. Open a preview containing a long pattern ID, metric name, explanation, or
    sample. Content must wrap without horizontal page/panel scrolling.
  4. Keep the SPA open across a new deployment, then navigate to a cold detail
    route. A stale dynamic-import chunk should trigger one reload and recover;
    repeated failure must show the error boundary rather than reload-loop.

Type of change

  • Bug fix (non-breaking)
  • New feature (non-breaking)
  • Breaking change (config / API / default behavior) — displayed incident
    counts now default to the last 7 days instead of all time; APIs are
    additive and stored incidents are unchanged
  • Documentation only
  • Refactor (no functional change) — shared admin gateway guard
  • CI / build / chore

Checklist

  • go test ./... passes locally
  • go vet ./... is clean
  • Code is gofmt'd
  • Added or updated tests for the change
  • Updated user-facing docs under src/ if behavior changes — follow-up
    needed for count-window settings and streamed analysis UX
  • Updated ROADMAP.md if this closes a roadmap item — N/A; on-demand
    analysis and read-only investigation tools were already marked complete
  • No secrets, tokens, or webhook URLs introduced in source / YAML
  • No new third-party dependencies

Detailed changes

Streaming analyze

  • Added core.AnalyzeEvent, observer context plumbing, and lifecycle kinds for
    run, model, and tool activity.
  • Uses Eino chat/tool callbacks so every framework-dispatched tool is traced.
    Streamed model output is drained from Eino's per-handler stream copy and
    ordered before the terminal event.
  • Added POST /api/admin/incidents/:id/analyze/stream with SSE framing,
    X-Accel-Buffering: no, detached run lifetime, persistence, and terminal
    success/failure semantics.
  • Non-terminal observer events are non-blocking so a slow browser cannot stall
    the model. The terminal event gets bounded delivery so failure status and the
    persisted analysis ID cannot be silently dropped under backpressure.
  • Tool/model output is capped to the same persisted boundary. Plain/truncated
    tool responses are wrapped safely when stored as JSON.
  • The React client uses fetch + ReadableStream (POST and gateway-secret
    headers are required), batches events per animation frame, and falls back to
    the synchronous endpoint when streaming is unavailable.

Human tool activity

  • Added optional core.AnalyzeToolDisplayer and robust fallback humanization for
    custom/legacy tools. Existing AnalyzeTool implementations remain source
    compatible.
  • Added explicit activity names to all nine built-in read-only tools.
  • Added additive tool_display to live events while keeping persisted
    ToolCallTrace.Name and model dispatch names stable.
  • The UI renders first-person running, completed, and failed activity sentences
    and still accepts older events that contain only tool.

Incident count windows

  • Added storage-backed CountSettings with recognized windows and a 7-day
    default.
  • Added optional storage.IncidentWindowCounter plus implementations for
    Postgres, file, and memory. Postgres uses one indexed COUNT/FILTER query
    bounded by created_at; zero time means unbounded.
  • Added dedicated CountSettingsController with authenticated GET/PUT endpoints.
  • Count and list responses now include additive count_window; explicit search
    stays unbounded and returns all.
  • Settings are resolved once at the handler boundary. Backends without the
    optional counter apply the cutoff locally rather than returning unbounded
    values under a bounded label.
  • Settings UI lives under System while retaining the existing tuning URL
    value so old deep links continue working.

Shared security and UI consistency

  • Replaced eight byte-identical per-controller auth functions with one
    adminGatewayGuard, preserving enterprise-auth short-circuiting and the
    constant-time gateway-secret comparison.
  • Decisions detect/shadow/spike rows now match other dense tables: rows are
    inert, explicit links remain links, the eye opens a preview, and Enter opens
    the same preview.
  • Added one-time session-scoped recovery for stale dynamic-import chunks without
    reload loops.
  • Added shared card/overlay wrapping rules for long unbroken IDs, payloads,
    explanations, tool data, and evidence.
  • Product width scale is CSS-defined: max-w-lg = 720px list previews,
    max-w-xl = 960px analysis modal, max-w-2xl = 1200px wide dialogs.

Compatibility and intentional behavior

  • No enterprise-specific symbols or logic were added to OSS. The private module
    continues to compile and test against this change.
  • Analyze tool display names and all response fields are additive.
  • The count default changes only displayed/count API totals; incident storage,
    list paging, and historical rows remain unchanged.
  • Incident search is intentionally unbounded so an explicit ID/text lookup can
    find older incidents even when dashboards use a recent window.
  • Framework-level tool_error remains defensive wiring. Normal tool failures
    are returned to ReAct as structured results so the model can adapt instead of
    aborting the run.

Verification

  • OSS: gofmt, go vet ./..., go test ./...PASS.
  • Enterprise compatibility: go build ./..., go vet ./..., go test ./...
    PASS.
  • UI: TypeScript, ESLint, production Vite build — PASS.
  • Vitest: 704 tests across 54 files — PASS.
  • git diff --checkPASS.
  • Postgres window-count path was separately exercised against an isolated test
    database; memory and file parity run in the normal suite.

Known limitations

  • User-facing docs under src/ have not yet been updated for the new count
    setting and live-analysis transcript.
  • A backend implementing neither the bounded pager nor window counter falls back
    to materializing its provider-bounded history before applying the count
    cutoff. All shipping OSS storage providers implement the optimized path.
  • The count setting is read from storage per count/list request. This is one blob
    read, not an N+1 query; a short process-local TTL may be worthwhile if polling
    volume grows.

@hoalongnatsu
hoalongnatsu merged commit 8e531da into main Aug 26, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant