Skip to content

feat(archive): add local-save archive with subscription UI#1442

Draft
wpfleger96 wants to merge 7 commits into
mainfrom
duncan/local-save-archive
Draft

feat(archive): add local-save archive with subscription UI#1442
wpfleger96 wants to merge 7 commits into
mainfrom
duncan/local-save-archive

Conversation

@wpfleger96

@wpfleger96 wpfleger96 commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Add local SQLite archive for relay messages (P1–P3).

What this does

Adds a complete local-save archive feature: users can subscribe to specific channels or scopes, the app opens live relay subscriptions for each saved config, and matched events are persisted to a per-identity SQLite database in the Buzz nest (~/.buzz/archive/archive.db).

Backend (P1–P2)

Four Tauri commands: create_save_subscription, list_save_subscriptions, delete_save_subscription, archive_events.

Access control at every stage:

  • Subscribe-time: create_save_subscription runs a per-scope access probe against the relay (NIP-98 authed /query). Channel memberships are verified via kind 39002 + open-channel fallback. Event readability via ids filter. owner_p restricted to current identity's own pubkey.
  • Persist-time: archive_events re-verifies every candidate via scoped relay query ({ids, #h/#e, kinds}). The relay returning the event IS the proof of scope membership — no local tag re-derivation, so h-less channel events can persist when relay confirms them. Only events the relay returns are written.
  • Ephemeral path (owner_p, kind 24200 observer frames): relay never stores these, so local validation applies instead (sig/id + kind + #p + agent tag + frame=telemetry + author == agent).

Subscription kinds enforcement: both paths filter by the subscription's allowed kinds list before insert.

SQLite schema (archive/store.rs): three tables — archived_events (raw JSON + metadata), archived_event_scopes (many-to-many), save_subscriptions. WAL mode. GC: raw event rows are removed when their last scope row is deleted. Idempotent upserts.

Send-safety: rusqlite::Connection is !Send. All DB work is bracketed in sync plan_archive / commit_archive functions that drop the connection before any .await, matching the pattern in persona_events.rs. Pipeline internals live in archive/pipeline.rs to keep mod.rs under the 1000-line file-size gate.

Frontend (P3)

tauriArchive.ts — typed wrappers for all 4 commands. The SaveSubscription.kinds column is stored as a JSON text string in SQLite; the wrapper decodes it to number[] once at the boundary, failing closed on malformed JSON. ArchiveCandidate struct fields use verbatim Rust snake_case names (raw_event_json, matched_scope, scope_type, scope_value) — Tauri 2 only camelCases top-level command arg names, not nested struct fields. Emits a module-level subscription-change notification after create/delete.

archiveSyncManager.ts — always-on manager started at AppShell mount via useArchiveSync(). Opens one live relay subscription per saved config (channel_h#h, owner_p#p, referenced_e#e; all limit: 0 for live-only). Buffers matched events and flushes to archive_events in batches (≥25 events or 2s idle). Resubscribes automatically on subscription-change signal. Tears down cleanly on workspace switch, flushing buffered events.

LocalArchiveSettingsCard.tsx — settings UI at Settings → Local Archive (Archive icon, no feature gate, opt-in by construction). Lists active subscriptions with delete. Add flow for:

  • channel_h: channel picker from joined channels + kind preset checkboxes. Presets derived from CHANNEL_MESSAGE_EVENT_KINDS + KIND_STREAM_MESSAGE_DIFF (Messages & posts), CHANNEL_AUX_EVENT_KINDS (Reactions, edits & deletions), and CHANNEL_EVENT_KINDS (All channel activity). No raw numeric literals.
  • owner_p: fixed kinds [24200] (agent observer frames).
  • referenced_e: backend-only in v1 (no sane UI affordance for "paste an event id").

Tests

Rust (39 tests, src-tauri): persistent path (channel_h + referenced_e persist/drop, h-less events via scoped relay proof), kind mismatch drops on both paths, no-subscription drops, mixed batch exact counts, ephemeral path (valid/invalid frames, kind enforcement, missing tags, tampered events), store schema (idempotent upserts, GC, list scoping).

JS (20 new tests, archiveSyncManager.test.mjs): kinds decoding (valid, empty, non-array, wrong types, malformed JSON), preset kind arrays (exact saved arrays asserted, verified against constants — edits not misclassified as messages, forum kinds included), subscription-change notifier contract, manager lifecycle (filter shapes, flush on batch size, resubscribe on create/delete, flush on destroy).

E2E (click-path, local)

With just dev from the worktree:

  1. Settings → Local Archive → choose a channel → select event types → Save
  2. Chat in that channel
  3. sqlite3 ~/.buzz-dev/archive/archive.db 'SELECT id,kind FROM archived_events; SELECT scope_type,scope_value FROM archived_event_scopes;'

Files changed

desktop/src-tauri/src/archive/mod.rs        (Tauri commands + ephemeral validation + tests)
desktop/src-tauri/src/archive/pipeline.rs   (plan_archive / query_buckets / commit_archive)
desktop/src-tauri/src/archive/store.rs      (SQLite schema + CRUD)
desktop/src/shared/api/tauriArchive.ts      (API wrappers + kinds decoder + change notifier)
desktop/src/features/local-archive/archiveSyncManager.ts     (live forwarding manager)
desktop/src/features/local-archive/archiveSyncManager.test.mjs
desktop/src/features/local-archive/ui/LocalArchiveSettingsCard.tsx
desktop/src/features/settings/ui/SettingsPanels.tsx  (local-archive section)
desktop/src/features/settings/ui/SettingsView.tsx    (sidebar nav)
desktop/src/app/AppShell.tsx                         (useArchiveSync mount)

npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 and others added 4 commits July 1, 2026 16:30
SQLite store in the Buzz nest (~/.buzz/archive/archive.db) with
three tables:

- archived_events       — one raw event row per (identity, relay, id)
- archived_event_scopes — many-to-many scope membership rows
- save_subscriptions    — which scopes the user has subscribed to save

Four Tauri commands:

- archive_events        — batch-first; two proof paths (persistent vs ephemeral)
- create_save_subscription — per-scope access probe before persisting
- list_save_subscriptions  — scoped to current identity + relay
- delete_save_subscription — subscription row only; retention decoupled (v1)

Persistent scopes (channel_h, referenced_e) use a batched authed /query
to the relay as the access proof. Ephemeral scope (owner_p, kind 24200)
uses local validation: valid sig/id, kind 24200, #p == identity, agent
tag present, frame == telemetry, author == agent tag. Both paths are
fail-closed.

relay_url is canonicalized to WS-form via relay_ws_url_with_override
everywhere so the same relay never splits rows across wss:// and https://
variants.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Three IMPORTANT defects from Thufir pass 1:

1. Kinds not enforced at archive time: add get_subscription_kinds() to
   store.rs (returns Option<String> via OptionalExtension). Both paths
   now parse the kinds JSON and reject events whose kind is not in the
   subscription's allowed list.

2. Persistent proof path used unscoped relay filter: replace bare
   {ids}-only /query with {ids, #h/#e: scope_value, kinds: allowed_kinds}.
   The relay returning an event for this scoped filter IS proof of scope
   membership — no local tag re-derivation needed, which also correctly
   admits h-less events matched via StoredEvent.channel_id fallback.

3. Missing end-to-end tests: add plan_archive / commit_archive split
   (extracted from archive_events for Send-safety and testability),
   plus 14 new tests covering: persistent persists/drops, h-less event
   via scoped filter proof, kind mismatch drops (both paths), no-sub
   drops, mixed batch exact counts, ephemeral persists/drops,
   ephemeral kind-not-in-subscription drops.

Send-safety: rusqlite::Connection is !Send. archive_events now follows
the managed_agents/persona_events.rs pattern — DB work in scoped blocks
that drop the connection before any .await — so the Tauri command future
is Send.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
archive/mod.rs exceeded the 1000-line gate (1201 lines per the check).
Move plan_archive / query_buckets / commit_archive and their private
types (Parsed, Bucket, ArchivePlan, BucketWithResult) to a new
archive/pipeline.rs submodule.

mod.rs: 1200 → 915 lines (gate counts 916, well under 1000).
pipeline.rs: 312 lines.
Zero logic changes — pure structural move.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wpfleger96 wpfleger96 marked this pull request as draft July 1, 2026 20:59
npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 and others added 2 commits July 1, 2026 18:56
Add the frontend layer for local-save archive (P3):

- tauriArchive.ts: typed wrappers for all 4 Tauri commands. Decodes
  the SaveSubscription.kinds column (stored as a JSON text string in
  SQLite) into number[] once at the boundary. Emits a module-level
  subscription-change notification after create/delete so the sync
  manager reloads without manager instances threaded through props.
  Wire-shape: ArchiveCandidate fields use verbatim Rust names
  (raw_event_json, matched_scope, scope_type, scope_value) — Tauri 2
  only camelCases top-level command arg names, not nested struct fields.

- archiveSyncManager.ts: always-on manager started at AppShell mount
  via useArchiveSync(). Opens one live relay subscription per saved
  config (channel_h → #h, owner_p → #p, referenced_e → #e; all with
  limit: 0 for live-only). Buffers matched events and flushes to
  archive_events in batches (≥25 events or 2s idle). Resubscribes
  automatically on subscription-change signal; tears down cleanly on
  workspace switch, flushing any buffered events.

- archiveSyncManager.test.mjs: 20 tests covering kinds decoding (valid,
  empty, non-array, wrong types, malformed JSON), preset kind arrays
  (messages/aux/all derived from shared constants — exact saved arrays
  asserted), subscription-change notifier contract, manager lifecycle
  (filter shapes, flush on batch size, resubscribe on create/delete,
  flush on destroy).

- LocalArchiveSettingsCard.tsx: settings UI wired into SettingsPanels
  (new 'local-archive' section, Archive icon, no feature gate). Lists
  active subscriptions with delete; add flow for channel_h (channel
  picker from joined channels + kind presets) and owner_p (fixed kinds
  [24200]). Kind presets derived from CHANNEL_MESSAGE_EVENT_KINDS +
  KIND_STREAM_MESSAGE_DIFF, CHANNEL_AUX_EVENT_KINDS, and
  CHANNEL_EVENT_KINDS — never raw numeric literals.

- SettingsPanels.tsx + SettingsView.tsx: local-archive section wired
  into the sidebar nav under Personal alongside custom-emoji.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Record<string, unknown> was not assignable to RelaySubscriptionFilter,
which requires kinds: number[] and limit: number. Importing the type
and narrowing the return type catches this at tsc rather than at runtime.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wpfleger96 wpfleger96 changed the title feat(archive): add local-save archive backend feat(archive): add local-save archive with subscription UI Jul 1, 2026
Two fixes from Thufir pass 1 review:

1. Kinds-aware resubscribe (IMPORTANT): the active-sub key now encodes
   scope+kinds (scopeType:scopeValue:sorted-kinds). When the same scope
   is upserted via create_save_subscription with a different preset, the
   stale key is absent from the wanted set and gets torn down; the open
   loop immediately starts a new subscription with the updated filter.
   No second bookkeeping map — the single wanted-set diff handles both
   removals and kind changes uniformly.

2. Constructor DI (MINOR): ArchiveSyncManager now accepts optional deps
   (relayClient, listSaveSubscriptions, archiveEvents, onSubscriptionChange,
   flushBatchSize, flushIdleMs) with production defaults. The TestableArchiveSyncManager
   copy in the test file is gone — all 8 manager tests now exercise the real
   class via makeManager(relay, archive). Adds test:
   manager_resubscribes_with_new_filter_when_kinds_upserted.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant